2024-05-17T21:43:00.739 INFO:root:teuthology version: 0.0.1.dev276+g2127e87 2024-05-17T21:43:00.739 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --archive /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/orig.config.yaml 2024-05-17T21:43:00.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T21:43:00.868 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748 branch: wip-pdonnell-testing-20240517.012458-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7711748' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240517.012458-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected - object missing on disk - Invalid tag char sha1: 931809a9e5d295bd61984693cac44fe131ded4bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 931809a9e5d295bd61984693cac44fe131ded4bc subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.012458-debug sha1: 931809a9e5d295bd61984693cac44fe131ded4bc owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1967 sha1: 931809a9e5d295bd61984693cac44fe131ded4bc sleep_before_teardown: 0 subset: 59/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.012458-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 931809a9e5d295bd61984693cac44fe131ded4bc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHIJSEf2xUkpW3eHjsfvK7YtmfqgX3883DVbalL/SIYwu5MUc7EdJkXnn4TILu6qHtNPxjItao2dspgdczSK9zY= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJkjvXa8Xt41YJV9XAJXpQtbtdT07UftajZBF+cxC3bkLKNg48U6RGlNQAxXUm9pZrO4lEkA9eZOTGvMqxSJ7qc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: install-fix teuthology_sha1: 2127e87e3419439de550d2abbd7ed2aee2cdfb0e timestamp: 2024-05-17_18:30:44 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-17T21:43:00.868 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa; will attempt to use it 2024-05-17T21:43:00.869 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa/tasks 2024-05-17T21:43:00.869 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-17T21:43:00.870 INFO:teuthology.task.internal:Checking packages... 2024-05-17T21:43:00.891 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '931809a9e5d295bd61984693cac44fe131ded4bc' 2024-05-17T21:43:00.891 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-17T21:43:00.891 INFO:teuthology.packaging:ref: None 2024-05-17T21:43:00.891 INFO:teuthology.packaging:tag: None 2024-05-17T21:43:00.891 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:43:00.891 INFO:teuthology.packaging:sha1: 931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:43:00.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240517.012458-debug 2024-05-17T21:43:01.108 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3794-g931809a9-1jammy 2024-05-17T21:43:01.110 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-17T21:43:01.117 INFO:teuthology.task.internal:no buildpackages task found 2024-05-17T21:43:01.117 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-17T21:43:01.156 INFO:teuthology.task.internal:Saving configuration 2024-05-17T21:43:01.171 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-17T21:43:01.184 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-17T21:43:01.207 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:37:09.282120', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDdV5RX7czYUkSYQ61j5hmTgfxuE14AsjXr4V1z/yOCx'} 2024-05-17T21:43:01.222 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-17 21:37:09.280559', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKagJbveTB+uuq7xGlEQkYuh+x+JRNvxrzfzjpN9LZwB'} 2024-05-17T21:43:01.223 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-17T21:43:01.229 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-17T21:43:01.229 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-17T21:43:01.229 INFO:teuthology.run_tasks:Running task console_log... 2024-05-17T21:43:01.299 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3d237ee9e0>, signals=[15]) 2024-05-17T21:43:01.299 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-17T21:43:01.306 INFO:teuthology.task.internal:Opening connections... 2024-05-17T21:43:01.306 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:43:01.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:43:01.389 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:43:01.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:43:01.472 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-17T21:43:01.478 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-05-17T21:43:01.483 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-05-17T21:43:01.484 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-05-17T21:43:01.531 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Ubuntu" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="22.04" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_CODENAME=jammy 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:ID=ubuntu 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE=debian 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-17T21:43:01.532 INFO:teuthology.orchestra.run.smithi083.stdout:UBUNTU_CODENAME=jammy 2024-05-17T21:43:01.533 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-05-17T21:43:01.668 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-05-17T21:43:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-05-17T21:43:01.674 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="22.04" 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=jammy 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2024-05-17T21:43:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2024-05-17T21:43:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-17T21:43:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-17T21:43:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-17T21:43:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-17T21:43:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=jammy 2024-05-17T21:43:01.721 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-05-17T21:43:01.741 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-17T21:43:01.750 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-17T21:43:01.766 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-17T21:43:01.767 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-05-17T21:43:01.769 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-05-17T21:43:01.774 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-17T21:43:01.792 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-17T21:43:01.792 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-05-17T21:43:01.817 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-05-17T21:43:01.936 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-17T21:43:01.984 INFO:teuthology.run_tasks:Running task kernel... 2024-05-17T21:43:01.998 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-17T21:43:01.998 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-17T21:43:01.998 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T21:43:01.999 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-17T21:43:02.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:43:02.003 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-05-17T21:43:02.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:43:02.004 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-05-17T21:43:02.053 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-56-generic 2024-05-17T21:43:02.053 INFO:teuthology.orchestra.run.smithi155.stdout:5.15.0-56-generic 2024-05-17T21:43:02.053 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.15.0-56-generic 2024-05-17T21:43:02.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-05-17T21:43:02.054 INFO:teuthology.task.kernel:Running kernel on smithi155: 5.15.0-56-generic 2024-05-17T21:43:02.054 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-05-17T21:43:02.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-05-17T21:43:02.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-05-17T21:43:02.335 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-17T21:43:02.362 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:43:02.362 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-17T21:43:02.364 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-17T21:43:02.454 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-17T21:43:02.462 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:43:02.543 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-17T21:43:02.604 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,467 kB] 2024-05-17T21:43:02.622 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,467 kB] 2024-05-17T21:43:02.677 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [631 kB] 2024-05-17T21:43:02.732 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [470 kB] 2024-05-17T21:43:02.732 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [252 kB] 2024-05-17T21:43:02.735 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-17T21:43:02.735 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-17T21:43:02.735 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,866 kB] 2024-05-17T21:43:02.746 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [470 kB] 2024-05-17T21:43:02.754 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [252 kB] 2024-05-17T21:43:02.758 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,678 kB] 2024-05-17T21:43:02.760 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-17T21:43:02.760 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,866 kB] 2024-05-17T21:43:02.770 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [317 kB] 2024-05-17T21:43:02.775 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-17T21:43:02.775 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [603 kB] 2024-05-17T21:43:02.788 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-05-17T21:43:02.806 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [164 kB] 2024-05-17T21:43:02.807 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-17T21:43:02.816 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-17T21:43:02.817 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [317 kB] 2024-05-17T21:43:02.823 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-17T21:43:02.823 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-05-17T21:43:02.836 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [603 kB] 2024-05-17T21:43:02.843 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [164 kB] 2024-05-17T21:43:02.849 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-17T21:43:02.849 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [311 kB] 2024-05-17T21:43:02.849 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-17T21:43:02.849 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-17T21:43:02.850 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-17T21:43:02.855 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-17T21:43:02.856 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-17T21:43:02.858 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,923 kB] 2024-05-17T21:43:02.883 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-17T21:43:02.884 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-17T21:43:02.884 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-17T21:43:02.884 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-17T21:43:02.885 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-17T21:43:02.889 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [327 kB] 2024-05-17T21:43:02.893 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-17T21:43:02.893 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-05-17T21:43:02.906 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-17T21:43:02.906 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [702 kB] 2024-05-17T21:43:02.912 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [246 kB] 2024-05-17T21:43:02.914 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-17T21:43:02.914 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-17T21:43:02.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-17T21:43:02.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-17T21:43:02.915 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-17T21:43:02.936 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-17T21:43:02.937 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-17T21:43:02.938 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-17T21:43:02.938 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-17T21:43:02.938 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-17T21:43:02.938 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-17T21:43:02.939 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-17T21:43:02.939 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-17T21:43:03.044 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [631 kB] 2024-05-17T21:43:03.245 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,678 kB] 2024-05-17T21:43:03.398 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [311 kB] 2024-05-17T21:43:03.412 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-17T21:43:03.413 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-17T21:43:03.414 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,923 kB] 2024-05-17T21:43:03.477 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [327 kB] 2024-05-17T21:43:03.484 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-17T21:43:03.649 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [702 kB] 2024-05-17T21:43:04.226 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-05-17T21:43:04.337 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [246 kB] 2024-05-17T21:43:04.357 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-17T21:43:04.359 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-17T21:43:04.359 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-17T21:43:04.362 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-17T21:43:04.363 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-17T21:43:04.363 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-17T21:43:04.367 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-17T21:43:04.371 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-17T21:43:04.537 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-17T21:43:04.619 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-17T21:43:04.701 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-17T21:43:04.755 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-17T21:43:04.806 INFO:teuthology.orchestra.run.smithi155.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-17T21:43:08.510 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 13.7 MB in 2s (7,091 kB/s) 2024-05-17T21:43:08.980 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 13.7 MB in 3s (5,355 kB/s) 2024-05-17T21:43:09.810 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:43:09.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-17T21:43:09.885 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:43:10.087 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-05-17T21:43:10.087 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-05-17T21:43:10.258 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:43:10.258 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:43:10.258 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:43:10.259 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:43:10.259 INFO:teuthology.orchestra.run.smithi083.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-17T21:43:10.259 INFO:teuthology.orchestra.run.smithi083.stdout: python2.7-minimal 2024-05-17T21:43:10.259 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:43:10.260 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-05-17T21:43:10.260 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-17T21:43:10.260 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-generic linux-image-5.15.0-107-generic 2024-05-17T21:43:10.260 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-5.15.0-107-generic linux-modules-extra-5.15.0-107-generic 2024-05-17T21:43:10.262 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-05-17T21:43:10.262 INFO:teuthology.orchestra.run.smithi083.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-17T21:43:10.262 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-05-17T21:43:10.262 INFO:teuthology.orchestra.run.smithi083.stdout: thermald 2024-05-17T21:43:10.284 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:43:10.302 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-17T21:43:10.305 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-05-17T21:43:10.305 INFO:teuthology.orchestra.run.smithi083.stdout: linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-17T21:43:10.305 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.15.0-107-generic linux-modules-5.15.0-107-generic 2024-05-17T21:43:10.305 INFO:teuthology.orchestra.run.smithi083.stdout: linux-modules-extra-5.15.0-107-generic 2024-05-17T21:43:10.306 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-05-17T21:43:10.307 INFO:teuthology.orchestra.run.smithi083.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-17T21:43:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-17T21:43:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 113 MB of archives. 2024-05-17T21:43:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-17T21:43:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-107-generic amd64 5.15.0-107.117 [22.7 MB] 2024-05-17T21:43:10.364 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:43:10.566 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-05-17T21:43:10.566 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-05-17T21:43:10.746 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout: python2.7-minimal 2024-05-17T21:43:10.747 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:43:10.748 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-05-17T21:43:10.749 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-17T21:43:10.749 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-generic linux-image-5.15.0-107-generic 2024-05-17T21:43:10.749 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-5.15.0-107-generic linux-modules-extra-5.15.0-107-generic 2024-05-17T21:43:10.750 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-05-17T21:43:10.751 INFO:teuthology.orchestra.run.smithi155.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-17T21:43:10.751 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-05-17T21:43:10.751 INFO:teuthology.orchestra.run.smithi155.stdout: thermald 2024-05-17T21:43:10.805 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-05-17T21:43:10.805 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-5.15.0-107 linux-headers-5.15.0-107-generic 2024-05-17T21:43:10.805 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.15.0-107-generic linux-modules-5.15.0-107-generic 2024-05-17T21:43:10.806 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-extra-5.15.0-107-generic 2024-05-17T21:43:10.807 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-05-17T21:43:10.807 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-17T21:43:10.830 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-107-generic amd64 5.15.0-107.117 [11.5 MB] 2024-05-17T21:43:10.872 INFO:teuthology.orchestra.run.smithi155.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-17T21:43:10.872 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 113 MB of archives. 2024-05-17T21:43:10.872 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-17T21:43:10.872 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-107-generic amd64 5.15.0-107.117 [22.7 MB] 2024-05-17T21:43:10.945 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-107-generic amd64 5.15.0-107.117 [63.9 MB] 2024-05-17T21:43:11.633 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-107-generic amd64 5.15.0-107.117 [11.5 MB] 2024-05-17T21:43:11.898 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-107-generic amd64 5.15.0-107.117 [63.9 MB] 2024-05-17T21:43:12.277 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.107.107 [1,694 B] 2024-05-17T21:43:12.277 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.107.107 [2,496 B] 2024-05-17T21:43:12.278 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107 all 5.15.0-107.117 [12.3 MB] 2024-05-17T21:43:12.503 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107-generic amd64 5.15.0-107.117 [2,887 kB] 2024-05-17T21:43:12.561 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.107.107 [2,354 B] 2024-05-17T21:43:12.949 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 113 MB in 2s (50.5 MB/s) 2024-05-17T21:43:13.122 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-5.15.0-107-generic. 2024-05-17T21:43:13.126 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.107.107 [1,694 B] 2024-05-17T21:43:13.126 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.107.107 [2,496 B] 2024-05-17T21:43:13.126 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107 all 5.15.0-107.117 [12.3 MB] 2024-05-17T21:43:13.649 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-107-generic amd64 5.15.0-107.117 [2,887 kB] 2024-05-17T21:43:13.693 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.107.107 [2,354 B] 2024-05-17T21:43:14.077 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 113 MB in 3s (39.4 MB/s) 2024-05-17T21:43:14.259 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-5.15.0-107-generic. 2024-05-17T21:43:15.100 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-17T21:43:15.104 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../0-linux-modules-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:15.170 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:16.235 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-17T21:43:16.239 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../0-linux-modules-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:16.307 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:17.271 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-image-5.15.0-107-generic. 2024-05-17T21:43:17.293 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../1-linux-image-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:17.387 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:17.849 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-107-generic. 2024-05-17T21:43:17.870 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:17.916 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:18.984 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-image-5.15.0-107-generic. 2024-05-17T21:43:19.005 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../1-linux-image-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:19.098 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:19.580 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-107-generic. 2024-05-17T21:43:19.600 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:19.637 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:24.960 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../3-linux-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:25.121 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:25.451 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:25.574 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-image-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:25.858 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../3-linux-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:25.885 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-107. 2024-05-17T21:43:25.896 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../5-linux-headers-5.15.0-107_5.15.0-107.117_all.deb ... 2024-05-17T21:43:25.935 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-17T21:43:25.960 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:26.251 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:26.372 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:26.709 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-5.15.0-107. 2024-05-17T21:43:26.731 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../5-linux-headers-5.15.0-107_5.15.0-107.117_all.deb ... 2024-05-17T21:43:26.775 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-17T21:43:31.947 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package linux-headers-5.15.0-107-generic. 2024-05-17T21:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../6-linux-headers-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:32.030 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:32.516 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-5.15.0-107-generic. 2024-05-17T21:43:32.553 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../6-linux-headers-5.15.0-107-generic_5.15.0-107.117_amd64.deb ... 2024-05-17T21:43:32.593 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:34.643 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:34.769 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking linux-headers-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:35.094 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-17T21:43:35.116 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.107.107_amd64.deb ... 2024-05-17T21:43:35.207 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:35.239 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-generic (5.15.0.107.107) over (5.15.0.56.54) ... 2024-05-17T21:43:35.358 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-headers-generic (5.15.0.107.107) ... 2024-05-17T21:43:35.471 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:35.589 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-5.15.0-107 (5.15.0-107.117) ... 2024-05-17T21:43:35.727 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:35.853 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-generic (5.15.0.107.107) ... 2024-05-17T21:43:35.991 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:36.773 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:43:36.773 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-17T21:43:36.773 INFO:teuthology.orchestra.run.smithi083.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:43:36.773 INFO:teuthology.orchestra.run.smithi083.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:36.932 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:37.322 INFO:teuthology.orchestra.run.smithi155.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:43:37.323 INFO:teuthology.orchestra.run.smithi155.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-17T21:43:37.323 INFO:teuthology.orchestra.run.smithi155.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:43:37.323 INFO:teuthology.orchestra.run.smithi155.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:37.494 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-extra-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:38.840 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:39.111 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:40.259 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-image-generic (5.15.0.107.107) ... 2024-05-17T21:43:40.359 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up linux-generic (5.15.0.107.107) ... 2024-05-17T21:43:40.477 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:40.541 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-17T21:43:40.541 INFO:teuthology.orchestra.run.smithi083.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:40.604 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-generic (5.15.0.107.107) ... 2024-05-17T21:43:40.730 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-generic (5.15.0.107.107) ... 2024-05-17T21:43:40.857 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for linux-image-5.15.0-107-generic (5.15.0-107.117) ... 2024-05-17T21:43:40.919 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-17T21:43:40.919 INFO:teuthology.orchestra.run.smithi155.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:54.867 INFO:teuthology.orchestra.run.smithi083.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-17T21:43:54.867 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub' 2024-05-17T21:43:54.893 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-17T21:43:54.893 INFO:teuthology.orchestra.run.smithi155.stdout:Sourcing file `/etc/default/grub' 2024-05-17T21:43:54.899 INFO:teuthology.orchestra.run.smithi083.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-17T21:43:54.908 INFO:teuthology.orchestra.run.smithi155.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-17T21:43:54.921 INFO:teuthology.orchestra.run.smithi083.stdout:Generating grub configuration file ... 2024-05-17T21:43:54.926 INFO:teuthology.orchestra.run.smithi155.stdout:Generating grub configuration file ... 2024-05-17T21:43:55.213 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:43:55.221 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:43:55.236 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:55.247 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:43:55.521 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:43:55.527 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-17T21:43:55.547 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:43:55.553 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-17T21:43:55.597 INFO:teuthology.orchestra.run.smithi083.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-17T21:43:55.602 INFO:teuthology.orchestra.run.smithi083.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-17T21:43:55.623 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-17T21:43:55.628 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-17T21:43:55.766 INFO:teuthology.orchestra.run.smithi083.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-17T21:43:55.766 INFO:teuthology.orchestra.run.smithi083.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-17T21:43:55.766 INFO:teuthology.orchestra.run.smithi083.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-17T21:43:55.787 INFO:teuthology.orchestra.run.smithi155.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-17T21:43:55.787 INFO:teuthology.orchestra.run.smithi155.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-17T21:43:55.787 INFO:teuthology.orchestra.run.smithi155.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-17T21:43:55.825 INFO:teuthology.orchestra.run.smithi155.stdout:done 2024-05-17T21:43:55.826 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-05-17T21:43:56.264 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 21 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-05-17T21:43:56.285 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 5.15.0.107.107 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-05-17T21:43:56.286 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-05-17T21:43:56.286 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-107-generic 2024-05-17T21:43:56.286 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T21:43:56.287 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T21:43:56.287 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-17T21:43:56.287 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T21:43:56.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-05-17T21:43:56.307 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-05-17T21:43:56.326 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-05-17T21:43:56.326 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-05-17T21:43:56.326 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-05-17T21:43:56.326 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.107.107 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-05-17T21:43:56.327 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-05-17T21:43:56.327 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-107-generic 2024-05-17T21:43:56.328 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-17T21:43:56.328 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-17T21:43:56.328 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-17T21:43:56.328 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-05-17T21:43:56.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get clean 2024-05-17T21:43:56.471 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-05-17T21:43:56.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-05-17T21:43:56.607 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:43:56.627 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-17T21:43:56.657 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-17T21:43:56.659 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:43:56.659 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-17T21:43:56.671 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-17T21:43:56.701 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-17T21:43:56.747 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-17T21:43:58.031 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:43:58.042 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:43:58.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-17T21:43:58.061 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-17T21:43:58.101 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:43:58.112 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:43:58.281 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-05-17T21:43:58.282 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-05-17T21:43:58.292 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-05-17T21:43:58.293 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-05-17T21:43:58.441 INFO:teuthology.orchestra.run.smithi083.stdout:linux-image-generic is already the newest version (5.15.0.107.107). 2024-05-17T21:43:58.441 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:43:58.441 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:43:58.441 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:43:58.441 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:43:58.442 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:43:58.442 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:43:58.442 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:43:58.454 INFO:teuthology.orchestra.run.smithi155.stdout:linux-image-generic is already the newest version (5.15.0.107.107). 2024-05-17T21:43:58.454 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:43:58.454 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:43:58.454 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:43:58.454 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:43:58.455 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:43:58.455 INFO:teuthology.orchestra.run.smithi155.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:43:58.455 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:43:58.477 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-17T21:43:58.479 DEBUG:teuthology.orchestra.run.smithi083:> dpkg -s linux-image-generic 2024-05-17T21:43:58.495 INFO:teuthology.orchestra.run.smithi155.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-17T21:43:58.497 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Package: linux-image-generic 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Status: install ok installed 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Priority: optional 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Section: kernel 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Installed-Size: 21 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Maintainer: Ubuntu Kernel Team 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: amd64 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Source: linux-meta 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Version: 5.15.0.107.107 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-17T21:43:58.537 INFO:teuthology.orchestra.run.smithi083.stdout:Recommends: thermald 2024-05-17T21:43:58.538 INFO:teuthology.orchestra.run.smithi083.stdout:Description: Generic Linux kernel image 2024-05-17T21:43:58.538 INFO:teuthology.orchestra.run.smithi083.stdout: This package will always depend on the latest generic kernel image 2024-05-17T21:43:58.538 INFO:teuthology.orchestra.run.smithi083.stdout: available. 2024-05-17T21:43:58.539 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 21 2024-05-17T21:43:58.554 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 5.15.0.107.107 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-107), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-5.15.0-107-generic, linux-modules-extra-5.15.0-107-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-05-17T21:43:58.555 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-05-17T21:43:58.555 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-05-17T21:43:58.586 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.YOmsxRJKiQ 2024-05-17T21:43:58.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YOmsxRJKiQ 2024-05-17T21:43:58.598 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.ujnrRW529G 2024-05-17T21:43:58.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ujnrRW529G 2024-05-17T21:43:58.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.YOmsxRJKiQ 2024-05-17T21:43:58.655 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.ujnrRW529G 2024-05-17T21:43:58.754 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.YOmsxRJKiQ is 10KB 2024-05-17T21:43:58.767 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.YOmsxRJKiQ 2024-05-17T21:43:58.769 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.ujnrRW529G is 10KB 2024-05-17T21:43:58.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.782 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.ujnrRW529G 2024-05-17T21:43:58.787 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.831 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:43:58.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.843 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:43:58.843 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.843 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-17T21:43:58.910 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-107-generic 2024-05-17T21:43:58.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo update-grub 2024-05-17T21:43:58.915 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-107-generic 2024-05-17T21:43:58.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo update-grub 2024-05-17T21:44:00.068 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub' 2024-05-17T21:44:00.069 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-17T21:44:00.072 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-05-17T21:44:00.103 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub' 2024-05-17T21:44:00.104 INFO:teuthology.orchestra.run.smithi083.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-17T21:44:00.106 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-05-17T21:44:00.247 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:44:00.253 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:44:00.287 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-107-generic 2024-05-17T21:44:00.293 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-107-generic 2024-05-17T21:44:00.490 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:44:00.496 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-17T21:44:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-17T21:44:00.511 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-17T21:44:00.538 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-17T21:44:00.543 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-17T21:44:00.550 INFO:teuthology.orchestra.run.smithi083.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-17T21:44:00.555 INFO:teuthology.orchestra.run.smithi083.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-17T21:44:00.665 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-17T21:44:00.665 INFO:teuthology.orchestra.run.smithi155.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-17T21:44:00.665 INFO:teuthology.orchestra.run.smithi155.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-17T21:44:00.677 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-05-17T21:44:00.693 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-05-17T21:44:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-17T21:44:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-17T21:44:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-17T21:44:00.724 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-05-17T21:44:00.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-05-17T21:44:30.726 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-17T21:44:30.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:44:30.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:44:30.734 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-17T21:44:30.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:44:30.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:44:49.183 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-05-17T21:44:49.183 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-05-17T21:44:58.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:44:58.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:44:58.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:44:58.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:45:01.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-05-17T21:45:01.247 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-05-17T21:45:13.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:45:13.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:45:13.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:45:13.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:45:31.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-05-17T21:45:46.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:45:46.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:45:47.217 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-05-17T21:45:48.418 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:45:48.419 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-107-generic"... 2024-05-17T21:45:48.419 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-05-17T21:45:48.465 INFO:teuthology.orchestra.run.smithi083.stdout:5.15.0-107-generic 2024-05-17T21:45:48.465 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-107-generic vs 5.15.0-107-generic 2024-05-17T21:45:48.465 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T21:45:48.465 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-17T21:45:49.465 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-17T21:45:49.466 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T21:45:49.990 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-05-17T21:45:50.035 DEBUG:teuthology.parallel:result is None 2024-05-17T21:46:13.261 DEBUG:teuthology.orchestra.remote:timed out 2024-05-17T21:46:28.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:46:28.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:46:28.493 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-05-17T21:46:29.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:46:29.083 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-107-generic"... 2024-05-17T21:46:29.084 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-05-17T21:46:29.129 INFO:teuthology.orchestra.run.smithi155.stdout:5.15.0-107-generic 2024-05-17T21:46:29.129 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-107-generic vs 5.15.0-107-generic 2024-05-17T21:46:29.129 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-17T21:46:29.129 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-17T21:46:30.130 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-17T21:46:30.131 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-17T21:46:30.467 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-05-17T21:46:30.503 DEBUG:teuthology.parallel:result is None 2024-05-17T21:46:30.503 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-17T21:46:30.511 INFO:teuthology.task.internal:Creating test directory... 2024-05-17T21:46:30.511 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T21:46:30.514 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-17T21:46:30.519 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-17T21:46:30.591 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-17T21:46:30.621 INFO:teuthology.task.internal:Creating archive directory... 2024-05-17T21:46:30.622 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T21:46:30.624 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-17T21:46:30.697 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-17T21:46:30.703 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-17T21:46:30.703 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T21:46:30.706 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-17T21:46:30.723 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:46:30.729 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:46:30.731 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:46:30.737 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-17T21:46:30.739 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-17T21:46:30.843 INFO:teuthology.task.internal:Configuring sudo... 2024-05-17T21:46:30.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T21:46:30.846 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-17T21:46:30.869 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-17T21:46:30.900 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-17T21:46:30.900 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T21:46:30.913 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-17T21:46:30.918 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:46:30.964 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:46:31.011 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:46:31.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T21:46:31.077 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-17T21:46:31.083 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-17T21:46:31.129 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:46:31.129 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-17T21:46:31.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-05-17T21:46:31.198 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-05-17T21:46:31.267 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-17T21:46:31.274 INFO:teuthology.task.internal:Starting timer... 2024-05-17T21:46:31.274 INFO:teuthology.run_tasks:Running task pcp... 2024-05-17T21:46:31.351 INFO:teuthology.run_tasks:Running task selinux... 2024-05-17T21:46:31.381 DEBUG:teuthology.task.selinux:Excluding smithi083: OS 'ubuntu' does not support SELinux 2024-05-17T21:46:31.381 DEBUG:teuthology.task.selinux:Excluding smithi155: OS 'ubuntu' does not support SELinux 2024-05-17T21:46:31.381 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-17T21:46:31.381 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-17T21:46:31.382 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-17T21:46:31.382 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-17T21:46:31.420 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-17T21:46:31.423 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-17T21:46:31.545 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-17T21:46:31.557 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-17T21:46:31.559 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-17T21:53:32.889 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-05-17T21:53:32.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:53:32.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:53:32.984 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-05-17T21:53:33.062 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-05-17T21:53:33.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:53:33.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-17T21:53:33.146 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-05-17T21:53:33.222 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-17T21:53:33.223 INFO:teuthology.run_tasks:Running task clock... 2024-05-17T21:53:33.235 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-17T21:53:33.235 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T21:53:33.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T21:53:33.238 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-17T21:53:33.238 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T21:53:33.267 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Command line: ntpd -gq 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: ---------------------------------------------------- 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: ntp-4 is maintained by Network Time Foundation, 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: corporation. Support and training for ntp-4 are 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: available at https://www.nwtime.org/support 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: ---------------------------------------------------- 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: proto: precision = 0.044 usec (-24) 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: basedate set to 2022-02-04 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: gps base set to 2022-02-06 (week 2196) 2024-05-17T21:53:33.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-17T21:53:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-17T21:53:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: restrict ::: KOD does nothing without LIMITED. 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen and drop on 0 v6wildcard [::]:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen normally on 2 lo 127.0.0.1:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen normally on 3 enp3s0f1 172.21.15.83:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen normally on 4 lo [::1]:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce91%5]:123 2024-05-17T21:53:33.270 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:33 ntpd[17252]: Listening on routing socket on fd #22 for interface updates 2024-05-17T21:53:33.288 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-17T21:53:33.288 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Command line: ntpd -gq 2024-05-17T21:53:33.288 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: ---------------------------------------------------- 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: ntp-4 is maintained by Network Time Foundation, 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: corporation. Support and training for ntp-4 are 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: available at https://www.nwtime.org/support 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: ---------------------------------------------------- 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: proto: precision = 0.044 usec (-24) 2024-05-17T21:53:33.289 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: basedate set to 2022-02-04 2024-05-17T21:53:33.290 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: gps base set to 2022-02-06 (week 2196) 2024-05-17T21:53:33.290 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-17T21:53:33.290 INFO:teuthology.orchestra.run.smithi155.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-17T21:53:33.290 INFO:teuthology.orchestra.run.smithi155.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: restrict ::: KOD does nothing without LIMITED. 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen and drop on 0 v6wildcard [::]:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen normally on 2 lo 127.0.0.1:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen normally on 3 enp3s0f1 172.21.15.155:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen normally on 4 lo [::1]:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6ded%5]:123 2024-05-17T21:53:33.291 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:33 ntpd[17261]: Listening on routing socket on fd #22 for interface updates 2024-05-17T21:53:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:34 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:34 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:34.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:34.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:34.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:34.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:34.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:34 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:35.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:35 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:36.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:36.292 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:36 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:37 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:38.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:38 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:38.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:38.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:38 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:39.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:39.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:39.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:39.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:39 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:40.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:40.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:40 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:40.289 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.289 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:40.290 INFO:teuthology.orchestra.run.smithi155.stdout:17 May 21:53:40 ntpd[17261]: ntpd: time slew -0.000493 s 2024-05-17T21:53:40.291 INFO:teuthology.orchestra.run.smithi155.stdout:ntpd: time slew -0.000493s 2024-05-17T21:53:40.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-17T21:53:40.291 INFO:teuthology.orchestra.run.smithi155.stderr:17 May 21:53:40 ntpd[17261]: can't open /var/log/ntpstats/loopstats.20240517: Permission denied 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:41.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:41.267 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: can't open /var/log/ntpstats/rawstats.20240517: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: can't open /var/log/ntpstats/peerstats.20240517: Permission denied 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stdout:17 May 21:53:41 ntpd[17252]: ntpd: time slew -0.000940 s 2024-05-17T21:53:41.268 INFO:teuthology.orchestra.run.smithi083.stdout:ntpd: time slew -0.000940s 2024-05-17T21:53:41.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-17T21:53:41.269 INFO:teuthology.orchestra.run.smithi083.stderr:17 May 21:53:41 ntpd[17252]: can't open /var/log/ntpstats/loopstats.20240517: Permission denied 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:41.318 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-17T21:53:41.319 INFO:teuthology.run_tasks:Running task install... 2024-05-17T21:53:41.329 DEBUG:teuthology.task.install:project ceph 2024-05-17T21:53:41.330 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}} 2024-05-17T21:53:41.330 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'} 2024-05-17T21:53:41.330 INFO:teuthology.task.install:Using flavor: default 2024-05-17T21:53:41.335 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-17T21:53:41.335 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-17T21:53:41.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-key list | grep Ceph 2024-05-17T21:53:41.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-key list | grep Ceph 2024-05-17T21:53:41.395 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-17T21:53:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-17T21:53:41.427 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-17T21:53:41.427 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph.com (release key) 2024-05-17T21:53:41.427 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-17T21:53:41.427 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-17T21:53:41.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:53:41.445 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-17T21:53:41.445 INFO:teuthology.orchestra.run.smithi083.stdout:uid [ unknown] Ceph.com (release key) 2024-05-17T21:53:41.446 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-05-17T21:53:41.446 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-05-17T21:53:41.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:53:41.566 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default/ 2024-05-17T21:53:41.567 INFO:teuthology.task.install.deb:Package version is 19.0.0-3794-g931809a9-1jammy 2024-05-17T21:53:41.585 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default/ 2024-05-17T21:53:41.585 INFO:teuthology.task.install.deb:Package version is 19.0.0-3794-g931809a9-1jammy 2024-05-17T21:53:41.678 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:53:41.678 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-17T21:53:41.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-05-17T21:53:41.755 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:53:41.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-17T21:53:41.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo apt-get update 2024-05-17T21:53:41.932 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-17T21:53:41.938 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:53:41.954 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-17T21:53:41.975 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy InRelease 2024-05-17T21:53:42.012 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-17T21:53:42.013 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-17T21:53:42.013 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-17T21:53:42.044 INFO:teuthology.orchestra.run.smithi083.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-17T21:53:42.046 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy InRelease 2024-05-17T21:53:42.050 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-17T21:53:42.082 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-17T21:53:42.087 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-17T21:53:42.088 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-17T21:53:42.119 INFO:teuthology.orchestra.run.smithi083.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-17T21:53:42.134 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-05-17T21:53:42.155 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-05-17T21:53:42.169 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-17T21:53:42.198 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-17T21:53:42.272 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 258 kB in 0s (787 kB/s) 2024-05-17T21:53:42.385 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 28.5 kB in 1s (55.9 kB/s) 2024-05-17T21:53:43.255 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:53:43.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3794-g931809a9-1jammy cephadm=19.0.0-3794-g931809a9-1jammy ceph-mds=19.0.0-3794-g931809a9-1jammy ceph-mgr=19.0.0-3794-g931809a9-1jammy ceph-common=19.0.0-3794-g931809a9-1jammy ceph-fuse=19.0.0-3794-g931809a9-1jammy ceph-test=19.0.0-3794-g931809a9-1jammy ceph-volume=19.0.0-3794-g931809a9-1jammy radosgw=19.0.0-3794-g931809a9-1jammy python3-rados=19.0.0-3794-g931809a9-1jammy python3-rgw=19.0.0-3794-g931809a9-1jammy python3-cephfs=19.0.0-3794-g931809a9-1jammy python3-rbd=19.0.0-3794-g931809a9-1jammy libcephfs2=19.0.0-3794-g931809a9-1jammy libcephfs-dev=19.0.0-3794-g931809a9-1jammy librados2=19.0.0-3794-g931809a9-1jammy librbd1=19.0.0-3794-g931809a9-1jammy rbd-fuse=19.0.0-3794-g931809a9-1jammy python3-cephfs=19.0.0-3794-g931809a9-1jammy cephfs-shell=19.0.0-3794-g931809a9-1jammy cephfs-top=19.0.0-3794-g931809a9-1jammy cephfs-mirror=19.0.0-3794-g931809a9-1jammy 2024-05-17T21:53:43.331 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:53:43.357 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:53:43.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3794-g931809a9-1jammy cephadm=19.0.0-3794-g931809a9-1jammy ceph-mds=19.0.0-3794-g931809a9-1jammy ceph-mgr=19.0.0-3794-g931809a9-1jammy ceph-common=19.0.0-3794-g931809a9-1jammy ceph-fuse=19.0.0-3794-g931809a9-1jammy ceph-test=19.0.0-3794-g931809a9-1jammy ceph-volume=19.0.0-3794-g931809a9-1jammy radosgw=19.0.0-3794-g931809a9-1jammy python3-rados=19.0.0-3794-g931809a9-1jammy python3-rgw=19.0.0-3794-g931809a9-1jammy python3-cephfs=19.0.0-3794-g931809a9-1jammy python3-rbd=19.0.0-3794-g931809a9-1jammy libcephfs2=19.0.0-3794-g931809a9-1jammy libcephfs-dev=19.0.0-3794-g931809a9-1jammy librados2=19.0.0-3794-g931809a9-1jammy librbd1=19.0.0-3794-g931809a9-1jammy rbd-fuse=19.0.0-3794-g931809a9-1jammy python3-cephfs=19.0.0-3794-g931809a9-1jammy cephfs-shell=19.0.0-3794-g931809a9-1jammy cephfs-top=19.0.0-3794-g931809a9-1jammy cephfs-mirror=19.0.0-3794-g931809a9-1jammy 2024-05-17T21:53:43.427 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:53:43.540 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-05-17T21:53:43.541 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-05-17T21:53:43.604 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-05-17T21:53:43.605 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-05-17T21:53:43.727 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:53:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:53:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:53:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:53:43.727 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:53:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:53:43.728 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:53:43.728 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-05-17T21:53:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-17T21:53:43.728 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-17T21:53:43.729 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-17T21:53:43.730 INFO:teuthology.orchestra.run.smithi083.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-17T21:53:43.731 INFO:teuthology.orchestra.run.smithi083.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-17T21:53:43.733 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-05-17T21:53:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-17T21:53:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-17T21:53:43.733 INFO:teuthology.orchestra.run.smithi083.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout:Recommended packages: 2024-05-17T21:53:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: btrfs-tools 2024-05-17T21:53:43.778 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:53:43.778 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:53:43.778 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:53:43.778 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:53:43.779 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:53:43.779 INFO:teuthology.orchestra.run.smithi155.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:53:43.779 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:53:43.779 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-05-17T21:53:43.780 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-17T21:53:43.780 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-17T21:53:43.780 INFO:teuthology.orchestra.run.smithi155.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-17T21:53:43.780 INFO:teuthology.orchestra.run.smithi155.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-05-17T21:53:43.781 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-05-17T21:53:43.782 INFO:teuthology.orchestra.run.smithi155.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-05-17T21:53:43.783 INFO:teuthology.orchestra.run.smithi155.stdout: btrfs-tools 2024-05-17T21:53:43.835 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-05-17T21:53:43.835 INFO:teuthology.orchestra.run.smithi083.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-17T21:53:43.835 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-17T21:53:43.835 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-17T21:53:43.836 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-17T21:53:43.836 INFO:teuthology.orchestra.run.smithi083.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-17T21:53:43.836 INFO:teuthology.orchestra.run.smithi083.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-17T21:53:43.836 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-17T21:53:43.837 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-17T21:53:43.838 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-17T21:53:43.839 INFO:teuthology.orchestra.run.smithi083.stdout: socat unzip xmlstarlet zip 2024-05-17T21:53:43.886 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-05-17T21:53:43.886 INFO:teuthology.orchestra.run.smithi155.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-17T21:53:43.886 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-17T21:53:43.886 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-05-17T21:53:43.886 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-05-17T21:53:43.888 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-05-17T21:53:43.889 INFO:teuthology.orchestra.run.smithi155.stdout: socat unzip xmlstarlet zip 2024-05-17T21:53:43.955 INFO:teuthology.orchestra.run.smithi155.stdout:0 upgraded, 124 newly installed, 0 to remove and 307 not upgraded. 2024-05-17T21:53:43.955 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 186 MB of archives. 2024-05-17T21:53:43.955 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 952 MB of additional disk space will be used. 2024-05-17T21:53:43.956 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-17T21:53:43.976 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-17T21:53:43.988 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-17T21:53:44.024 INFO:teuthology.orchestra.run.smithi083.stdout:0 upgraded, 124 newly installed, 0 to remove and 307 not upgraded. 2024-05-17T21:53:44.024 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 186 MB of archives. 2024-05-17T21:53:44.024 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 952 MB of additional disk space will be used. 2024-05-17T21:53:44.024 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-05-17T21:53:44.035 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-17T21:53:44.056 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3794-g931809a9-1jammy [3,531 kB] 2024-05-17T21:53:44.059 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-17T21:53:44.063 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-17T21:53:44.076 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-17T21:53:44.106 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-17T21:53:44.107 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3794-g931809a9-1jammy [3,531 kB] 2024-05-17T21:53:44.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-17T21:53:44.132 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-17T21:53:44.144 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-17T21:53:44.148 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-17T21:53:44.148 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-17T21:53:44.152 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-05-17T21:53:44.156 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-17T21:53:44.158 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-17T21:53:44.160 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-17T21:53:44.162 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-17T21:53:44.162 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-17T21:53:44.171 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-17T21:53:44.174 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-17T21:53:44.175 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-17T21:53:44.176 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-17T21:53:44.194 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-17T21:53:44.195 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-17T21:53:44.195 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-17T21:53:44.195 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-17T21:53:44.196 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-17T21:53:44.196 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-17T21:53:44.213 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-17T21:53:44.214 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-17T21:53:44.216 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-17T21:53:44.216 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-17T21:53:44.233 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-17T21:53:44.233 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-17T21:53:44.234 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-17T21:53:44.234 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-17T21:53:44.235 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-17T21:53:44.236 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-17T21:53:44.252 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-17T21:53:44.253 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-17T21:53:44.258 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-17T21:53:44.258 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-17T21:53:44.272 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-17T21:53:44.273 INFO:teuthology.orchestra.run.smithi155.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-17T21:53:44.275 INFO:teuthology.orchestra.run.smithi155.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-17T21:53:44.278 INFO:teuthology.orchestra.run.smithi155.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-17T21:53:44.279 INFO:teuthology.orchestra.run.smithi155.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-17T21:53:44.280 INFO:teuthology.orchestra.run.smithi155.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-17T21:53:44.292 INFO:teuthology.orchestra.run.smithi155.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-17T21:53:44.295 INFO:teuthology.orchestra.run.smithi155.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-17T21:53:44.295 INFO:teuthology.orchestra.run.smithi155.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-17T21:53:44.296 INFO:teuthology.orchestra.run.smithi155.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-17T21:53:44.320 INFO:teuthology.orchestra.run.smithi155.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-17T21:53:44.354 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-17T21:53:44.379 INFO:teuthology.orchestra.run.smithi155.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-17T21:53:44.380 INFO:teuthology.orchestra.run.smithi155.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-17T21:53:44.381 INFO:teuthology.orchestra.run.smithi155.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-17T21:53:44.382 INFO:teuthology.orchestra.run.smithi155.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-17T21:53:44.383 INFO:teuthology.orchestra.run.smithi155.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-17T21:53:44.403 INFO:teuthology.orchestra.run.smithi155.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-17T21:53:44.406 INFO:teuthology.orchestra.run.smithi155.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-17T21:53:44.406 INFO:teuthology.orchestra.run.smithi155.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-17T21:53:44.420 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3794-g931809a9-1jammy [3,290 kB] 2024-05-17T21:53:44.423 INFO:teuthology.orchestra.run.smithi155.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-17T21:53:44.423 INFO:teuthology.orchestra.run.smithi155.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-17T21:53:44.432 INFO:teuthology.orchestra.run.smithi155.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-17T21:53:44.432 INFO:teuthology.orchestra.run.smithi155.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-17T21:53:44.432 INFO:teuthology.orchestra.run.smithi155.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-17T21:53:44.433 INFO:teuthology.orchestra.run.smithi155.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-17T21:53:44.434 INFO:teuthology.orchestra.run.smithi155.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-17T21:53:44.434 INFO:teuthology.orchestra.run.smithi155.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-17T21:53:44.437 INFO:teuthology.orchestra.run.smithi155.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-17T21:53:44.448 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-17T21:53:44.451 INFO:teuthology.orchestra.run.smithi155.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-17T21:53:44.452 INFO:teuthology.orchestra.run.smithi155.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-17T21:53:44.453 INFO:teuthology.orchestra.run.smithi155.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-17T21:53:44.462 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-17T21:53:44.471 INFO:teuthology.orchestra.run.smithi155.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-17T21:53:44.474 INFO:teuthology.orchestra.run.smithi155.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-17T21:53:44.474 INFO:teuthology.orchestra.run.smithi155.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-17T21:53:44.475 INFO:teuthology.orchestra.run.smithi155.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-17T21:53:44.476 INFO:teuthology.orchestra.run.smithi155.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-17T21:53:44.480 INFO:teuthology.orchestra.run.smithi155.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-17T21:53:44.481 INFO:teuthology.orchestra.run.smithi155.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-17T21:53:44.490 INFO:teuthology.orchestra.run.smithi155.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-17T21:53:44.490 INFO:teuthology.orchestra.run.smithi155.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-17T21:53:44.491 INFO:teuthology.orchestra.run.smithi155.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-17T21:53:44.509 INFO:teuthology.orchestra.run.smithi155.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-17T21:53:44.512 INFO:teuthology.orchestra.run.smithi155.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-17T21:53:44.513 INFO:teuthology.orchestra.run.smithi155.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-17T21:53:44.513 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-17T21:53:44.521 INFO:teuthology.orchestra.run.smithi155.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-17T21:53:44.521 INFO:teuthology.orchestra.run.smithi155.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-17T21:53:44.521 INFO:teuthology.orchestra.run.smithi155.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-17T21:53:44.523 INFO:teuthology.orchestra.run.smithi155.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-17T21:53:44.532 INFO:teuthology.orchestra.run.smithi155.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-17T21:53:44.557 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3794-g931809a9-1jammy [729 kB] 2024-05-17T21:53:44.573 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3794-g931809a9-1jammy [356 kB] 2024-05-17T21:53:44.584 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3794-g931809a9-1jammy [32.8 kB] 2024-05-17T21:53:44.586 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3794-g931809a9-1jammy [180 kB] 2024-05-17T21:53:44.591 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3794-g931809a9-1jammy [67.4 kB] 2024-05-17T21:53:44.593 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3794-g931809a9-1jammy [334 kB] 2024-05-17T21:53:44.605 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3794-g931809a9-1jammy [8,756 kB] 2024-05-17T21:53:44.687 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-17T21:53:44.696 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-17T21:53:44.723 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-17T21:53:44.731 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-17T21:53:44.897 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-17T21:53:44.943 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3794-g931809a9-1jammy [112 kB] 2024-05-17T21:53:44.945 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3794-g931809a9-1jammy [474 kB] 2024-05-17T21:53:44.955 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3794-g931809a9-1jammy [28.8 MB] 2024-05-17T21:53:45.473 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-17T21:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-17T21:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-17T21:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-17T21:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-17T21:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-17T21:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-17T21:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-17T21:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-17T21:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-17T21:53:45.695 INFO:teuthology.orchestra.run.smithi155.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3794-g931809a9-1jammy [3,290 kB] 2024-05-17T21:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-17T21:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-17T21:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-17T21:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-17T21:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-17T21:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-17T21:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-17T21:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3794-g931809a9-1jammy [4,914 kB] 2024-05-17T21:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3794-g931809a9-1jammy [245 kB] 2024-05-17T21:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout:Get:44 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3794-g931809a9-1jammy [125 kB] 2024-05-17T21:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout:Get:45 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3794-g931809a9-1jammy [1,544 kB] 2024-05-17T21:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-17T21:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-17T21:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-17T21:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-17T21:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3794-g931809a9-1jammy [6,152 kB] 2024-05-17T21:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-17T21:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3794-g931809a9-1jammy [22.5 MB] 2024-05-17T21:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-17T21:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-17T21:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-17T21:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-17T21:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-17T21:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-17T21:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-17T21:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-17T21:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-17T21:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-17T21:53:47.022 INFO:teuthology.orchestra.run.smithi155.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3794-g931809a9-1jammy [729 kB] 2024-05-17T21:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-17T21:53:47.328 INFO:teuthology.orchestra.run.smithi155.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3794-g931809a9-1jammy [356 kB] 2024-05-17T21:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3794-g931809a9-1jammy [14.0 kB] 2024-05-17T21:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3794-g931809a9-1jammy [858 kB] 2024-05-17T21:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3794-g931809a9-1jammy [2,376 kB] 2024-05-17T21:53:47.395 INFO:teuthology.orchestra.run.smithi083.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3794-g931809a9-1jammy [792 kB] 2024-05-17T21:53:47.405 INFO:teuthology.orchestra.run.smithi083.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3794-g931809a9-1jammy [149 kB] 2024-05-17T21:53:47.406 INFO:teuthology.orchestra.run.smithi083.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3794-g931809a9-1jammy [2,222 kB] 2024-05-17T21:53:47.418 INFO:teuthology.orchestra.run.smithi083.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-17T21:53:47.443 INFO:teuthology.orchestra.run.smithi155.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3794-g931809a9-1jammy [32.8 kB] 2024-05-17T21:53:47.444 INFO:teuthology.orchestra.run.smithi155.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3794-g931809a9-1jammy [180 kB] 2024-05-17T21:53:47.481 INFO:teuthology.orchestra.run.smithi083.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3794-g931809a9-1jammy [8,625 kB] 2024-05-17T21:53:47.532 INFO:teuthology.orchestra.run.smithi083.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-17T21:53:47.534 INFO:teuthology.orchestra.run.smithi155.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3794-g931809a9-1jammy [67.4 kB] 2024-05-17T21:53:47.536 INFO:teuthology.orchestra.run.smithi155.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3794-g931809a9-1jammy [334 kB] 2024-05-17T21:53:47.577 INFO:teuthology.orchestra.run.smithi083.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-17T21:53:47.659 INFO:teuthology.orchestra.run.smithi083.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-17T21:53:47.672 INFO:teuthology.orchestra.run.smithi083.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-17T21:53:47.685 INFO:teuthology.orchestra.run.smithi083.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-17T21:53:47.692 INFO:teuthology.orchestra.run.smithi155.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3794-g931809a9-1jammy [8,756 kB] 2024-05-17T21:53:47.725 INFO:teuthology.orchestra.run.smithi083.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-17T21:53:47.793 INFO:teuthology.orchestra.run.smithi083.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3794-g931809a9-1jammy [14.2 kB] 2024-05-17T21:53:47.793 INFO:teuthology.orchestra.run.smithi083.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3794-g931809a9-1jammy [48.1 MB] 2024-05-17T21:53:47.987 INFO:teuthology.orchestra.run.smithi083.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-17T21:53:47.989 INFO:teuthology.orchestra.run.smithi083.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-17T21:53:47.992 INFO:teuthology.orchestra.run.smithi083.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-17T21:53:48.156 INFO:teuthology.orchestra.run.smithi083.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-17T21:53:48.482 INFO:teuthology.orchestra.run.smithi083.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-17T21:53:48.888 INFO:teuthology.orchestra.run.smithi083.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-17T21:53:48.897 INFO:teuthology.orchestra.run.smithi083.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-17T21:53:48.898 INFO:teuthology.orchestra.run.smithi083.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-17T21:53:48.968 INFO:teuthology.orchestra.run.smithi083.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-17T21:53:48.968 INFO:teuthology.orchestra.run.smithi083.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-17T21:53:48.970 INFO:teuthology.orchestra.run.smithi083.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-17T21:53:48.971 INFO:teuthology.orchestra.run.smithi083.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-17T21:53:48.972 INFO:teuthology.orchestra.run.smithi083.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-17T21:53:49.138 INFO:teuthology.orchestra.run.smithi083.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-17T21:53:49.234 INFO:teuthology.orchestra.run.smithi083.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-17T21:53:49.330 INFO:teuthology.orchestra.run.smithi083.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-17T21:53:49.587 INFO:teuthology.orchestra.run.smithi083.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-17T21:53:49.659 INFO:teuthology.orchestra.run.smithi083.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-17T21:53:49.733 INFO:teuthology.orchestra.run.smithi083.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-17T21:53:49.734 INFO:teuthology.orchestra.run.smithi083.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3794-g931809a9-1jammy [124 kB] 2024-05-17T21:53:49.734 INFO:teuthology.orchestra.run.smithi083.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-17T21:53:49.737 INFO:teuthology.orchestra.run.smithi083.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3794-g931809a9-1jammy [240 kB] 2024-05-17T21:53:49.742 INFO:teuthology.orchestra.run.smithi083.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3794-g931809a9-1jammy [33.9 kB] 2024-05-17T21:53:49.743 INFO:teuthology.orchestra.run.smithi083.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-17T21:53:49.743 INFO:teuthology.orchestra.run.smithi083.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3794-g931809a9-1jammy [23.4 kB] 2024-05-17T21:53:49.743 INFO:teuthology.orchestra.run.smithi083.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3794-g931809a9-1jammy [38.9 kB] 2024-05-17T21:53:49.744 INFO:teuthology.orchestra.run.smithi083.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3794-g931809a9-1jammy [17.4 MB] 2024-05-17T21:53:49.749 INFO:teuthology.orchestra.run.smithi083.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-05-17T21:53:49.750 INFO:teuthology.orchestra.run.smithi083.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-17T21:53:49.756 INFO:teuthology.orchestra.run.smithi083.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-17T21:53:49.935 INFO:teuthology.orchestra.run.smithi083.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-17T21:53:50.347 INFO:teuthology.orchestra.run.smithi083.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3794-g931809a9-1jammy [92.6 kB] 2024-05-17T21:53:50.470 INFO:teuthology.orchestra.run.smithi083.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-17T21:53:50.475 INFO:teuthology.orchestra.run.smithi083.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-17T21:53:50.514 INFO:teuthology.orchestra.run.smithi083.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-17T21:53:50.515 INFO:teuthology.orchestra.run.smithi083.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-17T21:53:50.521 INFO:teuthology.orchestra.run.smithi083.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-17T21:53:50.524 INFO:teuthology.orchestra.run.smithi083.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-17T21:53:50.554 INFO:teuthology.orchestra.run.smithi155.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3794-g931809a9-1jammy [112 kB] 2024-05-17T21:53:50.564 INFO:teuthology.orchestra.run.smithi083.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-17T21:53:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-17T21:53:50.646 INFO:teuthology.orchestra.run.smithi083.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-17T21:53:50.647 INFO:teuthology.orchestra.run.smithi083.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-17T21:53:50.654 INFO:teuthology.orchestra.run.smithi155.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3794-g931809a9-1jammy [474 kB] 2024-05-17T21:53:50.691 INFO:teuthology.orchestra.run.smithi155.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3794-g931809a9-1jammy [28.8 MB] 2024-05-17T21:53:50.812 INFO:teuthology.orchestra.run.smithi083.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-17T21:53:51.254 INFO:teuthology.orchestra.run.smithi083.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-17T21:53:51.271 INFO:teuthology.orchestra.run.smithi083.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-17T21:53:52.172 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 186 MB in 8s (24.1 MB/s) 2024-05-17T21:53:52.312 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-17T21:53:52.401 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-17T21:53:52.406 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-17T21:53:52.452 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-17T21:53:52.713 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-17T21:53:52.739 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-17T21:53:52.772 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-17T21:53:53.058 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cmd2. 2024-05-17T21:53:53.084 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-17T21:53:53.117 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-17T21:53:53.419 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-17T21:53:53.445 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-17T21:53:53.519 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-17T21:53:53.864 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-17T21:53:53.890 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-17T21:53:53.931 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-17T21:53:54.276 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-17T21:53:54.302 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-17T21:53:54.342 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-17T21:53:54.738 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-17T21:53:54.764 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:53:54.804 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:53:55.070 INFO:teuthology.orchestra.run.smithi155.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3794-g931809a9-1jammy [4,914 kB] 2024-05-17T21:53:55.207 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-17T21:53:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:53:55.282 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:53:55.593 INFO:teuthology.orchestra.run.smithi155.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3794-g931809a9-1jammy [245 kB] 2024-05-17T21:53:55.626 INFO:teuthology.orchestra.run.smithi155.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3794-g931809a9-1jammy [125 kB] 2024-05-17T21:53:55.645 INFO:teuthology.orchestra.run.smithi155.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3794-g931809a9-1jammy [1,544 kB] 2024-05-17T21:53:55.652 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-17T21:53:55.679 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:53:55.719 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:53:55.825 INFO:teuthology.orchestra.run.smithi155.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3794-g931809a9-1jammy [6,152 kB] 2024-05-17T21:53:56.030 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-17T21:53:56.045 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-17T21:53:56.080 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-17T21:53:56.417 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librados2. 2024-05-17T21:53:56.443 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../010-librados2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:56.483 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librados2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:56.522 INFO:teuthology.orchestra.run.smithi155.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3794-g931809a9-1jammy [22.5 MB] 2024-05-17T21:53:56.978 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librbd1. 2024-05-17T21:53:57.004 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../011-librbd1_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:57.044 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librbd1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:57.489 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs2. 2024-05-17T21:53:57.515 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:57.547 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:57.816 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rados. 2024-05-17T21:53:57.838 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:57.883 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rados (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:58.211 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-17T21:53:58.237 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:53:58.277 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-argparse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:58.580 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cephfs. 2024-05-17T21:53:58.606 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:58.646 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cephfs (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:58.687 INFO:teuthology.orchestra.run.smithi155.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3794-g931809a9-1jammy [14.0 kB] 2024-05-17T21:53:58.688 INFO:teuthology.orchestra.run.smithi155.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3794-g931809a9-1jammy [858 kB] 2024-05-17T21:53:58.704 INFO:teuthology.orchestra.run.smithi155.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3794-g931809a9-1jammy [2,376 kB] 2024-05-17T21:53:58.815 INFO:teuthology.orchestra.run.smithi155.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3794-g931809a9-1jammy [792 kB] 2024-05-17T21:53:58.856 INFO:teuthology.orchestra.run.smithi155.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3794-g931809a9-1jammy [149 kB] 2024-05-17T21:53:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3794-g931809a9-1jammy [2,222 kB] 2024-05-17T21:53:58.966 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-17T21:53:58.992 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:53:59.011 INFO:teuthology.orchestra.run.smithi155.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3794-g931809a9-1jammy [8,625 kB] 2024-05-17T21:53:59.032 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:53:59.421 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-prettytable. 2024-05-17T21:53:59.447 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-17T21:53:59.496 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-17T21:53:59.534 INFO:teuthology.orchestra.run.smithi155.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3794-g931809a9-1jammy [14.2 kB] 2024-05-17T21:53:59.535 INFO:teuthology.orchestra.run.smithi155.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3794-g931809a9-1jammy [48.1 MB] 2024-05-17T21:53:59.774 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rbd. 2024-05-17T21:53:59.788 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:53:59.832 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rbd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:00.169 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-17T21:54:00.195 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-17T21:54:00.226 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-17T21:54:00.504 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-17T21:54:00.530 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-17T21:54:00.571 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-17T21:54:00.883 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-17T21:54:00.909 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-17T21:54:00.949 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-17T21:54:01.278 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua5.1. 2024-05-17T21:54:01.304 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-17T21:54:01.344 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-17T21:54:01.964 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-any. 2024-05-17T21:54:01.980 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-17T21:54:02.010 INFO:teuthology.orchestra.run.smithi155.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3794-g931809a9-1jammy [124 kB] 2024-05-17T21:54:02.022 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-17T21:54:02.043 INFO:teuthology.orchestra.run.smithi155.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3794-g931809a9-1jammy [240 kB] 2024-05-17T21:54:02.049 INFO:teuthology.orchestra.run.smithi155.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3794-g931809a9-1jammy [33.9 kB] 2024-05-17T21:54:02.053 INFO:teuthology.orchestra.run.smithi155.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3794-g931809a9-1jammy [23.4 kB] 2024-05-17T21:54:02.054 INFO:teuthology.orchestra.run.smithi155.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3794-g931809a9-1jammy [38.9 kB] 2024-05-17T21:54:02.056 INFO:teuthology.orchestra.run.smithi155.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3794-g931809a9-1jammy [17.4 MB] 2024-05-17T21:54:02.292 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package zip. 2024-05-17T21:54:02.318 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-17T21:54:02.358 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking zip (3.0-12build2) ... 2024-05-17T21:54:02.578 INFO:teuthology.orchestra.run.smithi155.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.012458-debug/931809a9e5d295bd61984693cac44fe131ded4bc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3794-g931809a9-1jammy [92.6 kB] 2024-05-17T21:54:02.687 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package unzip. 2024-05-17T21:54:02.713 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-17T21:54:02.753 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-17T21:54:03.107 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package luarocks. 2024-05-17T21:54:03.134 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-17T21:54:03.174 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-17T21:54:03.194 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 186 MB in 19s (9,953 kB/s) 2024-05-17T21:54:03.334 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyperclip. 2024-05-17T21:54:03.432 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-17T21:54:03.438 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-05-17T21:54:03.490 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-05-17T21:54:03.549 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package librgw2. 2024-05-17T21:54:03.575 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../027-librgw2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:03.615 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking librgw2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:03.760 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-17T21:54:03.786 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-17T21:54:03.817 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-17T21:54:04.087 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cmd2. 2024-05-17T21:54:04.114 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-05-17T21:54:04.153 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-05-17T21:54:04.376 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rgw. 2024-05-17T21:54:04.391 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:04.434 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:04.532 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-17T21:54:04.558 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-17T21:54:04.632 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-17T21:54:04.788 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-17T21:54:04.814 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-17T21:54:04.854 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-17T21:54:04.951 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-17T21:54:04.978 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-17T21:54:05.018 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-17T21:54:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libradosstriper1. 2024-05-17T21:54:05.150 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:05.190 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libradosstriper1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:05.329 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-17T21:54:05.356 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-17T21:54:05.396 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-17T21:54:05.619 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-common. 2024-05-17T21:54:05.645 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:05.694 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:05.833 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-17T21:54:05.859 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:54:05.907 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:54:06.286 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-17T21:54:06.312 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:54:06.352 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:54:06.672 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-17T21:54:06.698 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-17T21:54:06.738 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:54:07.100 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-17T21:54:07.126 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-17T21:54:07.166 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-17T21:54:07.466 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-base. 2024-05-17T21:54:07.469 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librados2. 2024-05-17T21:54:07.492 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:07.495 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../010-librados2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:07.527 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librados2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:07.543 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-base (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:07.963 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librbd1. 2024-05-17T21:54:07.989 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../011-librbd1_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:08.029 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librbd1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:08.096 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-17T21:54:08.122 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-17T21:54:08.162 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-17T21:54:08.465 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cheroot. 2024-05-17T21:54:08.491 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-17T21:54:08.531 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-17T21:54:08.682 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs2. 2024-05-17T21:54:08.709 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:08.749 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:08.862 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-17T21:54:08.886 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-17T21:54:08.926 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-17T21:54:09.035 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rados. 2024-05-17T21:54:09.050 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:09.093 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rados (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:09.204 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-17T21:54:09.220 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-17T21:54:09.254 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-17T21:54:09.395 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-17T21:54:09.422 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:09.453 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-argparse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:09.524 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-17T21:54:09.551 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-17T21:54:09.591 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-17T21:54:09.673 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cephfs. 2024-05-17T21:54:09.699 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:09.748 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cephfs (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:09.952 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempora. 2024-05-17T21:54:09.978 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-17T21:54:10.027 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-17T21:54:10.059 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-17T21:54:10.085 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:10.125 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:10.405 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-portend. 2024-05-17T21:54:10.431 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-17T21:54:10.444 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-prettytable. 2024-05-17T21:54:10.471 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-05-17T21:54:10.480 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-17T21:54:10.511 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-17T21:54:10.772 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rbd. 2024-05-17T21:54:10.791 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-17T21:54:10.798 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:10.817 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-17T21:54:10.838 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rbd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:10.858 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-17T21:54:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-17T21:54:11.200 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-17T21:54:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-17T21:54:11.226 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-17T21:54:11.252 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-17T21:54:11.274 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-17T21:54:11.544 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-17T21:54:11.559 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-17T21:54:11.593 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-17T21:54:11.623 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-natsort. 2024-05-17T21:54:11.650 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-17T21:54:11.689 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-17T21:54:11.846 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-17T21:54:11.862 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-17T21:54:11.896 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-17T21:54:11.985 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-logutils. 2024-05-17T21:54:12.011 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-17T21:54:12.042 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-17T21:54:12.249 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua5.1. 2024-05-17T21:54:12.276 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-17T21:54:12.316 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-17T21:54:12.329 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-17T21:54:12.356 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-17T21:54:12.396 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-17T21:54:12.644 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-any. 2024-05-17T21:54:12.670 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-05-17T21:54:12.699 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-mako. 2024-05-17T21:54:12.710 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-17T21:54:12.725 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-17T21:54:12.766 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-17T21:54:12.946 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package zip. 2024-05-17T21:54:12.961 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-05-17T21:54:12.996 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking zip (3.0-12build2) ... 2024-05-17T21:54:13.102 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-17T21:54:13.129 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-17T21:54:13.168 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-17T21:54:13.299 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package unzip. 2024-05-17T21:54:13.325 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-17T21:54:13.366 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-17T21:54:13.438 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-17T21:54:13.454 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-17T21:54:13.488 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-17T21:54:13.816 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webob. 2024-05-17T21:54:13.842 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-17T21:54:13.882 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-17T21:54:13.911 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package luarocks. 2024-05-17T21:54:13.937 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-17T21:54:13.977 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-17T21:54:14.236 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-waitress. 2024-05-17T21:54:14.262 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-17T21:54:14.305 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-17T21:54:14.343 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librgw2. 2024-05-17T21:54:14.369 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../027-librgw2_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:14.409 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librgw2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:14.597 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-tempita. 2024-05-17T21:54:14.624 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-17T21:54:14.663 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-17T21:54:14.941 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-paste. 2024-05-17T21:54:14.968 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-17T21:54:15.008 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-17T21:54:15.103 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rgw. 2024-05-17T21:54:15.118 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:15.161 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:15.328 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-17T21:54:15.353 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-17T21:54:15.386 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-17T21:54:15.498 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-17T21:54:15.524 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-17T21:54:15.564 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-17T21:54:15.673 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-17T21:54:15.700 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-17T21:54:15.739 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-17T21:54:15.892 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libradosstriper1. 2024-05-17T21:54:15.918 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:15.959 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libradosstriper1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:16.042 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-webtest. 2024-05-17T21:54:16.069 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-17T21:54:16.109 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-17T21:54:16.421 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pecan. 2024-05-17T21:54:16.447 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-17T21:54:16.487 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-17T21:54:16.825 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-certifi. 2024-05-17T21:54:16.852 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-17T21:54:16.891 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-17T21:54:17.062 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-common. 2024-05-17T21:54:17.089 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:17.128 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:18.087 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-idna. 2024-05-17T21:54:18.115 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-17T21:54:18.154 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-17T21:54:18.423 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-urllib3. 2024-05-17T21:54:18.450 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-17T21:54:18.481 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-17T21:54:18.667 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-base. 2024-05-17T21:54:18.693 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:18.734 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-base (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:18.843 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests. 2024-05-17T21:54:18.871 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-17T21:54:18.918 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-17T21:54:19.263 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-17T21:54:19.290 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-17T21:54:19.329 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-17T21:54:19.482 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-17T21:54:19.506 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-17T21:54:19.546 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-17T21:54:19.674 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-17T21:54:19.701 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:19.749 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:19.872 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cheroot. 2024-05-17T21:54:19.898 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-17T21:54:19.940 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-17T21:54:20.113 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-17T21:54:20.140 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:20.179 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:20.260 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-17T21:54:20.286 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-17T21:54:20.318 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-17T21:54:20.502 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-17T21:54:20.526 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-17T21:54:20.565 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-17T21:54:20.587 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-17T21:54:20.613 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-17T21:54:20.653 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-17T21:54:20.868 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-openssl. 2024-05-17T21:54:20.895 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-17T21:54:20.935 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-17T21:54:20.939 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-17T21:54:20.966 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-17T21:54:21.006 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-17T21:54:21.238 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr. 2024-05-17T21:54:21.265 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:21.296 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:21.309 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempora. 2024-05-17T21:54:21.335 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-05-17T21:54:21.375 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-17T21:54:21.645 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-portend. 2024-05-17T21:54:21.660 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-05-17T21:54:21.694 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-17T21:54:21.741 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mon. 2024-05-17T21:54:21.768 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:21.808 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mon (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:21.997 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-17T21:54:22.024 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-17T21:54:22.064 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-17T21:54:22.375 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-17T21:54:22.402 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-17T21:54:22.420 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-osd. 2024-05-17T21:54:22.443 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-17T21:54:22.447 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:22.703 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-osd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:22.753 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-natsort. 2024-05-17T21:54:22.780 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-05-17T21:54:22.820 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-17T21:54:23.131 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-logutils. 2024-05-17T21:54:23.157 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-05-17T21:54:23.197 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-17T21:54:23.500 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-17T21:54:23.527 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-17T21:54:23.567 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-17T21:54:23.840 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph. 2024-05-17T21:54:23.855 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../068-ceph_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:23.886 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-mako. 2024-05-17T21:54:23.898 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:23.913 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-17T21:54:23.953 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-17T21:54:24.226 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-fuse. 2024-05-17T21:54:24.239 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-17T21:54:24.252 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:24.265 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-17T21:54:24.292 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:24.297 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-17T21:54:24.608 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-17T21:54:24.635 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-17T21:54:24.675 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-17T21:54:24.679 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mds. 2024-05-17T21:54:24.706 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:24.745 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mds (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:24.994 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webob. 2024-05-17T21:54:25.021 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-17T21:54:25.061 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-17T21:54:25.140 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephadm. 2024-05-17T21:54:25.167 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../071-cephadm_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:25.206 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:25.413 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-waitress. 2024-05-17T21:54:25.440 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-17T21:54:25.482 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-17T21:54:25.551 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-17T21:54:25.578 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-17T21:54:25.618 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-17T21:54:25.791 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempita. 2024-05-17T21:54:25.818 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-17T21:54:25.858 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-17T21:54:25.929 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-17T21:54:25.956 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-17T21:54:25.996 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-17T21:54:26.135 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-paste. 2024-05-17T21:54:26.162 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-17T21:54:26.202 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-17T21:54:26.522 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-17T21:54:26.548 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-17T21:54:26.588 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-17T21:54:26.866 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-17T21:54:26.892 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-17T21:54:26.932 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-17T21:54:27.017 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-babel. 2024-05-17T21:54:27.045 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-17T21:54:27.084 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-17T21:54:27.235 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webtest. 2024-05-17T21:54:27.262 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-05-17T21:54:27.302 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-17T21:54:27.404 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-jinja2. 2024-05-17T21:54:27.430 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-17T21:54:27.479 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-17T21:54:27.638 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pecan. 2024-05-17T21:54:27.664 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-17T21:54:27.704 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-17T21:54:27.823 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-17T21:54:27.850 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:27.890 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:28.051 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-certifi. 2024-05-17T21:54:28.077 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-05-17T21:54:28.117 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-17T21:54:28.235 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-17T21:54:28.262 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-17T21:54:28.301 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-17T21:54:28.445 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-idna. 2024-05-17T21:54:28.472 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-05-17T21:54:28.511 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-17T21:54:28.613 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-routes. 2024-05-17T21:54:28.639 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-17T21:54:28.679 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-17T21:54:28.797 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-urllib3. 2024-05-17T21:54:28.824 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-17T21:54:28.863 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-17T21:54:29.016 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-17T21:54:29.043 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:29.082 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:29.159 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-requests. 2024-05-17T21:54:29.185 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-17T21:54:29.225 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-17T21:54:29.536 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-17T21:54:29.563 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-17T21:54:29.756 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-17T21:54:30.056 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-17T21:54:30.082 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:30.122 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-17T21:54:30.395 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-17T21:54:30.437 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-17T21:54:30.518 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-17T21:54:30.545 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:30.585 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:30.862 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-17T21:54:30.889 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-17T21:54:30.925 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-joblib. 2024-05-17T21:54:30.929 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-17T21:54:30.952 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-17T21:54:30.991 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-17T21:54:31.232 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-openssl. 2024-05-17T21:54:31.258 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-05-17T21:54:31.298 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-17T21:54:31.336 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-17T21:54:31.363 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-17T21:54:31.505 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-17T21:54:31.653 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr. 2024-05-17T21:54:31.680 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:31.719 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:31.817 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-sklearn. 2024-05-17T21:54:31.843 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-17T21:54:31.883 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-17T21:54:32.072 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mon. 2024-05-17T21:54:32.099 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:32.139 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mon (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:32.643 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-17T21:54:32.670 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:32.701 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:32.734 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-osd. 2024-05-17T21:54:32.761 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:32.800 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-osd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:33.871 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-cachetools. 2024-05-17T21:54:33.878 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph. 2024-05-17T21:54:33.894 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../068-ceph_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:33.898 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-17T21:54:33.929 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-17T21:54:33.936 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:34.216 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-17T21:54:34.222 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-fuse. 2024-05-17T21:54:34.243 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-17T21:54:34.249 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:34.280 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:34.282 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-17T21:54:34.602 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-17T21:54:34.625 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mds. 2024-05-17T21:54:34.629 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-17T21:54:34.652 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:34.669 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-17T21:54:34.691 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mds (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:34.989 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-rsa. 2024-05-17T21:54:35.016 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-17T21:54:35.055 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-17T21:54:35.120 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephadm. 2024-05-17T21:54:35.146 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../071-cephadm_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:35.186 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:35.343 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-google-auth. 2024-05-17T21:54:35.370 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-17T21:54:35.409 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-17T21:54:35.530 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-17T21:54:35.557 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-17T21:54:35.597 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-17T21:54:35.687 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-17T21:54:35.714 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-17T21:54:35.745 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-17T21:54:35.924 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-17T21:54:35.940 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-17T21:54:35.982 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-17T21:54:36.065 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-websocket. 2024-05-17T21:54:36.092 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-17T21:54:36.132 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-17T21:54:36.452 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-17T21:54:36.479 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-17T21:54:36.518 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-17T21:54:37.021 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-babel. 2024-05-17T21:54:37.048 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-17T21:54:37.087 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-17T21:54:37.158 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-17T21:54:37.174 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:37.216 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:37.415 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jinja2. 2024-05-17T21:54:37.441 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-17T21:54:37.481 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-17T21:54:37.502 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-17T21:54:37.529 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-17T21:54:37.674 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-17T21:54:37.951 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-17T21:54:37.978 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:38.017 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:38.049 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-17T21:54:38.076 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-17T21:54:38.115 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-17T21:54:38.354 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-17T21:54:38.360 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package jq. 2024-05-17T21:54:38.381 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-05-17T21:54:38.387 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-17T21:54:38.418 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-17T21:54:38.420 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-17T21:54:38.654 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package socat. 2024-05-17T21:54:38.670 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-17T21:54:38.712 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-17T21:54:38.723 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-routes. 2024-05-17T21:54:38.750 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-17T21:54:38.789 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-17T21:54:39.041 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package xmlstarlet. 2024-05-17T21:54:39.056 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-17T21:54:39.099 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-17T21:54:39.142 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-17T21:54:39.169 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:39.217 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:39.445 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-test. 2024-05-17T21:54:39.461 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:39.495 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-test (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:40.638 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-17T21:54:40.654 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-17T21:54:40.697 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-17T21:54:41.192 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-joblib. 2024-05-17T21:54:41.219 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-17T21:54:41.258 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-17T21:54:41.587 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-17T21:54:41.614 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-17T21:54:41.722 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-17T21:54:41.900 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ceph-volume. 2024-05-17T21:54:41.927 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:41.966 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ceph-volume (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:42.024 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn. 2024-05-17T21:54:42.051 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-17T21:54:42.091 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-17T21:54:42.320 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-17T21:54:42.348 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:42.386 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-mirror (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:42.648 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-colorama. 2024-05-17T21:54:42.675 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-17T21:54:42.723 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-17T21:54:42.774 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-17T21:54:42.803 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:42.841 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:42.984 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-shell. 2024-05-17T21:54:43.000 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:43.042 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-shell (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:43.311 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package cephfs-top. 2024-05-17T21:54:43.335 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:43.378 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking cephfs-top (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:43.623 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-17T21:54:43.638 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:43.672 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libcephfs-dev (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:43.819 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cachetools. 2024-05-17T21:54:43.846 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-05-17T21:54:43.885 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-17T21:54:43.992 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-17T21:54:44.020 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-17T21:54:44.058 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-17T21:54:44.188 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-17T21:54:44.215 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-17T21:54:44.246 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-17T21:54:44.388 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-17T21:54:44.416 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-17T21:54:44.454 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-17T21:54:44.516 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-17T21:54:44.532 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-17T21:54:44.574 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-17T21:54:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package nvme-cli. 2024-05-17T21:54:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-17T21:54:44.858 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-17T21:54:44.877 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rsa. 2024-05-17T21:54:44.904 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-05-17T21:54:44.943 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-17T21:54:45.256 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pkg-config. 2024-05-17T21:54:45.264 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-google-auth. 2024-05-17T21:54:45.283 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-17T21:54:45.291 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-05-17T21:54:45.322 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-17T21:54:45.330 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-17T21:54:45.584 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-17T21:54:45.600 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-17T21:54:45.641 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-17T21:54:45.666 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-17T21:54:45.694 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-17T21:54:45.733 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-17T21:54:46.053 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-websocket. 2024-05-17T21:54:46.070 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-17T21:54:46.080 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-05-17T21:54:46.097 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-17T21:54:46.119 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-17T21:54:46.136 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-17T21:54:46.397 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-17T21:54:46.421 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-17T21:54:46.439 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pastescript. 2024-05-17T21:54:46.464 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-17T21:54:46.467 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-17T21:54:46.505 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-17T21:54:46.818 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pluggy. 2024-05-17T21:54:46.845 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-17T21:54:46.884 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-17T21:54:47.146 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-psutil. 2024-05-17T21:54:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-17T21:54:47.169 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:47.173 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-17T21:54:47.204 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-17T21:54:47.210 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:47.515 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-py. 2024-05-17T21:54:47.530 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-17T21:54:47.543 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-17T21:54:47.557 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-17T21:54:47.581 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-17T21:54:47.596 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-17T21:54:47.869 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pygments. 2024-05-17T21:54:47.882 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-17T21:54:47.896 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-17T21:54:47.910 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-17T21:54:47.926 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-17T21:54:47.949 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-17T21:54:48.193 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package jq. 2024-05-17T21:54:48.209 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-17T21:54:48.252 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-17T21:54:48.423 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-17T21:54:48.450 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-17T21:54:48.489 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-17T21:54:48.506 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package socat. 2024-05-17T21:54:48.520 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-17T21:54:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-17T21:54:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-toml. 2024-05-17T21:54:48.837 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-17T21:54:48.857 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package xmlstarlet. 2024-05-17T21:54:48.872 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-17T21:54:48.875 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-17T21:54:48.915 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-17T21:54:49.278 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-test. 2024-05-17T21:54:49.305 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:49.344 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-test (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:49.420 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-pytest. 2024-05-17T21:54:49.448 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-17T21:54:49.486 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-17T21:54:49.790 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package python3-simplejson. 2024-05-17T21:54:49.818 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-17T21:54:49.848 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-17T21:54:50.118 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-17T21:54:50.145 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-17T21:54:50.184 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-17T21:54:50.914 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package radosgw. 2024-05-17T21:54:50.941 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../122-radosgw_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:50.980 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking radosgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:51.856 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-volume. 2024-05-17T21:54:51.884 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:51.922 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-volume (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:52.075 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package rbd-fuse. 2024-05-17T21:54:52.093 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:52.133 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking rbd-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:52.318 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-mirror. 2024-05-17T21:54:52.346 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:52.385 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-mirror (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:52.447 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-17T21:54:52.688 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-colorama. 2024-05-17T21:54:52.696 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-17T21:54:52.716 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-05-17T21:54:52.754 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-05-17T21:54:52.822 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-17T21:54:52.989 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-shell. 2024-05-17T21:54:53.006 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:53.039 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-shell (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:53.092 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-17T21:54:53.300 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephfs-top. 2024-05-17T21:54:53.316 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3794-g931809a9-1jammy_all.deb ... 2024-05-17T21:54:53.358 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephfs-top (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:53.421 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-17T21:54:53.619 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-17T21:54:53.635 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:54:53.677 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs-dev (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:53.878 INFO:teuthology.orchestra.run.smithi083.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-17T21:54:53.949 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:53.988 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-17T21:54:54.016 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-17T21:54:54.055 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-17T21:54:54.359 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-17T21:54:54.387 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-05-17T21:54:54.442 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-17T21:54:54.483 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user cephadm....done 2024-05-17T21:54:54.631 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-17T21:54:54.804 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package nvme-cli. 2024-05-17T21:54:54.832 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-17T21:54:54.870 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-17T21:54:54.899 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-17T21:54:55.150 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-17T21:54:55.267 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package pkg-config. 2024-05-17T21:54:55.276 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-17T21:54:55.295 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-17T21:54:55.333 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-17T21:54:55.520 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-17T21:54:55.603 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-17T21:54:55.631 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-17T21:54:55.669 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-17T21:54:55.779 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-17T21:54:55.888 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-17T21:54:56.089 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-17T21:54:56.117 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-17T21:54:56.155 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-17T21:54:56.182 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-17T21:54:56.458 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastescript. 2024-05-17T21:54:56.486 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-05-17T21:54:56.524 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-17T21:54:56.542 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-17T21:54:56.810 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-17T21:54:56.853 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pluggy. 2024-05-17T21:54:56.881 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-17T21:54:56.919 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-17T21:54:57.053 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-17T21:54:57.239 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-psutil. 2024-05-17T21:54:57.267 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-17T21:54:57.305 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-17T21:54:57.329 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-17T21:54:57.573 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-17T21:54:57.641 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-py. 2024-05-17T21:54:57.669 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-05-17T21:54:57.716 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-17T21:54:57.816 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-17T21:54:58.045 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pygments. 2024-05-17T21:54:58.073 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-17T21:54:58.083 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-17T21:54:58.111 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-17T21:54:58.418 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-argparse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:54:58.590 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-17T21:54:58.618 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-17T21:54:58.656 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-17T21:54:58.677 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-17T21:54:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-17T21:54:58.913 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-17T21:54:58.942 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-toml. 2024-05-17T21:54:58.970 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-05-17T21:54:59.008 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-17T21:54:59.164 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-17T21:54:59.290 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-17T21:54:59.320 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pytest. 2024-05-17T21:54:59.348 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-17T21:54:59.386 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-17T21:54:59.541 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-17T21:54:59.714 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplejson. 2024-05-17T21:54:59.742 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-17T21:54:59.781 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-17T21:55:00.008 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-17T21:55:00.024 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-17T21:55:00.058 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-17T21:55:00.213 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-17T21:55:00.248 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-17T21:55:00.311 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-17T21:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-17T21:55:00.779 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package radosgw. 2024-05-17T21:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-17T21:55:00.807 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../122-radosgw_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:55:00.845 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking radosgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-17T21:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-17T21:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-17T21:55:01.752 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up zip (3.0-12build2) ... 2024-05-17T21:55:01.895 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-17T21:55:01.973 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package rbd-fuse. 2024-05-17T21:55:01.989 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3794-g931809a9-1jammy_amd64.deb ... 2024-05-17T21:55:02.031 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking rbd-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:02.343 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-17T21:55:02.577 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-17T21:55:02.580 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-17T21:55:02.703 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-17T21:55:02.840 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-17T21:55:02.967 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-17T21:55:02.974 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-17T21:55:03.084 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-17T21:55:03.284 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-17T21:55:03.385 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-17T21:55:03.713 INFO:teuthology.orchestra.run.smithi155.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-17T21:55:03.753 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-17T21:55:03.780 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:04.004 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-17T21:55:04.246 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user cephadm....done 2024-05-17T21:55:04.381 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-17T21:55:04.411 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-17T21:55:04.690 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-17T21:55:04.695 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-17T21:55:04.946 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-17T21:55:05.072 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-17T21:55:05.134 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-17T21:55:05.323 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-17T21:55:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-17T21:55:05.583 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-17T21:55:05.684 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-17T21:55:05.711 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-17T21:55:05.837 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:05.994 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-17T21:55:06.147 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-idna (3.3-1) ... 2024-05-17T21:55:06.346 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-17T21:55:06.464 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-17T21:55:06.597 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-05-17T21:55:06.881 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-17T21:55:07.158 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-17T21:55:07.451 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-17T21:55:07.654 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-17T21:55:07.693 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-17T21:55:07.906 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:07.944 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-17T21:55:08.049 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-17T21:55:08.204 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-argparse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:08.300 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-17T21:55:08.401 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-17T21:55:08.479 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-17T21:55:08.510 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-17T21:55:08.605 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-17T21:55:08.731 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-05-17T21:55:08.804 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-17T21:55:08.982 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-17T21:55:09.055 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-17T21:55:09.091 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-17T21:55:09.306 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:09.342 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-17T21:55:09.432 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-17T21:55:09.726 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-17T21:55:09.882 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-17T21:55:09.933 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-17T21:55:09.960 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-17T21:55:09.996 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-17T21:55:10.122 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-17T21:55:10.211 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-17T21:55:10.470 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-17T21:55:10.489 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-17T21:55:10.615 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-17T21:55:10.721 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-17T21:55:10.891 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-17T21:55:10.997 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-17T21:55:11.098 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-17T21:55:11.150 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-17T21:55:11.375 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-17T21:55:11.460 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up zip (3.0-12build2) ... 2024-05-17T21:55:11.476 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-17T21:55:11.586 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-17T21:55:11.744 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-17T21:55:12.045 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-17T21:55:12.304 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-17T21:55:12.313 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-17T21:55:12.430 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-17T21:55:12.557 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-17T21:55:12.572 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-17T21:55:12.655 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-17T21:55:12.698 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-17T21:55:12.824 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-17T21:55:12.883 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-17T21:55:13.010 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-17T21:55:13.108 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-17T21:55:13.394 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-17T21:55:13.501 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-17T21:55:13.654 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-17T21:55:13.744 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-17T21:55:13.780 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-17T21:55:14.039 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:14.121 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-17T21:55:14.165 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-17T21:55:14.413 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-05-17T21:55:14.475 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-17T21:55:14.773 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-17T21:55:14.776 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-17T21:55:15.061 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-17T21:55:15.124 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-17T21:55:15.314 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-17T21:55:15.358 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-17T21:55:15.484 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:15.565 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-17T21:55:15.785 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-idna (3.3-1) ... 2024-05-17T21:55:15.825 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-17T21:55:16.101 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-17T21:55:16.103 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-17T21:55:16.462 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-17T21:55:16.750 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-17T21:55:17.066 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-17T21:55:17.192 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librados2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:17.300 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-17T21:55:17.318 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:17.444 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-17T21:55:17.562 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:17.696 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-17T21:55:17.780 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:17.890 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libradosstriper1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:17.956 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-17T21:55:17.999 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-17T21:55:18.082 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-17T21:55:18.183 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-17T21:55:18.251 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librbd1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:18.360 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-modules-core (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:18.459 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-17T21:55:18.487 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:18.702 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-17T21:55:18.702 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-17T21:55:18.710 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-17T21:55:18.970 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:19.096 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-17T21:55:19.246 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libcephfs-dev (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:19.373 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rados (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:19.397 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-17T21:55:19.566 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up librgw2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:19.648 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-17T21:55:19.684 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-top (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:19.891 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-17T21:55:20.150 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-17T21:55:20.177 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-17T21:55:20.393 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-17T21:55:20.660 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-17T21:55:20.795 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-17T21:55:21.071 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-17T21:55:21.197 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-17T21:55:21.389 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rbd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:21.498 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-17T21:55:21.515 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up rbd-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:21.642 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-rgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:21.742 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up python3-cephfs (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:21.816 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-17T21:55:21.843 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-shell (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:21.961 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:22.067 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-17T21:55:22.176 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-17T21:55:22.302 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-17T21:55:22.390 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-17T21:55:22.495 INFO:teuthology.orchestra.run.smithi083.stdout:Adding group ceph....done 2024-05-17T21:55:22.637 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-17T21:55:22.763 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-17T21:55:22.815 INFO:teuthology.orchestra.run.smithi083.stdout:Adding system user ceph....done 2024-05-17T21:55:23.008 INFO:teuthology.orchestra.run.smithi083.stdout:Setting system user ceph properties....done 2024-05-17T21:55:23.019 INFO:teuthology.orchestra.run.smithi083.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-17T21:55:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-17T21:55:23.155 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-17T21:55:23.399 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-17T21:55:23.510 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-17T21:55:23.525 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-17T21:55:23.784 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-17T21:55:23.910 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-17T21:55:23.996 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up cephfs-mirror (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:24.203 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-17T21:55:24.203 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-17T21:55:24.236 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-17T21:55:24.537 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-17T21:55:24.698 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-test (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up radosgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:24.839 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-17T21:55:25.017 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-17T21:55:25.284 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-17T21:55:25.299 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:55:25.299 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:55:25.552 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-17T21:55:25.812 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-17T21:55:25.824 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-base (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:26.189 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-17T21:55:26.317 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-17T21:55:26.788 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-17T21:55:26.815 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mds (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:27.033 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-17T21:55:27.033 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-17T21:55:27.151 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-17T21:55:27.277 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librados2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:27.411 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:27.492 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:27.537 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-17T21:55:27.696 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-17T21:55:27.696 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-17T21:55:27.849 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:27.991 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libradosstriper1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:28.117 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-17T21:55:28.202 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-osd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:28.394 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librbd1 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-17T21:55:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-17T21:55:28.528 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-modules-core (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:28.654 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:28.864 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-17T21:55:28.864 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-17T21:55:28.954 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.079 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.230 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mon (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.322 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs-dev (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.431 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rados (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.441 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-17T21:55:29.441 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-17T21:55:29.540 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librgw2 (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.666 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-top (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:29.792 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-17T21:55:29.915 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:30.058 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:30.184 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-mgr-dashboard (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:30.310 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ceph-volume (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:30.979 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rbd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:30.986 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-17T21:55:31.097 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up rbd-fuse (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:31.106 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-17T21:55:31.206 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:31.259 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-17T21:55:31.307 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cephfs (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:31.433 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-shell (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:31.559 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-common (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:31.993 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-17T21:55:32.093 INFO:teuthology.orchestra.run.smithi155.stdout:Adding group ceph....done 2024-05-17T21:55:32.420 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user ceph....done 2024-05-17T21:55:32.580 INFO:teuthology.orchestra.run.smithi155.stdout:Setting system user ceph properties....done 2024-05-17T21:55:32.598 INFO:teuthology.orchestra.run.smithi155.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-17T21:55:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-17T21:55:32.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-17T21:55:32.684 INFO:teuthology.orchestra.run.smithi083.stdout:Reading package lists... 2024-05-17T21:55:32.734 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-17T21:55:32.886 INFO:teuthology.orchestra.run.smithi083.stdout:Building dependency tree... 2024-05-17T21:55:32.887 INFO:teuthology.orchestra.run.smithi083.stdout:Reading state information... 2024-05-17T21:55:33.061 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-17T21:55:33.080 INFO:teuthology.orchestra.run.smithi083.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-17T21:55:33.080 INFO:teuthology.orchestra.run.smithi083.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-17T21:55:33.080 INFO:teuthology.orchestra.run.smithi083.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-17T21:55:33.080 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:55:33.080 INFO:teuthology.orchestra.run.smithi083.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:55:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:55:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:55:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:55:33.081 INFO:teuthology.orchestra.run.smithi083.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:55:33.081 INFO:teuthology.orchestra.run.smithi083.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:55:33.082 INFO:teuthology.orchestra.run.smithi083.stdout:The following additional packages will be installed: 2024-05-17T21:55:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-17T21:55:33.082 INFO:teuthology.orchestra.run.smithi083.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-17T21:55:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-17T21:55:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-17T21:55:33.083 INFO:teuthology.orchestra.run.smithi083.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-17T21:55:33.084 INFO:teuthology.orchestra.run.smithi083.stdout:Suggested packages: 2024-05-17T21:55:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-17T21:55:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-17T21:55:33.084 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-17T21:55:33.128 INFO:teuthology.orchestra.run.smithi083.stdout:The following NEW packages will be installed: 2024-05-17T21:55:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-17T21:55:33.128 INFO:teuthology.orchestra.run.smithi083.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-17T21:55:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-17T21:55:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-17T21:55:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-17T21:55:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-17T21:55:33.129 INFO:teuthology.orchestra.run.smithi083.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-17T21:55:33.130 INFO:teuthology.orchestra.run.smithi083.stdout:The following packages will be upgraded: 2024-05-17T21:55:33.131 INFO:teuthology.orchestra.run.smithi083.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-17T21:55:33.223 INFO:teuthology.orchestra.run.smithi083.stdout:4 upgraded, 39 newly installed, 0 to remove and 303 not upgraded. 2024-05-17T21:55:33.223 INFO:teuthology.orchestra.run.smithi083.stdout:Need to get 55.7 MB of archives. 2024-05-17T21:55:33.223 INFO:teuthology.orchestra.run.smithi083.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-17T21:55:33.223 INFO:teuthology.orchestra.run.smithi083.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-17T21:55:33.384 INFO:teuthology.orchestra.run.smithi083.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-17T21:55:33.391 INFO:teuthology.orchestra.run.smithi083.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-17T21:55:33.431 INFO:teuthology.orchestra.run.smithi083.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-17T21:55:33.431 INFO:teuthology.orchestra.run.smithi083.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-17T21:55:33.432 INFO:teuthology.orchestra.run.smithi083.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-17T21:55:33.443 INFO:teuthology.orchestra.run.smithi083.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-17T21:55:33.444 INFO:teuthology.orchestra.run.smithi083.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-17T21:55:33.446 INFO:teuthology.orchestra.run.smithi083.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-17T21:55:33.448 INFO:teuthology.orchestra.run.smithi083.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-17T21:55:33.448 INFO:teuthology.orchestra.run.smithi083.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-17T21:55:33.450 INFO:teuthology.orchestra.run.smithi083.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-17T21:55:33.452 INFO:teuthology.orchestra.run.smithi083.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-17T21:55:33.472 INFO:teuthology.orchestra.run.smithi083.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-17T21:55:33.472 INFO:teuthology.orchestra.run.smithi083.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-17T21:55:33.473 INFO:teuthology.orchestra.run.smithi083.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-17T21:55:33.552 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephfs-mirror (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:33.761 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-17T21:55:33.761 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-05-17T21:55:33.821 INFO:teuthology.orchestra.run.smithi083.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-17T21:55:33.823 INFO:teuthology.orchestra.run.smithi083.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-17T21:55:33.823 INFO:teuthology.orchestra.run.smithi083.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-17T21:55:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-17T21:55:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-17T21:55:33.831 INFO:teuthology.orchestra.run.smithi083.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-17T21:55:33.833 INFO:teuthology.orchestra.run.smithi083.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-17T21:55:33.840 INFO:teuthology.orchestra.run.smithi083.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-17T21:55:33.840 INFO:teuthology.orchestra.run.smithi083.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-17T21:55:33.862 INFO:teuthology.orchestra.run.smithi083.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-17T21:55:33.863 INFO:teuthology.orchestra.run.smithi083.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-17T21:55:33.874 INFO:teuthology.orchestra.run.smithi083.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-17T21:55:33.900 INFO:teuthology.orchestra.run.smithi083.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-17T21:55:33.906 INFO:teuthology.orchestra.run.smithi083.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-17T21:55:33.907 INFO:teuthology.orchestra.run.smithi083.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-17T21:55:33.907 INFO:teuthology.orchestra.run.smithi083.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-17T21:55:33.908 INFO:teuthology.orchestra.run.smithi083.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-17T21:55:33.919 INFO:teuthology.orchestra.run.smithi083.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-17T21:55:33.921 INFO:teuthology.orchestra.run.smithi083.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-17T21:55:34.132 INFO:teuthology.orchestra.run.smithi083.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-17T21:55:34.148 INFO:teuthology.orchestra.run.smithi083.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-17T21:55:34.150 INFO:teuthology.orchestra.run.smithi083.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-17T21:55:34.153 INFO:teuthology.orchestra.run.smithi083.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-17T21:55:34.155 INFO:teuthology.orchestra.run.smithi083.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-17T21:55:34.159 INFO:teuthology.orchestra.run.smithi083.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-17T21:55:34.174 INFO:teuthology.orchestra.run.smithi083.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-17T21:55:34.183 INFO:teuthology.orchestra.run.smithi083.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-17T21:55:34.253 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-test (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:34.379 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up radosgw (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:34.586 INFO:teuthology.orchestra.run.smithi083.stdout:Preconfiguring packages ... 2024-05-17T21:55:34.753 INFO:teuthology.orchestra.run.smithi083.stdout:Fetched 55.7 MB in 1s (53.2 MB/s) 2024-05-17T21:55:34.843 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:55:34.843 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-17T21:55:35.071 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173986 files and directories currently installed.) 2024-05-17T21:55:35.076 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:35.224 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:35.297 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-base (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:35.627 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-17T21:55:35.853 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-17T21:55:36.099 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mds (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:36.332 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-17T21:55:36.332 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-17T21:55:36.809 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:37.036 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-17T21:55:37.036 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-17T21:55:37.146 INFO:teuthology.orchestra.run.smithi083.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-17T21:55:37.425 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package flex. 2024-05-17T21:55:37.520 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-osd (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:37.528 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173986 files and directories currently installed.) 2024-05-17T21:55:37.534 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-17T21:55:37.803 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-17T21:55:37.803 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-17T21:55:38.106 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-17T21:55:38.304 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-17T21:55:38.414 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:38.540 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mon (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:38.740 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:55:38.745 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-17T21:55:38.745 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-17T21:55:39.232 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-cephadm (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:39.282 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174059 files and directories currently installed.) 2024-05-17T21:55:39.287 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:39.375 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:39.421 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:39.501 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-dashboard (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:39.652 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-volume (19.0.0-3794-g931809a9-1jammy) ... 2024-05-17T21:55:39.796 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:39.911 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:40.022 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-17T21:55:40.139 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-17T21:55:40.212 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-17T21:55:40.283 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-17T21:55:40.586 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package bison. 2024-05-17T21:55:40.690 INFO:teuthology.orchestra.run.smithi083.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174059 files and directories currently installed.) 2024-05-17T21:55:40.696 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-17T21:55:40.755 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-17T21:55:41.165 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-17T21:55:41.192 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-17T21:55:41.231 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-17T21:55:41.342 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-17T21:55:41.567 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package dump. 2024-05-17T21:55:41.595 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-17T21:55:41.634 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking dump (0.4b47-3) ... 2024-05-17T21:55:41.896 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package indent. 2024-05-17T21:55:41.912 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:41.946 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-17T21:55:42.005 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-17T21:55:42.008 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-05-17T21:55:42.108 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-05-17T21:55:42.249 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-17T21:55:42.265 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-17T21:55:42.290 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-05-17T21:55:42.291 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-05-17T21:55:42.307 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-17T21:55:42.463 INFO:teuthology.orchestra.run.smithi155.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-17T21:55:42.464 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-17T21:55:42.465 INFO:teuthology.orchestra.run.smithi155.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-17T21:55:42.465 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-17T21:55:42.465 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-05-17T21:55:42.465 INFO:teuthology.orchestra.run.smithi155.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-05-17T21:55:42.465 INFO:teuthology.orchestra.run.smithi155.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-05-17T21:55:42.466 INFO:teuthology.orchestra.run.smithi155.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-05-17T21:55:42.466 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-05-17T21:55:42.466 INFO:teuthology.orchestra.run.smithi155.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-17T21:55:42.467 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-05-17T21:55:42.467 INFO:teuthology.orchestra.run.smithi155.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-05-17T21:55:42.467 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-05-17T21:55:42.467 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-05-17T21:55:42.506 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-05-17T21:55:42.507 INFO:teuthology.orchestra.run.smithi155.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-05-17T21:55:42.507 INFO:teuthology.orchestra.run.smithi155.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-05-17T21:55:42.507 INFO:teuthology.orchestra.run.smithi155.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-05-17T21:55:42.507 INFO:teuthology.orchestra.run.smithi155.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-05-17T21:55:42.507 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-05-17T21:55:42.508 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-05-17T21:55:42.508 INFO:teuthology.orchestra.run.smithi155.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-05-17T21:55:42.509 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-05-17T21:55:42.509 INFO:teuthology.orchestra.run.smithi155.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-05-17T21:55:42.566 INFO:teuthology.orchestra.run.smithi155.stdout:4 upgraded, 39 newly installed, 0 to remove and 303 not upgraded. 2024-05-17T21:55:42.566 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 55.7 MB of archives. 2024-05-17T21:55:42.566 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 202 MB of additional disk space will be used. 2024-05-17T21:55:42.566 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-05-17T21:55:42.719 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-05-17T21:55:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-17T21:55:42.728 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-05-17T21:55:42.748 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-17T21:55:42.770 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-05-17T21:55:42.771 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-05-17T21:55:42.772 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-05-17T21:55:42.782 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-05-17T21:55:42.782 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-05-17T21:55:42.785 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-05-17T21:55:42.786 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-05-17T21:55:42.786 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-05-17T21:55:42.788 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-05-17T21:55:42.791 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-05-17T21:55:42.795 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-17T21:55:42.809 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-05-17T21:55:42.809 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-05-17T21:55:42.811 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-05-17T21:55:43.089 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-17T21:55:43.117 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-17T21:55:43.139 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-05-17T21:55:43.141 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-05-17T21:55:43.141 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-05-17T21:55:43.144 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-05-17T21:55:43.144 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-05-17T21:55:43.148 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-05-17T21:55:43.150 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-05-17T21:55:43.155 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-17T21:55:43.157 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-05-17T21:55:43.157 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-05-17T21:55:43.219 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-05-17T21:55:43.240 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-05-17T21:55:43.383 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-05-17T21:55:43.453 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-05-17T21:55:43.481 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-05-17T21:55:43.492 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-17T21:55:43.503 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-05-17T21:55:43.521 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:43.524 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-05-17T21:55:43.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-05-17T21:55:43.559 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-17T21:55:43.595 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-05-17T21:55:43.621 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-05-17T21:55:43.837 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-17T21:55:43.865 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:43.872 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-05-17T21:55:43.893 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-05-17T21:55:43.912 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-17T21:55:43.914 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-05-17T21:55:43.935 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-05-17T21:55:43.956 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-05-17T21:55:43.978 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-05-17T21:55:44.030 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-05-17T21:55:44.052 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-05-17T21:55:44.214 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libjson-perl. 2024-05-17T21:55:44.242 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-17T21:55:44.289 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-17T21:55:44.459 INFO:teuthology.orchestra.run.smithi155.stdout:Preconfiguring packages ... 2024-05-17T21:55:44.609 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-17T21:55:44.615 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 55.7 MB in 2s (36.2 MB/s) 2024-05-17T21:55:44.637 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-17T21:55:44.676 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-17T21:55:44.930 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173986 files and directories currently installed.) 2024-05-17T21:55:44.937 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:45.107 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:45.703 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-05-17T21:55:46.421 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-17T21:55:46.448 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-17T21:55:46.479 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:55:46.612 INFO:teuthology.orchestra.run.smithi155.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-05-17T21:55:46.731 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-17T21:55:46.748 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-17T21:55:46.790 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:55:46.860 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package flex. 2024-05-17T21:55:46.943 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173986 files and directories currently installed.) 2024-05-17T21:55:46.949 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:46.992 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking flex (2.6.4-8build2) ... 2024-05-17T21:55:47.101 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-17T21:55:47.128 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-17T21:55:47.167 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:55:47.563 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-17T21:55:47.591 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-17T21:55:47.637 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-17T21:55:47.661 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-17T21:55:47.799 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-05-17T21:55:48.040 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-17T21:55:48.068 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-17T21:55:48.115 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-17T21:55:48.228 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:55:48.468 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-17T21:55:48.496 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:48.535 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:55:48.759 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174059 files and directories currently installed.) 2024-05-17T21:55:48.765 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:48.838 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-17T21:55:48.866 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:48.890 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:48.896 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:55:49.249 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-17T21:55:49.277 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:49.299 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-05-17T21:55:49.324 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:55:49.430 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-05-17T21:55:49.601 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-17T21:55:49.617 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-17T21:55:49.659 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:55:49.758 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-17T21:55:50.139 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package bison. 2024-05-17T21:55:50.214 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-17T21:55:50.241 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-17T21:55:50.244 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174059 files and directories currently installed.) 2024-05-17T21:55:50.250 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-05-17T21:55:50.280 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-17T21:55:50.300 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-05-17T21:55:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package modemmanager. 2024-05-17T21:55:50.603 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-17T21:55:50.633 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:55:50.708 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-05-17T21:55:50.736 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-05-17T21:55:50.775 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-05-17T21:55:51.094 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package dump. 2024-05-17T21:55:51.122 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-05-17T21:55:51.161 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking dump (0.4b47-3) ... 2024-05-17T21:55:51.273 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager. 2024-05-17T21:55:51.301 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-17T21:55:51.348 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-17T21:55:51.423 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package indent. 2024-05-17T21:55:51.451 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:51.490 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-17T21:55:51.742 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-05-17T21:55:51.758 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-05-17T21:55:51.800 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-17T21:55:51.906 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package ppp. 2024-05-17T21:55:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-17T21:55:52.163 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-05-17T21:55:52.191 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-05-17T21:55:52.227 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-17T21:55:52.229 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-17T21:55:52.474 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-05-17T21:55:52.490 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-05-17T21:55:52.524 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-05-17T21:55:52.527 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package pptp-linux. 2024-05-17T21:55:52.556 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-17T21:55:52.594 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-17T21:55:52.844 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libfl2:amd64. 2024-05-17T21:55:52.872 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:52.880 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-17T21:55:52.902 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-05-17T21:55:52.908 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-17T21:55:52.947 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-17T21:55:53.155 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-05-17T21:55:53.171 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-05-17T21:55:53.214 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-17T21:55:53.224 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-17T21:55:53.240 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-17T21:55:53.274 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-common (238) ... 2024-05-17T21:55:53.458 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjson-perl. 2024-05-17T21:55:53.486 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-05-17T21:55:53.524 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-05-17T21:55:53.553 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-17T21:55:53.581 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:53.636 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:55:53.819 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libllvm14:amd64. 2024-05-17T21:55:53.835 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-05-17T21:55:53.877 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-17T21:55:54.233 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-common. 2024-05-17T21:55:54.261 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-17T21:55:54.318 INFO:teuthology.orchestra.run.smithi083.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-17T21:55:54.347 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-common (238) ... 2024-05-17T21:55:54.782 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-14. 2024-05-17T21:55:54.810 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:54.843 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:55:55.481 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-05-17T21:55:55.508 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-17T21:55:55.547 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:55:55.958 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmbim-proxy. 2024-05-17T21:55:55.974 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-05-17T21:55:56.016 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:55:56.160 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql. 2024-05-17T21:55:56.189 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-17T21:55:56.226 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql (14+238) ... 2024-05-17T21:55:56.319 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-05-17T21:55:56.347 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-17T21:55:56.386 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:55:56.470 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-client. 2024-05-17T21:55:56.495 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-17T21:55:56.545 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-client (14+238) ... 2024-05-17T21:55:56.722 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libndp0:amd64. 2024-05-17T21:55:56.750 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-05-17T21:55:56.756 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-17T21:55:56.772 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-17T21:55:56.788 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-17T21:55:56.806 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-17T21:55:57.100 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl8.6. 2024-05-17T21:55:57.116 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libnm0:amd64. 2024-05-17T21:55:57.128 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-17T21:55:57.144 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-05-17T21:55:57.166 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-17T21:55:57.183 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-17T21:55:57.469 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package tcl. 2024-05-17T21:55:57.497 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-17T21:55:57.511 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libpq5:amd64. 2024-05-17T21:55:57.539 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:57.540 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-17T21:55:57.577 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:55:57.889 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package wpasupplicant. 2024-05-17T21:55:57.917 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-17T21:55:57.980 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-17T21:55:58.326 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-17T21:55:58.343 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-17T21:55:58.349 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-05-17T21:55:58.376 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-17T21:55:58.377 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:58.415 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:55:58.709 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libqmi-proxy. 2024-05-17T21:55:58.725 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:55:58.763 INFO:teuthology.orchestra.run.smithi083.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-17T21:55:58.768 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:55:58.792 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-17T21:55:58.838 INFO:teuthology.orchestra.run.smithi083.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-17T21:55:59.003 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-05-17T21:55:59.019 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-05-17T21:55:59.061 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:55:59.134 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-common (238) ... 2024-05-17T21:55:59.344 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-17T21:55:59.470 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-17T21:55:59.565 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-05-17T21:55:59.593 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-05-17T21:55:59.596 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:55:59.624 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-05-17T21:55:59.722 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-17T21:55:59.871 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-17T21:55:59.977 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package modemmanager. 2024-05-17T21:56:00.078 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-05-17T21:56:00.078 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:56:00.124 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:00.635 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-17T21:56:00.667 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package network-manager. 2024-05-17T21:56:00.695 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-05-17T21:56:00.722 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-17T21:56:00.733 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-05-17T21:56:01.099 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-17T21:56:01.100 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-17T21:56:01.374 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ppp. 2024-05-17T21:56:01.402 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-05-17T21:56:01.625 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-17T21:56:01.679 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-05-17T21:56:01.750 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-17T21:56:01.876 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:56:02.002 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-17T21:56:02.004 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package pptp-linux. 2024-05-17T21:56:02.031 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-05-17T21:56:02.070 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-05-17T21:56:02.306 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package network-manager-pptp. 2024-05-17T21:56:02.322 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-05-17T21:56:02.356 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-05-17T21:56:02.570 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-17T21:56:02.643 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client-common. 2024-05-17T21:56:02.670 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-05-17T21:56:02.705 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-17T21:56:02.709 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client-common (238) ... 2024-05-17T21:56:02.831 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-17T21:56:02.932 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-17T21:56:03.015 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client-14. 2024-05-17T21:56:03.043 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:56:03.081 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:03.578 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-17T21:56:03.713 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up dump (0.4b47-3) ... 2024-05-17T21:56:03.807 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-17T21:56:03.865 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:56:03.913 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-common. 2024-05-17T21:56:03.941 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-05-17T21:56:03.966 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-17T21:56:03.998 INFO:teuthology.orchestra.run.smithi155.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-05-17T21:56:04.027 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-common (238) ... 2024-05-17T21:56:04.067 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-17T21:56:04.193 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-17T21:56:04.302 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-17T21:56:04.384 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-17T21:56:04.445 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-17T21:56:04.496 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-14. 2024-05-17T21:56:04.524 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-05-17T21:56:04.573 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-17T21:56:04.574 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:04.750 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-17T21:56:04.876 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:56:05.002 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-17T21:56:05.162 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-client (14+238) ... 2024-05-17T21:56:05.262 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:which means they are currently configured by ifupdown: 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:- enp3s0f1 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:remove their configuration from /etc/network/interfaces. 2024-05-17T21:56:05.738 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:05.758 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql. 2024-05-17T21:56:05.775 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-05-17T21:56:05.817 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql (14+238) ... 2024-05-17T21:56:05.851 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-17T21:56:06.044 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-client. 2024-05-17T21:56:06.072 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-05-17T21:56:06.110 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-client (14+238) ... 2024-05-17T21:56:06.172 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-17T21:56:06.363 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package postgresql-contrib. 2024-05-17T21:56:06.379 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-05-17T21:56:06.421 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking postgresql-contrib (14+238) ... 2024-05-17T21:56:06.518 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-17T21:56:06.749 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package tcl8.6. 2024-05-17T21:56:06.777 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-05-17T21:56:06.815 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-17T21:56:07.073 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:56:07.101 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package tcl. 2024-05-17T21:56:07.130 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-05-17T21:56:07.173 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-05-17T21:56:07.233 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-17T21:56:07.359 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-17T21:56:07.485 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-common (238) ... 2024-05-17T21:56:07.529 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package wpasupplicant. 2024-05-17T21:56:07.557 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-05-17T21:56:07.596 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-17T21:56:07.951 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-05-17T21:56:07.979 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-05-17T21:56:08.017 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-05-17T21:56:08.387 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package usb-modeswitch. 2024-05-17T21:56:08.415 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-05-17T21:56:08.454 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-17T21:56:08.656 INFO:teuthology.orchestra.run.smithi083.stdout:Adding user postgres to group ssl-cert 2024-05-17T21:56:08.752 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client-common (238) ... 2024-05-17T21:56:08.925 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:08.925 INFO:teuthology.orchestra.run.smithi083.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-17T21:56:08.944 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up flex (2.6.4-8build2) ... 2024-05-17T21:56:09.070 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-05-17T21:56:09.115 INFO:teuthology.orchestra.run.smithi083.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-17T21:56:09.115 INFO:teuthology.orchestra.run.smithi083.stdout:Removing obsolete dictionary files: 2024-05-17T21:56:09.196 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:09.322 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-05-17T21:56:09.448 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-05-17T21:56:09.583 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:09.684 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-17T21:56:10.092 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-05-17T21:56:10.180 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-05-17T21:56:10.228 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:10.550 INFO:teuthology.orchestra.run.smithi083.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-17T21:56:10.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-17T21:56:10.575 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-17T21:56:10.575 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:This user must also own the server process. 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:The database cluster will be initialized with locale "en_US". 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:The default text search configuration will be set to "english". 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:Data page checksums are disabled. 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-17T21:56:10.605 INFO:teuthology.orchestra.run.smithi083.stdout:creating subdirectories ... ok 2024-05-17T21:56:10.606 INFO:teuthology.orchestra.run.smithi083.stdout:selecting dynamic shared memory implementation ... posix 2024-05-17T21:56:10.620 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default max_connections ... 100 2024-05-17T21:56:10.641 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default shared_buffers ... 128MB 2024-05-17T21:56:10.651 INFO:teuthology.orchestra.run.smithi083.stdout:selecting default time zone ... Etc/UTC 2024-05-17T21:56:10.652 INFO:teuthology.orchestra.run.smithi083.stdout:creating configuration files ... ok 2024-05-17T21:56:10.765 INFO:teuthology.orchestra.run.smithi083.stdout:running bootstrap script ... ok 2024-05-17T21:56:11.067 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-05-17T21:56:11.176 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-05-17T21:56:11.302 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-05-17T21:56:11.402 INFO:teuthology.orchestra.run.smithi083.stdout:performing post-bootstrap initialization ... ok 2024-05-17T21:56:11.403 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-05-17T21:56:11.529 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-05-17T21:56:11.655 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-05-17T21:56:11.781 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-05-17T21:56:11.908 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-05-17T21:56:12.082 INFO:teuthology.orchestra.run.smithi083.stdout:syncing data to disk ... ok 2024-05-17T21:56:12.475 INFO:teuthology.orchestra.run.smithi083.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-17T21:56:12.513 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-05-17T21:56:12.639 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up dump (0.4b47-3) ... 2024-05-17T21:56:12.749 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-05-17T21:56:12.816 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:56:12.942 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-05-17T21:56:13.068 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up tcl (8.6.11+1build2) ... 2024-05-17T21:56:13.186 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjson-perl (4.04000-1) ... 2024-05-17T21:56:13.287 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-05-17T21:56:13.369 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-05-17T21:56:13.438 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-05-17T21:56:13.564 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-05-17T21:56:13.749 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-05-17T21:56:14.276 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:56:14.402 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-05-17T21:56:14.545 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-client (14+238) ... 2024-05-17T21:56:14.654 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:The following network interfaces were found in /etc/network/interfaces 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:which means they are currently configured by ifupdown: 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:- enp3s0f1 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:If you want to manage those interfaces with NetworkManager instead 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout:remove their configuration from /etc/network/interfaces. 2024-05-17T21:56:15.138 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:15.260 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:56:15.263 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-05-17T21:56:15.361 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-17T21:56:15.487 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up postgresql (14+238) ... 2024-05-17T21:56:15.592 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-05-17T21:56:15.613 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:56:15.739 INFO:teuthology.orchestra.run.smithi083.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:56:15.922 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-05-17T21:56:15.984 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-17T21:56:15.985 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-17T21:56:16.441 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-05-17T21:56:16.525 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-17T21:56:16.601 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-05-17T21:56:16.638 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-17T21:56:16.726 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-05-17T21:56:16.891 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-common (238) ... 2024-05-17T21:56:16.953 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-17T21:56:17.097 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-17T21:56:17.984 INFO:teuthology.orchestra.run.smithi155.stdout:Adding user postgres to group ssl-cert 2024-05-17T21:56:18.190 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:18.190 INFO:teuthology.orchestra.run.smithi155.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-05-17T21:56:18.414 INFO:teuthology.orchestra.run.smithi155.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-05-17T21:56:18.414 INFO:teuthology.orchestra.run.smithi155.stdout:Removing obsolete dictionary files: 2024-05-17T21:56:18.458 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-17T21:56:18.558 INFO:teuthology.orchestra.run.smithi083.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-17T21:56:18.996 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-05-17T21:56:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-17T21:56:19.052 DEBUG:teuthology.parallel:result is None 2024-05-17T21:56:19.380 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-05-17T21:56:19.701 INFO:teuthology.orchestra.run.smithi155.stdout:Creating new PostgreSQL cluster 14/main ... 2024-05-17T21:56:19.714 INFO:teuthology.orchestra.run.smithi155.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:The files belonging to this database system will be owned by user "postgres". 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:This user must also own the server process. 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:The database cluster will be initialized with locale "en_US". 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:The default text search configuration will be set to "english". 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:Data page checksums are disabled. 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:19.750 INFO:teuthology.orchestra.run.smithi155.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-05-17T21:56:19.751 INFO:teuthology.orchestra.run.smithi155.stdout:creating subdirectories ... ok 2024-05-17T21:56:19.751 INFO:teuthology.orchestra.run.smithi155.stdout:selecting dynamic shared memory implementation ... posix 2024-05-17T21:56:19.765 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default max_connections ... 100 2024-05-17T21:56:19.785 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default shared_buffers ... 128MB 2024-05-17T21:56:19.794 INFO:teuthology.orchestra.run.smithi155.stdout:selecting default time zone ... Etc/UTC 2024-05-17T21:56:19.795 INFO:teuthology.orchestra.run.smithi155.stdout:creating configuration files ... ok 2024-05-17T21:56:19.910 INFO:teuthology.orchestra.run.smithi155.stdout:running bootstrap script ... ok 2024-05-17T21:56:20.536 INFO:teuthology.orchestra.run.smithi155.stdout:performing post-bootstrap initialization ... ok 2024-05-17T21:56:21.161 INFO:teuthology.orchestra.run.smithi155.stdout:syncing data to disk ... ok 2024-05-17T21:56:21.565 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-05-17T21:56:24.320 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:56:24.437 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql-contrib (14+238) ... 2024-05-17T21:56:24.539 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up postgresql (14+238) ... 2024-05-17T21:56:24.656 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-05-17T21:56:24.782 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-05-17T21:56:25.039 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-05-17T21:56:25.039 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-05-17T21:56:25.602 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-17T21:56:25.703 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-17T21:56:25.997 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-17T21:56:26.141 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-17T21:56:27.478 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-05-17T21:56:27.577 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-17T21:56:28.070 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-17T21:56:28.075 DEBUG:teuthology.parallel:result is None 2024-05-17T21:56:28.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:56:28.297 DEBUG:teuthology.orchestra.run.smithi083:> dpkg-query -W -f '${Version}' ceph 2024-05-17T21:56:28.316 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3794-g931809a9-1jammy 2024-05-17T21:56:28.317 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794-g931809a9-1jammy 2024-05-17T21:56:28.317 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794-g931809a9-1jammy is installed. 2024-05-17T21:56:28.319 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=931809a9e5d295bd61984693cac44fe131ded4bc 2024-05-17T21:56:28.456 DEBUG:teuthology.orchestra.run.smithi155:> dpkg-query -W -f '${Version}' ceph 2024-05-17T21:56:28.475 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-3794-g931809a9-1jammy 2024-05-17T21:56:28.475 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3794-g931809a9-1jammy 2024-05-17T21:56:28.476 INFO:teuthology.task.install:The correct ceph version 19.0.0-3794-g931809a9-1jammy is installed. 2024-05-17T21:56:28.477 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-17T21:56:28.477 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:28.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T21:56:28.492 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:28.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-17T21:56:28.533 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-17T21:56:28.533 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:28.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T21:56:28.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T21:56:28.600 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:28.600 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-05-17T21:56:28.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-17T21:56:28.671 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-17T21:56:28.671 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:28.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T21:56:28.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T21:56:28.740 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:28.740 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-17T21:56:28.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-17T21:56:28.811 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-17T21:56:28.811 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:28.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T21:56:28.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T21:56:28.879 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:28.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-05-17T21:56:28.893 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-17T21:56:28.947 INFO:teuthology.run_tasks:Running task ceph... 2024-05-17T21:56:29.023 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-17T21:56:29.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-05-17T21:56:29.025 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2024-05-17T21:56:29.038 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-17T21:56:29.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-17T21:56:29.086 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-17T21:56:29.098 INFO:tasks.ceph:Creating extra log directories... 2024-05-17T21:56:29.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-17T21:56:29.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-17T21:56:29.161 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-17T21:56:29.162 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-17T21:56:29.162 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-17T21:56:29.162 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:29.162 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-17T21:56:29.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-17T21:56:29.276 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:29.276 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-17T21:56:29.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-17T21:56:29.360 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-17T21:56:29.360 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-17T21:56:29.360 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748', 'branch': 'wip-pdonnell-testing-20240517.012458-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7711748', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240517.012458-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240517.012458-debug', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1967, 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'sleep_before_teardown': 0, 'subset': '59/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240517.012458-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_931809a9e5d295bd61984693cac44fe131ded4bc/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHIJSEf2xUkpW3eHjsfvK7YtmfqgX3883DVbalL/SIYwu5MUc7EdJkXnn4TILu6qHtNPxjItao2dspgdczSK9zY=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJkjvXa8Xt41YJV9XAJXpQtbtdT07UftajZBF+cxC3bkLKNg48U6RGlNQAxXUm9pZrO4lEkA9eZOTGvMqxSJ7qc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '931809a9e5d295bd61984693cac44fe131ded4bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'install-fix', 'teuthology_sha1': '2127e87e3419439de550d2abbd7ed2aee2cdfb0e', 'timestamp': '2024-05-17_18:30:44', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-17T21:56:29.361 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-17T21:56:29.363 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-17T21:56:29.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-05-17T21:56:29.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2024-05-17T21:56:29.460 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:29.460 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T21:56:29.469 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T21:56:29.470 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-05-17T21:56:29.517 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T21:56:29.517 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-05-17 21:50:16.312668100 +0000 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-05-17 21:50:16.060670352 +0000 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-05-17 21:50:16.060670352 +0000 2024-05-17T21:56:29.518 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-05-17T21:56:29.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T21:56:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-05-17T21:56:29.571 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-05-17T21:56:29.572 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000434239 s, 1.2 MB/s 2024-05-17T21:56:29.573 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T21:56:29.622 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-05-17 21:50:16.776663956 +0000 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-05-17 21:50:16.528666171 +0000 2024-05-17T21:56:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-05-17 21:50:16.528666171 +0000 2024-05-17T21:56:29.670 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-05-17T21:56:29.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T21:56:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-05-17T21:56:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-05-17T21:56:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000412347 s, 1.2 MB/s 2024-05-17T21:56:29.724 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T21:56:29.775 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-05-17 21:50:17.240659812 +0000 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-05-17 21:50:17.004661919 +0000 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-05-17 21:50:17.004661919 +0000 2024-05-17T21:56:29.821 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-05-17T21:56:29.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T21:56:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-05-17T21:56:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-05-17T21:56:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000370644 s, 1.4 MB/s 2024-05-17T21:56:29.876 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T21:56:29.927 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-05-17T21:56:29.973 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T21:56:29.973 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:29.973 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-05-17T21:56:29.974 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:29.974 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-05-17 21:50:17.724655489 +0000 2024-05-17T21:56:29.974 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-05-17 21:50:17.472657739 +0000 2024-05-17T21:56:29.974 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-05-17 21:50:17.472657739 +0000 2024-05-17T21:56:29.974 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-05-17T21:56:29.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T21:56:30.027 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-05-17T21:56:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-05-17T21:56:30.028 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.00039876 s, 1.3 MB/s 2024-05-17T21:56:30.029 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T21:56:30.078 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:30.078 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:30.078 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-05-17T21:56:30.084 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-17T21:56:30.084 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-17 21:50:15.889820231 +0000 2024-05-17T21:56:30.132 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-17 21:50:15.885820266 +0000 2024-05-17T21:56:30.133 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-17 21:50:15.885820266 +0000 2024-05-17T21:56:30.133 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-17T21:56:30.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-17T21:56:30.186 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-17T21:56:30.186 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-17T21:56:30.186 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000442714 s, 1.2 MB/s 2024-05-17T21:56:30.188 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-17T21:56:30.237 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-05-17T21:56:30.283 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-17 21:50:16.585814021 +0000 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-17 21:50:16.353816092 +0000 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-17 21:50:16.353816092 +0000 2024-05-17T21:56:30.284 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-17T21:56:30.285 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-17T21:56:30.338 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-17T21:56:30.338 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-17T21:56:30.338 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000437436 s, 1.2 MB/s 2024-05-17T21:56:30.340 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-17T21:56:30.389 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-17 21:50:17.069809704 +0000 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-17 21:50:16.825811881 +0000 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-17 21:50:16.825811881 +0000 2024-05-17T21:56:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-17T21:56:30.440 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-17T21:56:30.494 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-17T21:56:30.494 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-17T21:56:30.494 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000410332 s, 1.2 MB/s 2024-05-17T21:56:30.495 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-17T21:56:30.545 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-05-17T21:56:30.591 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-17 21:50:17.305807599 +0000 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-17 21:50:17.301807633 +0000 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-17 21:50:17.301807633 +0000 2024-05-17T21:56:30.592 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-17T21:56:30.592 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-17T21:56:30.646 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-17T21:56:30.699 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-17T21:56:30.700 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000384072 s, 1.3 MB/s 2024-05-17T21:56:30.703 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-17T21:56:30.758 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:30.758 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-17T21:56:30.758 INFO:tasks.ceph:Generating config... 2024-05-17T21:56:30.759 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-17T21:56:30.759 INFO:tasks.ceph:[client] debug client = 20 2024-05-17T21:56:30.759 INFO:tasks.ceph:[client] debug ms = 1 2024-05-17T21:56:30.760 INFO:tasks.ceph:[client] pid file = 2024-05-17T21:56:30.760 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-17T21:56:30.760 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-17T21:56:30.760 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-17T21:56:30.761 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-17T21:56:30.761 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-17T21:56:30.762 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-17T21:56:30.763 INFO:tasks.ceph:Setting up mon.a... 2024-05-17T21:56:30.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-17T21:56:30.828 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2024-05-17T21:56:30.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-17T21:56:30.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-17T21:56:30.923 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '172.21.15.155'), ('mon.c', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]')] 2024-05-17T21:56:30.924 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-17T21:56:30.924 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:30.925 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-17T21:56:30.970 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --add b 172.21.15.155 --addv c '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:31.082 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:31.082 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T21:56:31.082 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-05-17T21:56:31.082 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2024-05-17T21:56:31.096189+0000 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-05-17T21:56:31.096189+0000 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 17 (quincy) 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.c 2024-05-17T21:56:31.083 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-17T21:56:31.087 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-17T21:56:31.092 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce... 2024-05-17T21:56:31.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T21:56:31.134 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-05-17T21:56:31.162 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T21:56:31.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.164 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-05-17T21:56:31.165 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-05-17T21:56:31.166 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-05-17T21:56:31.167 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-05-17T21:56:31.168 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T21:56:31.169 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T21:56:31.170 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:56:31.171 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-05-17T21:56:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-05-17T21:56:31.173 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-05-17T21:56:31.174 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T21:56:31.175 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T21:56:31.176 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-05-17T21:56:31.177 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.178 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-05-17T21:56:31.179 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-05-17T21:56:31.180 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-05-17T21:56:31.181 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-05-17T21:56:31.182 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T21:56:31.183 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-05-17T21:56:31.184 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T21:56:31.185 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T21:56:31.186 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-17T21:56:31.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-17T21:56:31.307 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-17T21:56:31.307 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:31.307 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-17T21:56:31.312 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:31.312 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-17T21:56:31.356 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:56:31.357 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:31.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-17T21:56:31.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-17T21:56:31.421 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:56:31.422 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:31.469 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:56:31.469 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:31.470 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-17T21:56:31.470 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-17T21:56:31.492 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:56:31.492 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:31.539 INFO:tasks.ceph:Setting up mon nodes... 2024-05-17T21:56:31.540 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-17T21:56:31.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-17T21:56:31.613 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-17T21:56:31.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-17T21:56:31.694 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-17T21:56:31.707 INFO:tasks.ceph:Setting up mds nodes... 2024-05-17T21:56:31.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-17T21:56:31.779 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-17T21:56:31.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-17T21:56:31.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-17T21:56:31.904 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-17T21:56:31.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-17T21:56:31.923 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-17T21:56:31.998 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-17T21:56:32.004 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-17T21:56:32.015 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-17T21:56:32.120 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-17T21:56:32.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-17T21:56:32.137 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-17T21:56:32.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-17T21:56:32.205 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-17T21:56:32.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-17T21:56:32.279 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-17T21:56:32.292 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-17T21:56:32.356 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-17T21:56:32.370 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-17T21:56:32.426 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-17T21:56:32.439 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-17T21:56:32.440 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-17T21:56:32.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:32.452 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:32.452 INFO:tasks.ceph:role: osd.0 2024-05-17T21:56:32.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:56:32.453 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-17T21:56:32.515 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:32.516 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:32.978 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-05-17T21:56:32.980 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-05-17T21:56:32.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:33.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:33.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:33.084 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:33.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:33.140 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:33.140 INFO:tasks.ceph:role: osd.1 2024-05-17T21:56:33.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:56:33.140 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:33.205 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:33.675 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-05-17T21:56:33.677 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-05-17T21:56:33.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:33.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:33.855 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:33.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:33.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-17T21:56:33.912 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:33.912 INFO:tasks.ceph:role: osd.2 2024-05-17T21:56:33.912 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:56:33.912 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:33.977 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:33.978 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:34.435 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-05-17T21:56:34.437 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-05-17T21:56:34.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-17T21:56:34.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-17T21:56:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:34.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:34.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-17T21:56:34.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:34.656 INFO:tasks.ceph:role: osd.3 2024-05-17T21:56:34.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:56:34.657 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:34.721 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:35.148 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-05-17T21:56:35.149 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-05-17T21:56:35.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-17T21:56:35.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-17T21:56:35.249 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:35.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:35.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:35.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.417+0000 7fcb1db007c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-17T21:56:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.421+0000 7fcb1db007c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-17T21:56:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.421+0000 7fcb1db007c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-17T21:56:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.421+0000 7fcb1db007c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-17T21:56:35.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.421+0000 7fcb1db007c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-17T21:56:35.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:35.425+0000 7fcb1db007c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-17T21:56:38.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-17T21:56:38.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:38.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.533+0000 7f3409f307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-17T21:56:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.533+0000 7f3409f307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-17T21:56:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.533+0000 7f3409f307c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-17T21:56:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.533+0000 7f3409f307c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-17T21:56:38.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.533+0000 7f3409f307c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-17T21:56:38.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:38.537+0000 7f3409f307c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-17T21:56:41.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-17T21:56:41.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-17T21:56:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-17T21:56:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-17T21:56:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-17T21:56:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-17T21:56:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:41.661+0000 7f093e0447c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-17T21:56:44.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-17T21:56:44.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:44.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-17T21:56:44.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-17T21:56:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-17T21:56:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-17T21:56:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-17T21:56:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:56:44.773+0000 7f7b353fa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-17T21:56:47.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-17T21:56:47.765 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-17T21:56:47.780 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:47.780 INFO:tasks.ceph:role: osd.4 2024-05-17T21:56:47.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:56:47.780 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-17T21:56:47.845 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:47.845 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:47.845 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:47.846 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:48.333 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-05-17T21:56:48.336 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-05-17T21:56:48.336 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-17T21:56:48.475 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-17T21:56:48.485 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:48.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:48.486 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-17T21:56:48.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:48.542 INFO:tasks.ceph:role: osd.5 2024-05-17T21:56:48.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:56:48.542 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:48.603 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:49.052 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-05-17T21:56:49.054 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-05-17T21:56:49.054 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-17T21:56:49.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-17T21:56:49.156 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:49.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:49.157 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-17T21:56:49.209 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:49.209 INFO:tasks.ceph:role: osd.6 2024-05-17T21:56:49.209 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:56:49.209 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:49.271 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:49.272 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:49.272 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:49.272 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:49.272 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:49.729 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-05-17T21:56:49.731 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-05-17T21:56:49.732 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-17T21:56:49.823 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-17T21:56:49.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:49.833 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:49.833 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-17T21:56:49.887 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-17T21:56:49.887 INFO:tasks.ceph:role: osd.7 2024-05-17T21:56:49.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:56:49.887 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-17T21:56:49.950 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-17T21:56:50.448 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2024-05-17T21:56:50.450 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2024-05-17T21:56:50.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-17T21:56:50.601 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-17T21:56:50.612 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: /sbin/restorecon: command not found 2024-05-17T21:56:50.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:56:50.613 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:50.757 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.767+0000 7f593b0607c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-17T21:56:50.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.771+0000 7f593b0607c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-17T21:56:50.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.771+0000 7f593b0607c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-17T21:56:50.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.771+0000 7f593b0607c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-17T21:56:50.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.771+0000 7f593b0607c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-17T21:56:50.766 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:50.775+0000 7f593b0607c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-17T21:56:53.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-17T21:56:53.769 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-17T21:56:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-17T21:56:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-17T21:56:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-17T21:56:53.881 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-17T21:56:53.882 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:53.891+0000 7f2fe59bf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-17T21:56:56.872 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-17T21:56:56.885 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:56:56.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-17T21:56:56.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-17T21:56:56.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-17T21:56:56.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-17T21:56:56.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-17T21:56:56.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:56:57.006+0000 7feba09c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-17T21:56:59.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:56:59.365 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:57:00.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-17T21:57:00.020 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:57:00.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.138+0000 7f649a3c27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-17T21:57:00.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.138+0000 7f649a3c27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-17T21:57:00.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.138+0000 7f649a3c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-17T21:57:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.138+0000 7f649a3c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-17T21:57:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.138+0000 7f649a3c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-17T21:57:00.132 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:00.142+0000 7f649a3c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-17T21:57:03.108 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-17T21:57:03.121 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-17T21:57:03.121 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-17T21:57:03.135 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-17T21:57:03.191 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-17T21:57:03.247 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-17T21:57:03.299 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-17T21:57:03.352 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-17T21:57:03.407 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-17T21:57:03.463 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-17T21:57:03.477 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.477 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-17T21:57:03.531 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-17T21:57:03.583 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.583 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-17T21:57:03.636 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.636 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-17T21:57:03.696 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.696 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-17T21:57:03.751 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-17T21:57:03.808 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.808 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-17T21:57:03.814 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:03.814 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-17T21:57:03.860 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.861 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-17T21:57:03.866 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:03.866 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-17T21:57:03.912 INFO:tasks.ceph:Adding keys to all mons... 2024-05-17T21:57:03.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-17T21:57:03.915 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-17T21:57:03.958 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2024-05-17T21:57:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw== 2024-05-17T21:57:03.958 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ== 2024-05-17T21:57:03.959 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T21:57:03.960 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2024-05-17T21:57:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ== 2024-05-17T21:57:03.961 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2024-05-17T21:57:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw== 2024-05-17T21:57:03.961 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2024-05-17T21:57:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw== 2024-05-17T21:57:03.966 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2024-05-17T21:57:03.966 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw== 2024-05-17T21:57:03.966 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ== 2024-05-17T21:57:03.967 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout:[client.2] 2024-05-17T21:57:03.968 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ== 2024-05-17T21:57:03.969 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-05-17T21:57:03.969 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw== 2024-05-17T21:57:03.969 INFO:teuthology.orchestra.run.smithi155.stdout:[client.3] 2024-05-17T21:57:03.969 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw== 2024-05-17T21:57:03.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-17T21:57:04.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-17T21:57:04.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.244 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.387 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.460 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-17T21:57:04.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-17T21:57:04.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.681 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-17T21:57:04.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.824 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.826 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.901 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:04.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-17T21:57:05.037 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.041 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.194 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-17T21:57:05.350 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-17T21:57:05.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-17T21:57:05.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-17T21:57:05.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-17T21:57:05.488 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-17T21:57:05.501 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-17T21:57:05.627 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-17T21:57:05.638 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-17T21:57:05.690 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-17T21:57:05.816 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-17T21:57:05.827 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:57:05.829 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-17T21:57:05.872 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-17T21:57:05.872 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-17T21:57:05.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-17T21:57:05.877 INFO:tasks.ceph.mon.a:Started 2024-05-17T21:57:05.877 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-17T21:57:05.878 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-17T21:57:05.917 INFO:tasks.ceph.mon.b:Started 2024-05-17T21:57:05.917 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-17T21:57:05.917 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-17T21:57:05.919 INFO:tasks.ceph.mon.c:Started 2024-05-17T21:57:05.919 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-17T21:57:05.919 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-17T21:57:05.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-17T21:57:05.922 INFO:tasks.ceph.mgr.y:Started 2024-05-17T21:57:05.922 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-17T21:57:05.922 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-17T21:57:05.925 INFO:tasks.ceph.mgr.x:Started 2024-05-17T21:57:05.925 INFO:tasks.ceph:Setting crush tunables to default 2024-05-17T21:57:05.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2024-05-17T21:57:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.076+0000 7f2dc432c640 1 Processor -- start 2024-05-17T21:57:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.076+0000 7f2dc432c640 1 -- start start 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc432c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc432c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc432c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dbc0b65a0 con 0x7f2dbc160fa0 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc432c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc0b6110 con 0x7f2dbc160650 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc432c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2dbc0b5500 con 0x7f2dbc15dcb0 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58894/0 (socket says 172.21.15.83:58894) 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3734324177 learned_addr learned my addr 172.21.15.83:0/3734324177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 msgr2=0x7f2dbc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 msgr2=0x7f2dbc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 msgr2=0x7f2dbc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 msgr2=0x7f2dbc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T21:57:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.080+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 msgr2=0x7f2dbc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 msgr2=0x7f2dbc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T21:57:06.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:57:06.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 msgr2=0x7f2dbc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-17T21:57:06.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 msgr2=0x7f2dbc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-17T21:57:06.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-17T21:57:06.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.284+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-17T21:57:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658866426 0 0) 0x7f2dbc0b65a0 con 0x7f2dbc160fa0 2024-05-17T21:57:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2da4003530 con 0x7f2dbc160fa0 2024-05-17T21:57:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1528694272 0 0) 0x7f2da4003530 con 0x7f2dbc160fa0 2024-05-17T21:57:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 msgr2=0x7f2dbc0b4910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T21:57:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.155:3300/0 conn(0x7f2dbc15dcb0 0x7f2dbc0b4910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 msgr2=0x7f2dbc160a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 --2- 172.21.15.83:0/3734324177 >> v2:172.21.15.83:3300/0 conn(0x7f2dbc160650 0x7f2dbc160a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc0b57e0 con 0x7f2dbc160fa0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2dac001a80 con 0x7f2dbc160fa0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 >> v1:172.21.15.155:6789/0 conn(0x7f2dbc160fa0 legacy=0x7f2dbc15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 0x7f2da4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da4004810 0x7f2da4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da40052d0 0x7f2da400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b6110 con 0x7f2da4003ab0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b5500 con 0x7f2da4004810 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2da400c2c0 con 0x7f2da40052d0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 0x7f2da4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc18c7640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da4004810 0x7f2da4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.348+0000 7f2dc28c9640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da40052d0 0x7f2da400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da40052d0 msgr2=0x7f2da400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da40052d0 0x7f2da400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da4004810 msgr2=0x7f2da4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da4004810 0x7f2da4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3734324177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4003530 con 0x7f2da4003ab0 2024-05-17T21:57:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.352+0000 7f2dc28c9640 1 -- 172.21.15.83:0/3734324177 reap_dead start 2024-05-17T21:57:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 0x7f2da4003f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0a07c0 tx=0x7f2db401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8247bcac5e721b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40ae020 con 0x7f2da4003ab0 2024-05-17T21:57:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db40a6620 con 0x7f2da4003ab0 2024-05-17T21:57:06.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc10c6640 1 -- 172.21.15.83:0/3734324177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db400d660 con 0x7f2da4003ab0 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc432c640 1 -- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 msgr2=0x7f2da4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 0x7f2da4003f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0a07c0 tx=0x7f2db401c920 comp rx=0 tx=0).stop 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc432c640 1 -- 172.21.15.83:0/3734324177 shutdown_connections 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3734324177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da4003ab0 0x7f2da4003f40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.360+0000 7f2dc432c640 1 -- 172.21.15.83:0/3734324177 >> 172.21.15.83:0/3734324177 conn(0x7f2dbc09b8b0 msgr2=0x7f2dbc0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.364+0000 7f2dc432c640 1 -- 172.21.15.83:0/3734324177 shutdown_connections 2024-05-17T21:57:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.364+0000 7f2dc432c640 1 -- 172.21.15.83:0/3734324177 wait complete. 2024-05-17T21:57:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.364+0000 7f2dc432c640 1 Processor -- start 2024-05-17T21:57:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.364+0000 7f2dc432c640 1 -- start start 2024-05-17T21:57:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 0x7f2dbc155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 0x7f2dbc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1fca10 0x7f2dbc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b4910 con 0x7f2dbc1fc600 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1613b0 con 0x7f2dbc1fca10 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2dbc0b4a90 con 0x7f2da4003ab0 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc18c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 0x7f2dbc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 0x7f2dbc155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc18c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 0x7f2dbc0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58930/0 (socket says 172.21.15.83:58930) 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc18c7640 1 -- 172.21.15.83:0/3550971674 learned_addr learned my addr 172.21.15.83:0/3550971674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc28c9640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1fca10 0x7f2dbc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1fca10 msgr2=0x7f2dbc0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1fca10 0x7f2dbc0b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3550971674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 msgr2=0x7f2dbc0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 0x7f2dbc0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 -- 172.21.15.83:0/3550971674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc15c0a0 con 0x7f2da4003ab0 2024-05-17T21:57:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc20c8640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 0x7f2dbc155cf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2db40ac000 tx=0x7f2da4005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40ab040 con 0x7f2da4003ab0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db40b9070 con 0x7f2da4003ab0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc15b000 con 0x7f2da4003ab0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db400d660 con 0x7f2da4003ab0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc15b5c0 con 0x7f2da4003ab0 2024-05-17T21:57:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.368+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dbc0b4e20 con 0x7f2da4003ab0 2024-05-17T21:57:06.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.376+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2db40b1020 con 0x7f2da4003ab0 2024-05-17T21:57:06.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.376+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2db40be6b0 con 0x7f2da4003ab0 2024-05-17T21:57:06.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.376+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2db4016670 con 0x7f2da4003ab0 2024-05-17T21:57:06.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.460+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2dbc070690 con 0x7f2da4003ab0 2024-05-17T21:57:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dbaffd640 1 -- 172.21.15.83:0/3550971674 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f2db40059a0 con 0x7f2da4003ab0 2024-05-17T21:57:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 msgr2=0x7f2dbc155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 0x7f2dbc155cf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2db40ac000 tx=0x7f2da4005840 comp rx=0 tx=0).stop 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 shutdown_connections 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2dbc1fca10 0x7f2dbc0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dbc1fc600 0x7f2dbc0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 --2- 172.21.15.83:0/3550971674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da4003ab0 0x7f2dbc155cf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 >> 172.21.15.83:0/3550971674 conn(0x7f2dbc09b8b0 msgr2=0x7f2dbc070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 shutdown_connections 2024-05-17T21:57:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:06.468+0000 7f2dc432c640 1 -- 172.21.15.83:0/3550971674 wait complete. 2024-05-17T21:57:06.477 INFO:tasks.ceph:check_enable_crimson: False 2024-05-17T21:57:06.477 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-17T21:57:06.477 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:06.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-17T21:57:06.488 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:06.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-17T21:57:06.538 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:06.545+0000 7f9740ca0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T21:57:06.547 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:06.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-17T21:57:06.560 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:06.572+0000 7f532101f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-17T21:57:06.571 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:57:06.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-17T21:57:06.627 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:06.627 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-17T21:57:06.641 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:06.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-17T21:57:06.697 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:06.698 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-17T21:57:06.752 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:57:06.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-17T21:57:06.809 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 72d56ff5-e600-4fb8-b3ec-af99c7b1feed 0 2024-05-17T21:57:07.003 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.009+0000 7f49dcdda640 1 Processor -- start 2024-05-17T21:57:07.004 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.009+0000 7f49dcdda640 1 -- start start 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49dcdda640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 0x7f49d80b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49dcdda640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f49d80b0410 0x7f49d80b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49dcdda640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d80b6fd0 con 0x7f49d806d8a0 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49dcdda640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f49d80b4cf0 con 0x7f49d80b0410 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49dcdda640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49d815dd90 con 0x7f49d80b6680 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 0x7f49d80b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d659c640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f49d806d8a0 0x7f49d815d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43514/0 (socket says 172.21.15.155:43514) 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d659c640 1 -- 172.21.15.155:0/565907613 learned_addr learned my addr 172.21.15.155:0/565907613 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 -- 172.21.15.155:0/565907613 >> v2:172.21.15.83:3300/0 conn(0x7f49d80b0410 msgr2=0x7f49d80b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.006 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 --2- 172.21.15.155:0/565907613 >> v2:172.21.15.83:3300/0 conn(0x7f49d80b0410 0x7f49d80b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 -- 172.21.15.155:0/565907613 >> v1:172.21.15.155:6789/0 conn(0x7f49d806d8a0 legacy=0x7f49d815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 -- 172.21.15.155:0/565907613 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80b5d00 con 0x7f49d80b6680 2024-05-17T21:57:07.007 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d559a640 1 -- 172.21.15.155:0/565907613 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2126810290 0 0) 0x7f49d00014a0 con 0x7f49d806d8a0 2024-05-17T21:57:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d5d9b640 1 --2- 172.21.15.155:0/565907613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 0x7f49d80b6a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097c0 tx=0x7f49c801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4717930b1d1071b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d559a640 1 -- 172.21.15.155:0/565907613 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c80b0070 con 0x7f49d80b6680 2024-05-17T21:57:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d559a640 1 -- 172.21.15.155:0/565907613 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c80ab600 con 0x7f49d80b6680 2024-05-17T21:57:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.013+0000 7f49d559a640 1 -- 172.21.15.155:0/565907613 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c80ab9d0 con 0x7f49d80b6680 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- 172.21.15.155:0/565907613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 msgr2=0x7f49d80b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 --2- 172.21.15.155:0/565907613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 0x7f49d80b6a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097c0 tx=0x7f49c801c410 comp rx=0 tx=0).stop 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- 172.21.15.155:0/565907613 shutdown_connections 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 --2- 172.21.15.155:0/565907613 >> v2:172.21.15.83:3300/0 conn(0x7f49d80b0410 0x7f49d80b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 --2- 172.21.15.155:0/565907613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d80b6680 0x7f49d80b6a90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- 172.21.15.155:0/565907613 >> 172.21.15.155:0/565907613 conn(0x7f49d80d1ed0 msgr2=0x7f49d80d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.009 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- 172.21.15.155:0/565907613 shutdown_connections 2024-05-17T21:57:07.010 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- 172.21.15.155:0/565907613 wait complete. 2024-05-17T21:57:07.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 Processor -- start 2024-05-17T21:57:07.011 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.017+0000 7f49dcdda640 1 -- start start 2024-05-17T21:57:07.013 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80b0410 0x7f49d8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d80b6680 0x7f49d8155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49d806de30 con 0x7f49d80b6680 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49d806dcb0 con 0x7f49d80b0410 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f49d8167560 con 0x7f49d806d8a0 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.014 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55636/0 (socket says 172.21.15.155:55636) 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 -- 172.21.15.155:0/3741510364 learned_addr learned my addr 172.21.15.155:0/3741510364 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d5d9b640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80b0410 0x7f49d8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d6d9d640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d80b6680 0x7f49d8155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 -- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80b0410 msgr2=0x7f49d8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80b0410 0x7f49d8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 -- 172.21.15.155:0/3741510364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d80b6680 msgr2=0x7f49d8155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d80b6680 0x7f49d8155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 -- 172.21.15.155:0/3741510364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81fa570 con 0x7f49d806d8a0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49d659c640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49d000cb40 tx=0x7f49d000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0017070 con 0x7f49d806d8a0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d000aca0 con 0x7f49d806d8a0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81fa850 con 0x7f49d806d8a0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0012620 con 0x7f49d806d8a0 2024-05-17T21:57:07.015 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81fae10 con 0x7f49d806d8a0 2024-05-17T21:57:07.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f49d001a070 con 0x7f49d806d8a0 2024-05-17T21:57:07.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f49d001cbc0 con 0x7f49d806d8a0 2024-05-17T21:57:07.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.021+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80b0820 con 0x7f49d806d8a0 2024-05-17T21:57:07.016 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.025+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f49d0021020 con 0x7f49d806d8a0 2024-05-17T21:57:07.021 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:07.029+0000 7f9740ca0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T21:57:07.077 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:07.088+0000 7f532101f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-17T21:57:07.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.093+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "72d56ff5-e600-4fb8-b3ec-af99c7b1feed", "id": 0} v 0) -- 0x7f49d8160ae0 con 0x7f49d806d8a0 2024-05-17T21:57:07.095 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49c6ffd640 1 -- 172.21.15.155:0/3741510364 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72d56ff5-e600-4fb8-b3ec-af99c7b1feed", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f49d80b6fd0 con 0x7f49d806d8a0 2024-05-17T21:57:07.095 INFO:teuthology.orchestra.run.smithi155.stdout:0 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 msgr2=0x7f49d8153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49d000cb40 tx=0x7f49d000a060 comp rx=0 tx=0).stop 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 shutdown_connections 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d80b6680 0x7f49d8155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80b0410 0x7f49d8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.096 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 --2- 172.21.15.155:0/3741510364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49d806d8a0 0x7f49d8153110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 >> 172.21.15.155:0/3741510364 conn(0x7f49d80d1ed0 msgr2=0x7f49d8167930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 shutdown_connections 2024-05-17T21:57:07.097 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.101+0000 7f49dcdda640 1 -- 172.21.15.155:0/3741510364 wait complete. 2024-05-17T21:57:07.115 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 57dc8e47-0c7f-4b36-a07a-b0f33948ed67 1 2024-05-17T21:57:07.233 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:07.241+0000 7f9740ca0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T21:57:07.259 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.265+0000 7fec671a3640 1 Processor -- start 2024-05-17T21:57:07.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.265+0000 7fec671a3640 1 -- start start 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec671a3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec671a3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec60068d70 con 0x7fec6006f480 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec671a3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec60070f00 con 0x7fec600d0250 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec671a3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fec6006f890 con 0x7fec600cf460 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55646/0 (socket says 172.21.15.155:55646) 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec65740640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fec600d0250 0x7fec6006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43530/0 (socket says 172.21.15.155:43530) 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 -- 172.21.15.155:0/1307926033 learned_addr learned my addr 172.21.15.155:0/1307926033 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.262 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec5f7fe640 1 -- 172.21.15.155:0/1307926033 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921589988 0 0) 0x7fec60070f00 con 0x7fec600d0250 2024-05-17T21:57:07.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec5f7fe640 1 -- 172.21.15.155:0/1307926033 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec50003530 con 0x7fec600d0250 2024-05-17T21:57:07.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 -- 172.21.15.155:0/1307926033 >> v1:172.21.15.155:6789/0 conn(0x7fec600d0250 legacy=0x7fec6006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 -- 172.21.15.155:0/1307926033 >> v1:172.21.15.83:6789/0 conn(0x7fec6006f480 legacy=0x7fec600cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.269+0000 7fec64f3f640 1 -- 172.21.15.155:0/1307926033 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec600711c0 con 0x7fec600cf460 2024-05-17T21:57:07.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec64f3f640 1 --2- 172.21.15.155:0/1307926033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098a0 tx=0x7fec4c01c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e0ee612f0238b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec5f7fe640 1 -- 172.21.15.155:0/1307926033 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b1070 con 0x7fec600cf460 2024-05-17T21:57:07.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec5f7fe640 1 -- 172.21.15.155:0/1307926033 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c0ac600 con 0x7fec600cf460 2024-05-17T21:57:07.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec5f7fe640 1 -- 172.21.15.155:0/1307926033 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0ac9d0 con 0x7fec600cf460 2024-05-17T21:57:07.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 -- 172.21.15.155:0/1307926033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 msgr2=0x7fec6015fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 --2- 172.21.15.155:0/1307926033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098a0 tx=0x7fec4c01c5a0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 -- 172.21.15.155:0/1307926033 shutdown_connections 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 --2- 172.21.15.155:0/1307926033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec600cf460 0x7fec6015fe00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 -- 172.21.15.155:0/1307926033 >> 172.21.15.155:0/1307926033 conn(0x7fec600ce6e0 msgr2=0x7fec600ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 -- 172.21.15.155:0/1307926033 shutdown_connections 2024-05-17T21:57:07.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.273+0000 7fec671a3640 1 -- 172.21.15.155:0/1307926033 wait complete. 2024-05-17T21:57:07.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 Processor -- start 2024-05-17T21:57:07.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- start start 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec6006f480 0x7fec601628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec600cf460 0x7fec6016a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec60070ca0 con 0x7fec600cf460 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fec600d15c0 con 0x7fec6006f480 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fec6016dbd0 con 0x7fec6014e230 2024-05-17T21:57:07.271 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec64f3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec6006f480 0x7fec601628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55656/0 (socket says 172.21.15.155:55656) 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 -- 172.21.15.155:0/131052382 learned_addr learned my addr 172.21.15.155:0/131052382 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5ffff640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec600cf460 0x7fec6016a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 -- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec6006f480 msgr2=0x7fec601628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec6006f480 0x7fec601628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 -- 172.21.15.155:0/131052382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec600cf460 msgr2=0x7fec6016a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec600cf460 0x7fec6016a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 -- 172.21.15.155:0/131052382 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec601605b0 con 0x7fec6014e230 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec65740640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c6f0 tx=0x7fec5800cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58017070 con 0x7fec6014e230 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58005db0 con 0x7fec6014e230 2024-05-17T21:57:07.272 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec60160890 con 0x7fec6014e230 2024-05-17T21:57:07.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800a880 con 0x7fec6014e230 2024-05-17T21:57:07.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec60160e50 con 0x7fec6014e230 2024-05-17T21:57:07.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec5801a070 con 0x7fec6014e230 2024-05-17T21:57:07.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fec5801c440 con 0x7fec6014e230 2024-05-17T21:57:07.273 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.277+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec600cdeb0 con 0x7fec6014e230 2024-05-17T21:57:07.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.281+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fec58021070 con 0x7fec6014e230 2024-05-17T21:57:07.279 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:07.288+0000 7f532101f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-17T21:57:07.341 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.349+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "57dc8e47-0c7f-4b36-a07a-b0f33948ed67", "id": 1} v 0) -- 0x7fec600d0860 con 0x7fec6014e230 2024-05-17T21:57:07.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec5d7fa640 1 -- 172.21.15.155:0/131052382 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57dc8e47-0c7f-4b36-a07a-b0f33948ed67", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fec580071a0 con 0x7fec6014e230 2024-05-17T21:57:07.349 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-05-17T21:57:07.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 msgr2=0x7fec6016a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c6f0 tx=0x7fec5800cbb0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 shutdown_connections 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec6014e230 0x7fec6016a5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec600cf460 0x7fec6016a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 --2- 172.21.15.155:0/131052382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec6006f480 0x7fec601628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 >> 172.21.15.155:0/131052382 conn(0x7fec600ce6e0 msgr2=0x7fec600b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 shutdown_connections 2024-05-17T21:57:07.352 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.357+0000 7fec671a3640 1 -- 172.21.15.155:0/131052382 wait complete. 2024-05-17T21:57:07.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8 2 2024-05-17T21:57:07.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.505+0000 7f49701ae640 1 Processor -- start 2024-05-17T21:57:07.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.505+0000 7f49701ae640 1 -- start start 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f49701ae640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f496815ea50 0x7f49680b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f49701ae640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49680af470 con 0x7f496806cea0 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f49701ae640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f496815e290 con 0x7f49680b4e50 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f49701ae640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f496815bae0 con 0x7f496815ea50 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496d749640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f49680b4e50 0x7f496806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43542/0 (socket says 172.21.15.155:43542) 2024-05-17T21:57:07.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496d749640 1 -- 172.21.15.155:0/2199942942 learned_addr learned my addr 172.21.15.155:0/2199942942 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496df4a640 1 --2- 172.21.15.155:0/2199942942 >> v2:172.21.15.83:3300/0 conn(0x7f496815ea50 0x7f49680b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1700167862 0 0) 0x7f496815e290 con 0x7f49680b4e50 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954003530 con 0x7f49680b4e50 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1983793423 0 0) 0x7f4954003530 con 0x7f49680b4e50 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 >> v2:172.21.15.83:3300/0 conn(0x7f496815ea50 msgr2=0x7f49680b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 --2- 172.21.15.155:0/2199942942 >> v2:172.21.15.83:3300/0 conn(0x7f496815ea50 0x7f49680b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 >> v1:172.21.15.83:6789/0 conn(0x7f496806cea0 legacy=0x7f496815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49680afcd0 con 0x7f49680b4e50 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f4958001700 con 0x7f49680b4e50 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 >> v1:172.21.15.155:6789/0 conn(0x7f49680b4e50 legacy=0x7f496806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954003a60 0x7f4954003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4954004810 0x7f4954004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 0x7f495400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f496815bae0 con 0x7f4954004810 2024-05-17T21:57:07.504 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f495400c290 con 0x7f4954005360 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f495400c490 con 0x7f4954003a60 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 0x7f495400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496df4a640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954003a60 0x7f4954003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496d749640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4954004810 0x7f4954004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 -- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954003a60 msgr2=0x7f4954003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954003a60 0x7f4954003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 -- 172.21.15.155:0/2199942942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4954004810 msgr2=0x7f4954004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4954004810 0x7f4954004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.505 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.509+0000 7f496e74b640 1 -- 172.21.15.155:0/2199942942 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4954003530 con 0x7f4954005360 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f496e74b640 1 -- 172.21.15.155:0/2199942942 reap_dead start 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f496e74b640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 0x7f495400b9d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49680731a0 tx=0x7f496001c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add26d0eeca4f2d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960001ae0 con 0x7f4954005360 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4960003980 con 0x7f4954005360 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f496cf48640 1 -- 172.21.15.155:0/2199942942 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960003d70 con 0x7f4954005360 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 msgr2=0x7f495400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 0x7f495400b9d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49680731a0 tx=0x7f496001c750 comp rx=0 tx=0).stop 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- 172.21.15.155:0/2199942942 shutdown_connections 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 --2- 172.21.15.155:0/2199942942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4954005360 0x7f495400b9d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- 172.21.15.155:0/2199942942 >> 172.21.15.155:0/2199942942 conn(0x7f496809b8b0 msgr2=0x7f49680b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- 172.21.15.155:0/2199942942 shutdown_connections 2024-05-17T21:57:07.507 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- 172.21.15.155:0/2199942942 wait complete. 2024-05-17T21:57:07.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 Processor -- start 2024-05-17T21:57:07.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.513+0000 7f49701ae640 1 -- start start 2024-05-17T21:57:07.510 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4968151e10 0x7f496815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49680cfb50 0x7f49680cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f496815ee60 con 0x7f4968151e10 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49680b3f10 con 0x7f49680cfb50 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f496815efe0 con 0x7f4954005360 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496e74b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49680cfb50 0x7f49680cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55674/0 (socket says 172.21.15.155:55674) 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 -- 172.21.15.155:0/405747183 learned_addr learned my addr 172.21.15.155:0/405747183 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496d749640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4968151e10 0x7f496815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.511 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 -- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49680cfb50 msgr2=0x7f49680cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49680cfb50 0x7f49680cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 -- 172.21.15.155:0/405747183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4968151e10 msgr2=0x7f496815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4968151e10 0x7f496815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 -- 172.21.15.155:0/405747183 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49680d04a0 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f496df4a640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4964011990 tx=0x7f4964011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496400fad0 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496400fc70 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4968152870 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964014440 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4968151610 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4964012040 con 0x7f4954005360 2024-05-17T21:57:07.512 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f496401daf0 con 0x7f4954005360 2024-05-17T21:57:07.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.517+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49680b4910 con 0x7f4954005360 2024-05-17T21:57:07.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.521+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f49680d04a0 con 0x7f4954005360 2024-05-17T21:57:07.586 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.593+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8", "id": 2} v 0) -- 0x7f496815d1f0 con 0x7f4954005360 2024-05-17T21:57:07.593 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f495e7fc640 1 -- 172.21.15.155:0/405747183 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f4964019030 con 0x7f4954005360 2024-05-17T21:57:07.594 INFO:teuthology.orchestra.run.smithi155.stdout:2 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 msgr2=0x7f4968156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4964011990 tx=0x7f4964011e50 comp rx=0 tx=0).stop 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 shutdown_connections 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49680cfb50 0x7f49680cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4968151e10 0x7f496815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 --2- 172.21.15.155:0/405747183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4954005360 0x7f4968156310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 >> 172.21.15.155:0/405747183 conn(0x7f496809b8b0 msgr2=0x7f49681573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 shutdown_connections 2024-05-17T21:57:07.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.601+0000 7f49701ae640 1 -- 172.21.15.155:0/405747183 wait complete. 2024-05-17T21:57:07.612 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 39aa7753-240c-48cb-8eca-ba1b7106cffc 3 2024-05-17T21:57:07.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 Processor -- start 2024-05-17T21:57:07.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 -- start start 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f33b006c790 0x7f33b006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b015f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b006d0e0 con 0x7f33b015ea50 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f33b015bae0 con 0x7f33b006c790 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b81d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00af470 con 0x7f33b015f240 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f33b006c790 0x7f33b006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f33b006c790 0x7f33b006cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39790/0 (socket says 172.21.15.155:39790) 2024-05-17T21:57:07.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 -- 172.21.15.155:0/2276327217 learned_addr learned my addr 172.21.15.155:0/2276327217 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b6772640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5f71640 1 --1- 172.21.15.155:0/2276327217 >> v1:172.21.15.155:6789/0 conn(0x7f33b015ea50 0x7f33b00b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43550/0 (socket says 172.21.15.155:43550) 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 -- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 msgr2=0x7f33b015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b015f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 -- 172.21.15.155:0/2276327217 >> v1:172.21.15.155:6789/0 conn(0x7f33b015ea50 legacy=0x7f33b00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b4f6f640 1 -- 172.21.15.155:0/2276327217 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036271461 0 0) 0x7f33a4001120 con 0x7f33b015ea50 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b5770640 1 -- 172.21.15.155:0/2276327217 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b015e290 con 0x7f33b006c790 2024-05-17T21:57:07.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.757+0000 7f33b6772640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:07.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b5770640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b006cba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009870 tx=0x7f33ac01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed7c7bbd596583a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b4f6f640 1 -- 172.21.15.155:0/2276327217 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0b2070 con 0x7f33b006c790 2024-05-17T21:57:07.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b4f6f640 1 -- 172.21.15.155:0/2276327217 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac0a9070 con 0x7f33b006c790 2024-05-17T21:57:07.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b4f6f640 1 -- 172.21.15.155:0/2276327217 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0ad880 con 0x7f33b006c790 2024-05-17T21:57:07.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 msgr2=0x7f33b006cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b006cba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009870 tx=0x7f33ac01c560 comp rx=0 tx=0).stop 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- 172.21.15.155:0/2276327217 shutdown_connections 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 --2- 172.21.15.155:0/2276327217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b006cba0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- 172.21.15.155:0/2276327217 >> 172.21.15.155:0/2276327217 conn(0x7f33b009b8b0 msgr2=0x7f33b00b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- 172.21.15.155:0/2276327217 shutdown_connections 2024-05-17T21:57:07.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- 172.21.15.155:0/2276327217 wait complete. 2024-05-17T21:57:07.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 Processor -- start 2024-05-17T21:57:07.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.761+0000 7f33b81d5640 1 -- start start 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 0x7f33b013ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b0156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00b3f10 con 0x7f33b015ea50 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00b5020 con 0x7f33b006c790 2024-05-17T21:57:07.759 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f33b015ee60 con 0x7f33b015f240 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b6772640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b0156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39796/0 (socket says 172.21.15.155:39796) 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 -- 172.21.15.155:0/3985113085 learned_addr learned my addr 172.21.15.155:0/3985113085 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5770640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 0x7f33b013ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 -- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 msgr2=0x7f33b0156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b0156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 -- 172.21.15.155:0/3985113085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 msgr2=0x7f33b013ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 0x7f33b013ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 -- 172.21.15.155:0/3985113085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b013f480 con 0x7f33b006c790 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5770640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 0x7f33b013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b5f71640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c430 tx=0x7f33a400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a40074e0 con 0x7f33b006c790 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a40047f0 con 0x7f33b006c790 2024-05-17T21:57:07.760 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4004be0 con 0x7f33b006c790 2024-05-17T21:57:07.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b014d830 con 0x7f33b006c790 2024-05-17T21:57:07.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b014ddf0 con 0x7f33b006c790 2024-05-17T21:57:07.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33a4011070 con 0x7f33b006c790 2024-05-17T21:57:07.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f33a4004d80 con 0x7f33b006c790 2024-05-17T21:57:07.761 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.765+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b006cba0 con 0x7f33b006c790 2024-05-17T21:57:07.763 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.769+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f33b006d0e0 con 0x7f33b006c790 2024-05-17T21:57:07.817 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:07.825+0000 7f9740ca0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T21:57:07.827 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.833+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "39aa7753-240c-48cb-8eca-ba1b7106cffc", "id": 3} v 0) -- 0x7f33b0168500 con 0x7f33b006c790 2024-05-17T21:57:07.833 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33a27fc640 1 -- 172.21.15.155:0/3985113085 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "39aa7753-240c-48cb-8eca-ba1b7106cffc", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f33a401d3f0 con 0x7f33b006c790 2024-05-17T21:57:07.833 INFO:teuthology.orchestra.run.smithi155.stdout:3 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 msgr2=0x7f33b014cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c430 tx=0x7f33a400c830 comp rx=0 tx=0).stop 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 shutdown_connections 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b015f240 0x7f33b0156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33b015ea50 0x7f33b013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 --2- 172.21.15.155:0/3985113085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b006c790 0x7f33b014cf30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 >> 172.21.15.155:0/3985113085 conn(0x7f33b009b8b0 msgr2=0x7f33b0167db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 shutdown_connections 2024-05-17T21:57:07.835 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.841+0000 7f33b81d5640 1 -- 172.21.15.155:0/3985113085 wait complete. 2024-05-17T21:57:07.845 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:07.856+0000 7f532101f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-17T21:57:07.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new a24fee4b-1bc7-4d13-92f9-db435b71cd60 4 2024-05-17T21:57:07.875 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:07.881+0000 7f9740ca0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T21:57:07.898 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:07.908+0000 7f532101f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-17T21:57:07.930 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:07.937+0000 7f9740ca0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T21:57:07.951 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:07.960+0000 7f532101f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-17T21:57:07.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 Processor -- start 2024-05-17T21:57:07.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 -- start start 2024-05-17T21:57:07.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f345009c9a0 0x7f34500711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f345009cdb0 0x7f3450071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500d1630 con 0x7f3450071c40 2024-05-17T21:57:07.987 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3450072050 con 0x7f345009c9a0 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f345736b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f34500b2ee0 con 0x7f345009cdb0 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3454906640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f345009cdb0 0x7f3450071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3455908640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3450071c40 0x7f34500ce360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43564/0 (socket says 172.21.15.155:43564) 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3455908640 1 -- 172.21.15.155:0/470681657 learned_addr learned my addr 172.21.15.155:0/470681657 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3455107640 1 --2- 172.21.15.155:0/470681657 >> v2:172.21.15.83:3300/0 conn(0x7f345009c9a0 0x7f34500711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3454906640 1 -- 172.21.15.155:0/470681657 >> v2:172.21.15.83:3300/0 conn(0x7f345009c9a0 msgr2=0x7f34500711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3454906640 1 --2- 172.21.15.155:0/470681657 >> v2:172.21.15.83:3300/0 conn(0x7f345009c9a0 0x7f34500711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3454906640 1 -- 172.21.15.155:0/470681657 >> v1:172.21.15.155:6789/0 conn(0x7f3450071c40 legacy=0x7f34500ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3454906640 1 -- 172.21.15.155:0/470681657 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500cfd20 con 0x7f345009cdb0 2024-05-17T21:57:07.988 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.993+0000 7f3447fff640 1 -- 172.21.15.155:0/470681657 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 259106858 0 0) 0x7f34500d1630 con 0x7f3450071c40 2024-05-17T21:57:07.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f3454906640 1 --2- 172.21.15.155:0/470681657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009cdb0 0x7f3450071700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34500cdc00 tx=0x7f344001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54feec8804bb5518 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f3447fff640 1 -- 172.21.15.155:0/470681657 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400b0070 con 0x7f345009cdb0 2024-05-17T21:57:07.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f3447fff640 1 -- 172.21.15.155:0/470681657 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34400ab600 con 0x7f345009cdb0 2024-05-17T21:57:07.990 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f3447fff640 1 -- 172.21.15.155:0/470681657 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400ab9d0 con 0x7f345009cdb0 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 -- 172.21.15.155:0/470681657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009cdb0 msgr2=0x7f3450071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 --2- 172.21.15.155:0/470681657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009cdb0 0x7f3450071700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34500cdc00 tx=0x7f344001c3f0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 -- 172.21.15.155:0/470681657 shutdown_connections 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 --2- 172.21.15.155:0/470681657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009cdb0 0x7f3450071700 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 --2- 172.21.15.155:0/470681657 >> v2:172.21.15.83:3300/0 conn(0x7f345009c9a0 0x7f34500711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 -- 172.21.15.155:0/470681657 >> 172.21.15.155:0/470681657 conn(0x7f34500d00b0 msgr2=0x7f34500d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 -- 172.21.15.155:0/470681657 shutdown_connections 2024-05-17T21:57:07.991 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:07.997+0000 7f345736b640 1 -- 172.21.15.155:0/470681657 wait complete. 2024-05-17T21:57:07.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 Processor -- start 2024-05-17T21:57:07.992 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- start start 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3450071c40 0x7f34500b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f345009cdb0 0x7f345009fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34500d1410 con 0x7f3450071c40 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f34500b8840 con 0x7f345009c9a0 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f34500cfad0 con 0x7f345009cdb0 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3455908640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f345009cdb0 0x7f345009fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39816/0 (socket says 172.21.15.155:39816) 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 -- 172.21.15.155:0/374731860 learned_addr learned my addr 172.21.15.155:0/374731860 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3455107640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3450071c40 0x7f34500b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 -- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f345009cdb0 msgr2=0x7f345009fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f345009cdb0 0x7f345009fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 -- 172.21.15.155:0/374731860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3450071c40 msgr2=0x7f34500b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3450071c40 0x7f34500b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 -- 172.21.15.155:0/374731860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500a1190 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3454906640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34400ae040 tx=0x7f3440019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400b8020 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34400b4630 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345009f2d0 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400b0040 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345009f890 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34400bb020 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f34400aa8b0 con 0x7f345009c9a0 2024-05-17T21:57:07.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.001+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34500d19e0 con 0x7f345009c9a0 2024-05-17T21:57:07.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.005+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f34400c2070 con 0x7f345009c9a0 2024-05-17T21:57:08.010 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.017+0000 7f9740ca0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T21:57:08.027 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.036+0000 7f532101f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-17T21:57:08.062 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.069+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a24fee4b-1bc7-4d13-92f9-db435b71cd60", "id": 4} v 0) -- 0x7f34500b0b40 con 0x7f345009c9a0 2024-05-17T21:57:08.063 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.069+0000 7f9740ca0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T21:57:08.071 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f3445ffb640 1 -- 172.21.15.155:0/374731860 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a24fee4b-1bc7-4d13-92f9-db435b71cd60", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3440016070 con 0x7f345009c9a0 2024-05-17T21:57:08.071 INFO:teuthology.orchestra.run.smithi155.stdout:4 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 msgr2=0x7f34501508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34400ae040 tx=0x7f3440019190 comp rx=0 tx=0).stop 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 shutdown_connections 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f345009cdb0 0x7f345009fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f345009c9a0 0x7f34501508d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 --2- 172.21.15.155:0/374731860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3450071c40 0x7f34500b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 >> 172.21.15.155:0/374731860 conn(0x7f34500d00b0 msgr2=0x7f34500d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 shutdown_connections 2024-05-17T21:57:08.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.077+0000 7f345736b640 1 -- 172.21.15.155:0/374731860 wait complete. 2024-05-17T21:57:08.080 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.088+0000 7f532101f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-17T21:57:08.089 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new cbbb1c7a-e4a3-4569-9900-ec42752e7266 5 2024-05-17T21:57:08.143 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.149+0000 7f9740ca0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T21:57:08.156 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.164+0000 7f532101f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-17T21:57:08.197 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.205+0000 7f9740ca0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T21:57:08.208 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.220+0000 7f532101f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-17T21:57:08.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 Processor -- start 2024-05-17T21:57:08.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 -- start start 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae406d0e0 con 0x7f0ae415ea50 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae415bae0 con 0x7f0ae415f240 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0aea6ca640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40af470 con 0x7f0ae406c790 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae8c67640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55716/0 (socket says 172.21.15.155:55716) 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae8c67640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39832/0 (socket says 172.21.15.155:39832) 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 -- 172.21.15.155:0/1646270117 learned_addr learned my addr 172.21.15.155:0/1646270117 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 -- 172.21.15.155:0/1646270117 >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 msgr2=0x7f0ae415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 --2- 172.21.15.155:0/1646270117 >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 -- 172.21.15.155:0/1646270117 >> v1:172.21.15.83:6789/0 conn(0x7f0ae415ea50 legacy=0x7f0ae40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:08.280 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae37fe640 1 -- 172.21.15.155:0/1646270117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae415e290 con 0x7f0ae406c790 2024-05-17T21:57:08.281 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.285+0000 7f0ae8c67640 1 --2- 172.21.15.155:0/1646270117 >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:08.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0ae37fe640 1 --2- 172.21.15.155:0/1646270117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009870 tx=0x7f0ad801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96c1684a8c800f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0ae2ffd640 1 -- 172.21.15.155:0/1646270117 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80b0070 con 0x7f0ae406c790 2024-05-17T21:57:08.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0ae2ffd640 1 -- 172.21.15.155:0/1646270117 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad80ab400 con 0x7f0ae406c790 2024-05-17T21:57:08.282 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0ae2ffd640 1 -- 172.21.15.155:0/1646270117 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80ab7d0 con 0x7f0ae406c790 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 -- 172.21.15.155:0/1646270117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 msgr2=0x7f0ae406cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/1646270117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009870 tx=0x7f0ad801c380 comp rx=0 tx=0).stop 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 -- 172.21.15.155:0/1646270117 shutdown_connections 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/1646270117 >> v2:172.21.15.155:3300/0 conn(0x7f0ae415f240 0x7f0ae415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/1646270117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae406c790 0x7f0ae406cba0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 -- 172.21.15.155:0/1646270117 >> 172.21.15.155:0/1646270117 conn(0x7f0ae409b8b0 msgr2=0x7f0ae40b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 -- 172.21.15.155:0/1646270117 shutdown_connections 2024-05-17T21:57:08.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.289+0000 7f0aea6ca640 1 -- 172.21.15.155:0/1646270117 wait complete. 2024-05-17T21:57:08.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 Processor -- start 2024-05-17T21:57:08.285 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- start start 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 0x7f0ae41680b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ae415ea50 0x7f0ae4161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40b3f10 con 0x7f0ae415ea50 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40b5020 con 0x7f0ae406c790 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae415ee60 con 0x7f0ae415f240 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae8c67640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 0x7f0ae41680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae8c67640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55720/0 (socket says 172.21.15.155:55720) 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae8c67640 1 -- 172.21.15.155:0/875870522 learned_addr learned my addr 172.21.15.155:0/875870522 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae37fe640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ae415ea50 0x7f0ae4161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 -- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 msgr2=0x7f0ae4159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 -- 172.21.15.155:0/875870522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ae415ea50 msgr2=0x7f0ae4161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ae415ea50 0x7f0ae4161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 -- 172.21.15.155:0/875870522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae415f760 con 0x7f0ae406c790 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae8c67640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae3fff640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 0x7f0ae41680b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b410 tx=0x7f0ad400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4015030 con 0x7f0ae406c790 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad4004020 con 0x7f0ae406c790 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae415fa40 con 0x7f0ae406c790 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4004450 con 0x7f0ae406c790 2024-05-17T21:57:08.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae4160000 con 0x7f0ae406c790 2024-05-17T21:57:08.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ad4018070 con 0x7f0ae406c790 2024-05-17T21:57:08.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0ad4003b00 con 0x7f0ae406c790 2024-05-17T21:57:08.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.293+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae406cba0 con 0x7f0ae406c790 2024-05-17T21:57:08.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.297+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0ad4003e20 con 0x7f0ae406c790 2024-05-17T21:57:08.353 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.361+0000 7f9740ca0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T21:57:08.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.365+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cbbb1c7a-e4a3-4569-9900-ec42752e7266", "id": 5} v 0) -- 0x7f0ae40d3360 con 0x7f0ae406c790 2024-05-17T21:57:08.358 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.368+0000 7f532101f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-17T21:57:08.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.369+0000 7f0ae0ff9640 1 -- 172.21.15.155:0/875870522 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbbb1c7a-e4a3-4569-9900-ec42752e7266", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0ad4003660 con 0x7f0ae406c790 2024-05-17T21:57:08.363 INFO:teuthology.orchestra.run.smithi155.stdout:5 2024-05-17T21:57:08.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.369+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 msgr2=0x7f0ae41680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 0x7f0ae41680b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b410 tx=0x7f0ad400b8d0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 shutdown_connections 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae415f240 0x7f0ae4159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ae415ea50 0x7f0ae4161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 --2- 172.21.15.155:0/875870522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae406c790 0x7f0ae41680b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 >> 172.21.15.155:0/875870522 conn(0x7f0ae409b8b0 msgr2=0x7f0ae4154730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 shutdown_connections 2024-05-17T21:57:08.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.373+0000 7f0aea6ca640 1 -- 172.21.15.155:0/875870522 wait complete. 2024-05-17T21:57:08.381 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 10f68bf4-5160-4d6f-a486-d651cc4e79bc 6 2024-05-17T21:57:08.417 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:08.425+0000 7f9740ca0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T21:57:08.420 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:08.428+0000 7f532101f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-17T21:57:08.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.565+0000 7f20e0f7a640 1 Processor -- start 2024-05-17T21:57:08.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.565+0000 7f20e0f7a640 1 -- start start 2024-05-17T21:57:08.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20e0f7a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20e0f7a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0b60e0 con 0x7f20dc06d8c0 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20e0f7a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0b63c0 con 0x7f20dc0b6f30 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20e0f7a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc0d1060 con 0x7f20dc071e30 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55734/0 (socket says 172.21.15.155:55734) 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 -- 172.21.15.155:0/2654531593 learned_addr learned my addr 172.21.15.155:0/2654531593 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d959a640 1 -- 172.21.15.155:0/2654531593 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893030672 0 0) 0x7f20dc0b63c0 con 0x7f20dc0b6f30 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 -- 172.21.15.155:0/2654531593 >> v1:172.21.15.155:6789/0 conn(0x7f20dc0b6f30 legacy=0x7f20dc071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 -- 172.21.15.155:0/2654531593 >> v1:172.21.15.83:6789/0 conn(0x7f20dc06d8c0 legacy=0x7f20dc0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:08.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.569+0000 7f20d9d9b640 1 -- 172.21.15.155:0/2654531593 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0d0cf0 con 0x7f20dc071e30 2024-05-17T21:57:08.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20d9d9b640 1 --2- 172.21.15.155:0/2654531593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20d00098a0 tx=0x7f20d001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf543b4008b5f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20d959a640 1 -- 172.21.15.155:0/2654531593 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00aa430 con 0x7f20dc071e30 2024-05-17T21:57:08.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20d959a640 1 -- 172.21.15.155:0/2654531593 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d00af070 con 0x7f20dc071e30 2024-05-17T21:57:08.565 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20d959a640 1 -- 172.21.15.155:0/2654531593 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d00aa7f0 con 0x7f20dc071e30 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- 172.21.15.155:0/2654531593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 msgr2=0x7f20dc15cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/2654531593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20d00098a0 tx=0x7f20d001c4f0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- 172.21.15.155:0/2654531593 shutdown_connections 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/2654531593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc071e30 0x7f20dc15cc40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- 172.21.15.155:0/2654531593 >> 172.21.15.155:0/2654531593 conn(0x7f20dc0b4330 msgr2=0x7f20dc0b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- 172.21.15.155:0/2654531593 shutdown_connections 2024-05-17T21:57:08.566 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- 172.21.15.155:0/2654531593 wait complete. 2024-05-17T21:57:08.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 Processor -- start 2024-05-17T21:57:08.567 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.573+0000 7f20e0f7a640 1 -- start start 2024-05-17T21:57:08.570 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc06d8c0 0x7f20dc143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 0x7f20dc148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0688b0 con 0x7f20dc06d8c0 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc068730 con 0x7f20dc14a610 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc068a30 con 0x7f20dc0b6f30 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 0x7f20dc148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20d9d9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:55740/0 (socket says 172.21.15.155:55740) 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20d9d9b640 1 -- 172.21.15.155:0/3351829489 learned_addr learned my addr 172.21.15.155:0/3351829489 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20da59c640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc06d8c0 0x7f20dc143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 -- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 msgr2=0x7f20dc147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.571 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 -- 172.21.15.155:0/3351829489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc06d8c0 msgr2=0x7f20dc143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc06d8c0 0x7f20dc143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 -- 172.21.15.155:0/3351829489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc144030 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20d9d9b640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20dad9d640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 0x7f20dc148380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20d400be00 tx=0x7f20d400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003980 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d4003b20 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc14adf0 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4004020 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc14b3b0 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20d4006d10 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f20d4014df0 con 0x7f20dc14a610 2024-05-17T21:57:08.572 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.577+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc15cc40 con 0x7f20dc14a610 2024-05-17T21:57:08.573 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.581+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f20d4014800 con 0x7f20dc14a610 2024-05-17T21:57:08.640 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.645+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "10f68bf4-5160-4d6f-a486-d651cc4e79bc", "id": 6} v 0) -- 0x7f20dc0b7480 con 0x7f20dc14a610 2024-05-17T21:57:08.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.653+0000 7f20caffd640 1 -- 172.21.15.155:0/3351829489 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10f68bf4-5160-4d6f-a486-d651cc4e79bc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f20d40149e0 con 0x7f20dc14a610 2024-05-17T21:57:08.648 INFO:teuthology.orchestra.run.smithi155.stdout:6 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 msgr2=0x7f20dc148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 0x7f20dc148380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f20d400be00 tx=0x7f20d400bef0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 shutdown_connections 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20dc14a610 0x7f20dc148380 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20dc0b6f30 0x7f20dc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 --2- 172.21.15.155:0/3351829489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc06d8c0 0x7f20dc143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 >> 172.21.15.155:0/3351829489 conn(0x7f20dc0b4330 msgr2=0x7f20dc15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.650 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 shutdown_connections 2024-05-17T21:57:08.651 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.657+0000 7f20e0f7a640 1 -- 172.21.15.155:0/3351829489 wait complete. 2024-05-17T21:57:08.667 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 3d54ba58-d840-4d28-87b4-0a564e3c58e6 7 2024-05-17T21:57:08.845 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 Processor -- start 2024-05-17T21:57:08.846 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 -- start start 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1f740cde70 0x7f1f7415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f740d20a0 con 0x7f1f740d2e60 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f740d2380 con 0x7f1f740d3580 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f79b29640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1f740d2660 con 0x7f1f740cde70 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f737fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1f740cde70 0x7f1f7415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.847 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f73fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1f740d3580 0x7f1f740d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43574/0 (socket says 172.21.15.155:43574) 2024-05-17T21:57:08.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f73fff640 1 -- 172.21.15.155:0/3638967969 learned_addr learned my addr 172.21.15.155:0/3638967969 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-05-17T21:57:08.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f737fe640 1 -- 172.21.15.155:0/3638967969 >> v1:172.21.15.155:6789/0 conn(0x7f1f740d3580 legacy=0x7f1f740d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f737fe640 1 -- 172.21.15.155:0/3638967969 >> v1:172.21.15.83:6789/0 conn(0x7f1f740d2e60 legacy=0x7f1f740ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:08.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f727fc640 1 -- 172.21.15.155:0/3638967969 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914216833 0 0) 0x7f1f740d2380 con 0x7f1f740d3580 2024-05-17T21:57:08.848 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.853+0000 7f1f737fe640 1 -- 172.21.15.155:0/3638967969 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f740d2940 con 0x7f1f740cde70 2024-05-17T21:57:08.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f737fe640 1 --2- 172.21.15.155:0/3638967969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f7415dbb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f680097c0 tx=0x7f1f6801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b3f3e923d67105 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f727fc640 1 -- 172.21.15.155:0/3638967969 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b1070 con 0x7f1f740cde70 2024-05-17T21:57:08.849 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f727fc640 1 -- 172.21.15.155:0/3638967969 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680ac410 con 0x7f1f740cde70 2024-05-17T21:57:08.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f727fc640 1 -- 172.21.15.155:0/3638967969 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680ac7e0 con 0x7f1f740cde70 2024-05-17T21:57:08.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- 172.21.15.155:0/3638967969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 msgr2=0x7f1f7415dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.850 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 --2- 172.21.15.155:0/3638967969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f7415dbb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f680097c0 tx=0x7f1f6801c4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- 172.21.15.155:0/3638967969 shutdown_connections 2024-05-17T21:57:08.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 --2- 172.21.15.155:0/3638967969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f7415dbb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- 172.21.15.155:0/3638967969 >> 172.21.15.155:0/3638967969 conn(0x7f1f74071180 msgr2=0x7f1f74071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- 172.21.15.155:0/3638967969 shutdown_connections 2024-05-17T21:57:08.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- 172.21.15.155:0/3638967969 wait complete. 2024-05-17T21:57:08.852 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 Processor -- start 2024-05-17T21:57:08.852 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.857+0000 7f1f79b29640 1 -- start start 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f740d2e60 0x7f1f74157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 0x7f1f740cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f740b7440 con 0x7f1f740d3580 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f740b72c0 con 0x7f1f740cde70 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f740b75c0 con 0x7f1f740d2e60 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:39878/0 (socket says 172.21.15.155:39878) 2024-05-17T21:57:08.855 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 -- 172.21.15.155:0/2973005207 learned_addr learned my addr 172.21.15.155:0/2973005207 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f72ffd640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f740d2e60 0x7f1f74157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f73fff640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 0x7f1f740cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 -- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f740d2e60 msgr2=0x7f1f74157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f740d2e60 0x7f1f74157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 -- 172.21.15.155:0/2973005207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 msgr2=0x7f1f740cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 0x7f1f740cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 -- 172.21.15.155:0/2973005207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f740caca0 con 0x7f1f740cde70 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f73fff640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 0x7f1f740cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f737fe640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f6801a0d0 tx=0x7f1f68019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b1040 con 0x7f1f740cde70 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680bb070 con 0x7f1f740cde70 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f740caf80 con 0x7f1f740cde70 2024-05-17T21:57:08.856 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b5650 con 0x7f1f740cde70 2024-05-17T21:57:08.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f740cb540 con 0x7f1f740cde70 2024-05-17T21:57:08.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f680b57f0 con 0x7f1f740cde70 2024-05-17T21:57:08.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f1f680bf3c0 con 0x7f1f740cde70 2024-05-17T21:57:08.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.861+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7415dbb0 con 0x7f1f740cde70 2024-05-17T21:57:08.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.865+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1f68016090 con 0x7f1f740cde70 2024-05-17T21:57:08.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.933+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3d54ba58-d840-4d28-87b4-0a564e3c58e6", "id": 7} v 0) -- 0x7f1f74071730 con 0x7f1f740cde70 2024-05-17T21:57:08.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.941+0000 7f1f5bfff640 1 -- 172.21.15.155:0/2973005207 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3d54ba58-d840-4d28-87b4-0a564e3c58e6", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f1f680c0460 con 0x7f1f740cde70 2024-05-17T21:57:08.937 INFO:teuthology.orchestra.run.smithi155.stdout:7 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 msgr2=0x7f1f74157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f6801a0d0 tx=0x7f1f68019240 comp rx=0 tx=0).stop 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 shutdown_connections 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f740d3580 0x7f1f740cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f740d2e60 0x7f1f74157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 --2- 172.21.15.155:0/2973005207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f740cde70 0x7f1f74157840 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 >> 172.21.15.155:0/2973005207 conn(0x7f1f74071180 msgr2=0x7f1f740d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 shutdown_connections 2024-05-17T21:57:08.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:08.945+0000 7f1f79b29640 1 -- 172.21.15.155:0/2973005207 wait complete. 2024-05-17T21:57:08.955 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-17T21:57:08.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-17T21:57:08.958 INFO:tasks.ceph.osd.0:Started 2024-05-17T21:57:08.958 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-17T21:57:08.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-17T21:57:08.961 INFO:tasks.ceph.osd.1:Started 2024-05-17T21:57:08.961 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-17T21:57:08.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-17T21:57:08.963 INFO:tasks.ceph.osd.2:Started 2024-05-17T21:57:08.963 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-17T21:57:08.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-17T21:57:08.965 INFO:tasks.ceph.osd.3:Started 2024-05-17T21:57:08.966 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-17T21:57:08.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-17T21:57:08.968 INFO:tasks.ceph.osd.4:Started 2024-05-17T21:57:08.968 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-17T21:57:08.968 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-17T21:57:08.972 INFO:tasks.ceph.osd.5:Started 2024-05-17T21:57:08.972 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-17T21:57:08.972 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-17T21:57:08.974 INFO:tasks.ceph.osd.6:Started 2024-05-17T21:57:08.974 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-17T21:57:08.974 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-17T21:57:08.977 INFO:tasks.ceph.osd.7:Started 2024-05-17T21:57:08.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:09.065 INFO:tasks.ceph.mgr.y.smithi083.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-17T21:57:09.065 INFO:tasks.ceph.mgr.y.smithi083.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-17T21:57:09.066 INFO:tasks.ceph.mgr.y.smithi083.stderr: from numpy import show_config as show_numpy_config 2024-05-17T21:57:09.076 INFO:tasks.ceph.mgr.x.smithi155.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-17T21:57:09.076 INFO:tasks.ceph.mgr.x.smithi155.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-17T21:57:09.076 INFO:tasks.ceph.mgr.x.smithi155.stderr: from numpy import show_config as show_numpy_config 2024-05-17T21:57:09.106 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:09.116+0000 7f532101f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T21:57:09.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.140+0000 7fd0a6c1f640 1 Processor -- start 2024-05-17T21:57:09.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.140+0000 7fd0a6c1f640 1 -- start start 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a00d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00d2280 con 0x7fd0a01580f0 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a0158500 con 0x7fd0a00b3120 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a015efc0 con 0x7fd0a00d1930 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a49bb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd0a01580f0 0x7fd0a00af370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41600/0 (socket says 172.21.15.83:41600) 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a49bb640 1 -- 172.21.15.83:0/3917377772 learned_addr learned my addr 172.21.15.83:0/3917377772 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09ffff640 1 --2- 172.21.15.83:0/3917377772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a00d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3341391272 0 0) 0x7fd08c001120 con 0x7fd0a01580f0 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd090003530 con 0x7fd0a01580f0 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477999110 0 0) 0x7fd0a0158500 con 0x7fd0a00b3120 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09ffff640 1 -- 172.21.15.83:0/3917377772 >> v1:172.21.15.155:6789/0 conn(0x7fd0a00b3120 legacy=0x7fd0a00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09ffff640 1 -- 172.21.15.83:0/3917377772 >> v1:172.21.15.83:6789/0 conn(0x7fd0a01580f0 legacy=0x7fd0a00af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09ffff640 1 -- 172.21.15.83:0/3917377772 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00b4920 con 0x7fd0a00d1930 2024-05-17T21:57:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09ffff640 1 --2- 172.21.15.83:0/3917377772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a00d1d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0940097f0 tx=0x7fd09401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac63c93ee4f04f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940aa610 con 0x7fd0a00d1930 2024-05-17T21:57:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0940af070 con 0x7fd0a00d1930 2024-05-17T21:57:09.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd09f7fe640 1 -- 172.21.15.83:0/3917377772 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940aa9d0 con 0x7fd0a00d1930 2024-05-17T21:57:09.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/3917377772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 msgr2=0x7fd0a00d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/3917377772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a00d1d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0940097f0 tx=0x7fd09401c4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/3917377772 shutdown_connections 2024-05-17T21:57:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/3917377772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a00d1d40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/3917377772 >> 172.21.15.83:0/3917377772 conn(0x7fd0a009b8b0 msgr2=0x7fd0a00b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/3917377772 shutdown_connections 2024-05-17T21:57:09.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.144+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/3917377772 wait complete. 2024-05-17T21:57:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 Processor -- start 2024-05-17T21:57:09.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 -- start start 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a00b3120 0x7fd0a0152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a01f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00afa80 con 0x7fd0a01580f0 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00b4ee0 con 0x7fd0a00b3120 2024-05-17T21:57:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00afc00 con 0x7fd0a00d1930 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 -- 172.21.15.83:0/210850423 learned_addr learned my addr 172.21.15.83:0/210850423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd09ffff640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a01f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a49bb640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a00b3120 0x7fd0a0152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 -- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 msgr2=0x7fd0a01f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a01f9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 -- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a00b3120 msgr2=0x7fd0a0152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a00b3120 0x7fd0a0152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 -- 172.21.15.83:0/210850423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a009ca00 con 0x7fd0a01580f0 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a51bc640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c730 tx=0x7fd09800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098003b80 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098003d40 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a009cce0 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.148+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980052b0 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.152+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a009d2a0 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.152+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd098005660 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.152+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd09800ad10 con 0x7fd0a01580f0 2024-05-17T21:57:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.152+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a00d1d40 con 0x7fd0a01580f0 2024-05-17T21:57:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.152+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd09800a640 con 0x7fd0a01580f0 2024-05-17T21:57:09.144 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:09.149+0000 7f9740ca0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-17T21:57:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0a00b2ec0 con 0x7fd0a01580f0 2024-05-17T21:57:09.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd09d7fa640 1 -- 172.21.15.83:0/210850423 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd09800a820 con 0x7fd0a01580f0 2024-05-17T21:57:09.201 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:09.202 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:08.940892+0000","last_up_change":"0.000000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 msgr2=0x7fd0a009daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c730 tx=0x7fd09800cbf0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 shutdown_connections 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a01580f0 0x7fd0a009daa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a00d1930 0x7fd0a01f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 --2- 172.21.15.83:0/210850423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a00b3120 0x7fd0a0152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 >> 172.21.15.83:0/210850423 conn(0x7fd0a009b8b0 msgr2=0x7fd0a00d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 shutdown_connections 2024-05-17T21:57:09.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:09.212+0000 7fd0a6c1f640 1 -- 172.21.15.83:0/210850423 wait complete. 2024-05-17T21:57:09.220 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-17T21:57:09.220 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-17T21:57:09.243 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:09.252+0000 7f532101f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T21:57:09.274 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:09.281+0000 7f9740ca0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-17T21:57:09.361 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-05-17T21:57:09.369+0000 7f462b86c7c0 -1 Falling back to public interface 2024-05-17T21:57:09.396 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:09.404+0000 7f532101f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T21:57:09.430 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:09.437+0000 7f9740ca0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-17T21:57:09.459 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:09.468+0000 7f532101f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T21:57:09.502 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:09.509+0000 7f9740ca0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-17T21:57:09.592 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:09.600+0000 7f532101f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T21:57:09.597 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-05-17T21:57:09.608+0000 7f5646d4d7c0 -1 Falling back to public interface 2024-05-17T21:57:09.614 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-05-17T21:57:09.624+0000 7f90214107c0 -1 Falling back to public interface 2024-05-17T21:57:09.622 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-05-17T21:57:09.629+0000 7fb0f2c5e7c0 -1 Falling back to public interface 2024-05-17T21:57:09.625 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-05-17T21:57:09.636+0000 7fa3b996d7c0 -1 Falling back to public interface 2024-05-17T21:57:09.626 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-05-17T21:57:09.633+0000 7f4f925db7c0 -1 Falling back to public interface 2024-05-17T21:57:09.636 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:09.641+0000 7f9740ca0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-17T21:57:09.638 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-05-17T21:57:09.645+0000 7f384a44d7c0 -1 Falling back to public interface 2024-05-17T21:57:09.645 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-05-17T21:57:09.656+0000 7f6b8009a7c0 -1 Falling back to public interface 2024-05-17T21:57:10.025 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:10.165 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 Processor -- start 2024-05-17T21:57:10.165 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 -- start start 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee40161780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee400cca10 con 0x7fee4015ddd0 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee40068dd0 con 0x7fee400ce1f0 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee47b99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee40068b80 con 0x7fee400d0e90 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee45134640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fee4015ddd0 0x7fee400cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41606/0 (socket says 172.21.15.83:41606) 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee45134640 1 -- 172.21.15.83:0/1456476430 learned_addr learned my addr 172.21.15.83:0/1456476430 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee40161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.168 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491459875 0 0) 0x7fee400cca10 con 0x7fee4015ddd0 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee2c003530 con 0x7fee4015ddd0 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3515278758 0 0) 0x7fee2c003530 con 0x7fee4015ddd0 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 msgr2=0x7fee40161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee40161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 >> v1:172.21.15.155:6789/0 conn(0x7fee400ce1f0 legacy=0x7fee4006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee400d0b70 con 0x7fee4015ddd0 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fee30001720 con 0x7fee4015ddd0 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 >> v1:172.21.15.83:6789/0 conn(0x7fee4015ddd0 legacy=0x7fee400cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee2c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee2c004030 0x7fee2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee2c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee40068b80 con 0x7fee2c004d50 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c00bc10 con 0x7fee2c004030 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c00be10 con 0x7fee400d0e90 2024-05-17T21:57:10.169 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee2c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee45935640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee2c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 -- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 msgr2=0x7fee2c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400d0e90 0x7fee2c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 -- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee2c004030 msgr2=0x7fee2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee2c004030 0x7fee2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.170 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.176+0000 7fee46136640 1 -- 172.21.15.83:0/1456476430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c003530 con 0x7fee2c004d50 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee46136640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee2c005220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee40073570 tx=0x7fee3801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfacc6537186bd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee380b2070 con 0x7fee2c004d50 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee380a5c50 con 0x7fee2c004d50 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee44933640 1 -- 172.21.15.83:0/1456476430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38009c10 con 0x7fee2c004d50 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 -- 172.21.15.83:0/1456476430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 msgr2=0x7fee2c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 --2- 172.21.15.83:0/1456476430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee2c005220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee40073570 tx=0x7fee3801cc40 comp rx=0 tx=0).stop 2024-05-17T21:57:10.171 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee46136640 1 -- 172.21.15.83:0/1456476430 reap_dead start 2024-05-17T21:57:10.172 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 -- 172.21.15.83:0/1456476430 shutdown_connections 2024-05-17T21:57:10.172 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 -- 172.21.15.83:0/1456476430 >> 172.21.15.83:0/1456476430 conn(0x7fee400ce730 msgr2=0x7fee400ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:10.172 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 -- 172.21.15.83:0/1456476430 shutdown_connections 2024-05-17T21:57:10.172 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.180+0000 7fee47b99640 1 -- 172.21.15.83:0/1456476430 wait complete. 2024-05-17T21:57:10.174 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 Processor -- start 2024-05-17T21:57:10.174 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 -- start start 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee4006f340 0x7fee400a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400b0f80 0x7fee400a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee400c5620 con 0x7fee2c004d50 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee40162600 con 0x7fee4006f340 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.184+0000 7fee47b99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee400d13e0 con 0x7fee400b0f80 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51400/0 (socket says 172.21.15.83:51400) 2024-05-17T21:57:10.178 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 -- 172.21.15.83:0/3395487866 learned_addr learned my addr 172.21.15.83:0/3395487866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45134640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee4006f340 0x7fee400a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee46136640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400b0f80 0x7fee400a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 -- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400b0f80 msgr2=0x7fee400a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400b0f80 0x7fee400a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 -- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee4006f340 msgr2=0x7fee400a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee4006f340 0x7fee400a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 -- 172.21.15.83:0/3395487866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee400cb370 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee45935640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e4d0 tx=0x7fee3c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0052e0 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3c005480 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c007250 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee400a0220 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee400a07e0 con 0x7fee2c004d50 2024-05-17T21:57:10.179 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee3c0073f0 con 0x7fee2c004d50 2024-05-17T21:57:10.180 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fee3c013af0 con 0x7fee2c004d50 2024-05-17T21:57:10.180 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee400ce1f0 con 0x7fee2c004d50 2024-05-17T21:57:10.180 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.188+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fee3c013e10 con 0x7fee2c004d50 2024-05-17T21:57:10.229 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.240+0000 7f532101f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T21:57:10.238 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee4009fed0 con 0x7fee2c004d50 2024-05-17T21:57:10.238 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee367fc640 1 -- 172.21.15.83:0/3395487866 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fee3c003a10 con 0x7fee2c004d50 2024-05-17T21:57:10.239 INFO:teuthology.misc.health.smithi083.stdout: 2024-05-17T21:57:10.239 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":10,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:08.940892+0000","last_up_change":"0.000000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:10.239 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 msgr2=0x7fee400a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e4d0 tx=0x7fee3c00e990 comp rx=0 tx=0).stop 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 shutdown_connections 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee400b0f80 0x7fee400a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee4006f340 0x7fee400a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 --2- 172.21.15.83:0/3395487866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee2c004d50 0x7fee400a1c70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 >> 172.21.15.83:0/3395487866 conn(0x7fee400ce730 msgr2=0x7fee400b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 shutdown_connections 2024-05-17T21:57:10.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:10.248+0000 7fee47b99640 1 -- 172.21.15.83:0/3395487866 wait complete. 2024-05-17T21:57:10.256 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-17T21:57:10.261 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.269+0000 7f9740ca0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-17T21:57:10.307 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.316+0000 7f532101f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T21:57:10.338 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.345+0000 7f9740ca0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-17T21:57:10.489 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.496+0000 7f532101f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T21:57:10.518 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.525+0000 7f9740ca0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-17T21:57:10.562 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.572+0000 7f532101f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T21:57:10.591 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.597+0000 7f9740ca0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-17T21:57:10.852 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.859+0000 7f532101f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T21:57:10.879 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.885+0000 7f9740ca0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-17T21:57:10.913 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:10.923+0000 7f532101f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T21:57:10.941 INFO:tasks.ceph.mgr.x.smithi155.stderr:2024-05-17T21:57:10.949+0000 7f9740ca0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-17T21:57:11.966 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:11.975+0000 7f52c7ad0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-17T21:57:11.967 INFO:tasks.ceph.mgr.y.smithi083.stderr:2024-05-17T21:57:11.975+0000 7f52c7ad0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-17T21:57:13.210 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-05-17T21:57:13.212+0000 7f462b86c7c0 -1 osd.6 0 log_to_monitors true 2024-05-17T21:57:13.345 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-05-17T21:57:13.355+0000 7f5646d4d7c0 -1 osd.1 0 log_to_monitors true 2024-05-17T21:57:13.471 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-05-17T21:57:13.479+0000 7f90214107c0 -1 osd.2 0 log_to_monitors true 2024-05-17T21:57:13.478 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-05-17T21:57:13.484+0000 7f384a44d7c0 -1 osd.4 0 log_to_monitors true 2024-05-17T21:57:13.489 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-05-17T21:57:13.496+0000 7fb0f2c5e7c0 -1 osd.5 0 log_to_monitors true 2024-05-17T21:57:13.491 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-05-17T21:57:13.499+0000 7f6b8009a7c0 -1 osd.3 0 log_to_monitors true 2024-05-17T21:57:13.542 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-05-17T21:57:13.548+0000 7f4f925db7c0 -1 osd.7 0 log_to_monitors true 2024-05-17T21:57:13.568 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-05-17T21:57:13.575+0000 7fa3b996d7c0 -1 osd.0 0 log_to_monitors true 2024-05-17T21:57:17.061 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:17.127 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-05-17T21:57:17.132+0000 7f4627813640 -1 osd.6 0 waiting for initial osdmap 2024-05-17T21:57:17.150 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-05-17T21:57:17.159+0000 7fa3b5914640 -1 osd.0 0 waiting for initial osdmap 2024-05-17T21:57:17.154 INFO:tasks.ceph.osd.6.smithi155.stderr:2024-05-17T21:57:17.160+0000 7f461f614640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.176 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-05-17T21:57:17.180+0000 7f3846c08640 -1 osd.4 0 waiting for initial osdmap 2024-05-17T21:57:17.180 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-05-17T21:57:17.184+0000 7f4f8e582640 -1 osd.7 0 waiting for initial osdmap 2024-05-17T21:57:17.182 INFO:tasks.ceph.osd.0.smithi083.stderr:2024-05-17T21:57:17.191+0000 7fa3ad715640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.187 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-05-17T21:57:17.195+0000 7f6b7c041640 -1 osd.3 0 waiting for initial osdmap 2024-05-17T21:57:17.188 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-05-17T21:57:17.192+0000 7fb0eec05640 -1 osd.5 0 waiting for initial osdmap 2024-05-17T21:57:17.198 INFO:tasks.ceph.osd.7.smithi155.stderr:2024-05-17T21:57:17.204+0000 7f4f86383640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.198 INFO:tasks.ceph.osd.4.smithi155.stderr:2024-05-17T21:57:17.204+0000 7f383e1f5640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.214 INFO:tasks.ceph.osd.5.smithi155.stderr:2024-05-17T21:57:17.220+0000 7fb0e6a06640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.228 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-05-17T21:57:17.235+0000 7f5642cf4640 -1 osd.1 0 waiting for initial osdmap 2024-05-17T21:57:17.234 INFO:tasks.ceph.osd.3.smithi083.stderr:2024-05-17T21:57:17.243+0000 7f6b73e42640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.238 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 Processor -- start 2024-05-17T21:57:17.238 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 -- start start 2024-05-17T21:57:17.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3e4070fa0 0x7fa3e4160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:17.240 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e40ce0e0 con 0x7fa3e415cb50 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e40ce3c0 con 0x7fa3e41612d0 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e9c59640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e406dc60 con 0x7fa3e4070fa0 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3daffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3e4070fa0 0x7fa3e4160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3e415cb50 0x7fa3e4160680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55664/0 (socket says 172.21.15.83:55664) 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e37fe640 1 -- 172.21.15.83:0/1053624774 learned_addr learned my addr 172.21.15.83:0/1053624774 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e2ffd640 1 -- 172.21.15.83:0/1053624774 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801681403 0 0) 0x7fa3e40ce0e0 con 0x7fa3e415cb50 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3e2ffd640 1 -- 172.21.15.83:0/1053624774 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3d0003530 con 0x7fa3e415cb50 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3daffd640 1 -- 172.21.15.83:0/1053624774 >> v1:172.21.15.155:6789/0 conn(0x7fa3e41612d0 legacy=0x7fa3e40ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3daffd640 1 -- 172.21.15.83:0/1053624774 >> v1:172.21.15.83:6789/0 conn(0x7fa3e415cb50 legacy=0x7fa3e4160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.241 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.247+0000 7fa3daffd640 1 -- 172.21.15.83:0/1053624774 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e4068850 con 0x7fa3e4070fa0 2024-05-17T21:57:17.244 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3daffd640 1 --2- 172.21.15.83:0/1053624774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4160d90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36161aa6f23e419 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:17.244 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e2ffd640 1 -- 172.21.15.83:0/1053624774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b0070 con 0x7fa3e4070fa0 2024-05-17T21:57:17.244 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e2ffd640 1 -- 172.21.15.83:0/1053624774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0ab600 con 0x7fa3e4070fa0 2024-05-17T21:57:17.244 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e2ffd640 1 -- 172.21.15.83:0/1053624774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0ab9d0 con 0x7fa3e4070fa0 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 -- 172.21.15.83:0/1053624774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e4070fa0 msgr2=0x7fa3e4160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/1053624774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4160d90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0097c0 tx=0x7fa3cc01c4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 -- 172.21.15.83:0/1053624774 shutdown_connections 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/1053624774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4160d90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 -- 172.21.15.83:0/1053624774 >> 172.21.15.83:0/1053624774 conn(0x7fa3e40ce730 msgr2=0x7fa3e40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 -- 172.21.15.83:0/1053624774 shutdown_connections 2024-05-17T21:57:17.245 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.251+0000 7fa3e9c59640 1 -- 172.21.15.83:0/1053624774 wait complete. 2024-05-17T21:57:17.246 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 Processor -- start 2024-05-17T21:57:17.246 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- start start 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3e415cb50 0x7fa3e415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e416a980 con 0x7fa3e41612d0 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e416a800 con 0x7fa3e4070fa0 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e416ab00 con 0x7fa3e415cb50 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:17.249 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41854/0 (socket says 172.21.15.83:41854) 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 -- 172.21.15.83:0/17329947 learned_addr learned my addr 172.21.15.83:0/17329947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e37fe640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3daffd640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3e415cb50 0x7fa3e415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 -- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3e415cb50 msgr2=0x7fa3e415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3e415cb50 0x7fa3e415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 -- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e4070fa0 msgr2=0x7fa3e4161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 -- 172.21.15.83:0/17329947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e40d0ce0 con 0x7fa3e41612d0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e3fff640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c770 tx=0x7fa3dc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc003b80 con 0x7fa3e41612d0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3dc003d20 con 0x7fa3e41612d0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e40b7b80 con 0x7fa3e41612d0 2024-05-17T21:57:17.250 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00aa90 con 0x7fa3e41612d0 2024-05-17T21:57:17.251 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.255+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e4161cb0 con 0x7fa3e41612d0 2024-05-17T21:57:17.251 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e40ccd80 con 0x7fa3e41612d0 2024-05-17T21:57:17.251 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 93151+0+0 (secure 0 0 0) 0x7fa3dc00ac30 con 0x7fa3e41612d0 2024-05-17T21:57:17.251 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e0ff9640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 0x7fa3c40718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:17.252 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-05-17T21:57:17.259+0000 7f901d3b7640 -1 osd.2 0 waiting for initial osdmap 2024-05-17T21:57:17.252 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e37fe640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 0x7fa3c40718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:17.252 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fa3dc08e260 con 0x7fa3e41612d0 2024-05-17T21:57:17.252 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.259+0000 7fa3e37fe640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 0x7fa3c40718b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4007a70 tx=0x7fa3d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:17.258 INFO:tasks.ceph.osd.1.smithi083.stderr:2024-05-17T21:57:17.267+0000 7f563aaf5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.273 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.283+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa3dc05c050 con 0x7fa3e41612d0 2024-05-17T21:57:17.290 INFO:tasks.ceph.osd.2.smithi083.stderr:2024-05-17T21:57:17.299+0000 7f90151b8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-17T21:57:17.427 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.435+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3e40c9ab0 con 0x7fa3e41612d0 2024-05-17T21:57:17.427 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.435+0000 7fa3e0ff9640 1 -- 172.21.15.83:0/17329947 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fa3dc05b950 con 0x7fa3e41612d0 2024-05-17T21:57:17.428 INFO:teuthology.misc.health.smithi083.stdout: 2024-05-17T21:57:17.428 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":12,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:14.974974+0000","last_up_change":"0.000000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:17.430 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 msgr2=0x7fa3c40718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.430 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 0x7fa3c40718b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4007a70 tx=0x7fa3d4009040 comp rx=0 tx=0).stop 2024-05-17T21:57:17.430 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 msgr2=0x7fa3e40b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:17.430 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c770 tx=0x7fa3dc00cc30 comp rx=0 tx=0).stop 2024-05-17T21:57:17.431 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 shutdown_connections 2024-05-17T21:57:17.431 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e41612d0 0x7fa3e40b6d70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.431 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3e415cb50 0x7fa3e415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.431 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa3c406f3a0 0x7fa3c40718b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.431 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 --2- 172.21.15.83:0/17329947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3e4070fa0 0x7fa3e4161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:17.432 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 >> 172.21.15.83:0/17329947 conn(0x7fa3e40ce730 msgr2=0x7fa3e40b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:17.432 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 shutdown_connections 2024-05-17T21:57:17.432 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:17.439+0000 7fa3e9c59640 1 -- 172.21.15.83:0/17329947 wait complete. 2024-05-17T21:57:17.446 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-17T21:57:24.253 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:24.407 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 Processor -- start 2024-05-17T21:57:24.407 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 -- start start 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd700161050 0x7fd70015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7000d2280 con 0x7fd700073e40 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7000ce9d0 con 0x7fd7000d1e70 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd704fd6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd7000cecb0 con 0x7fd700161050 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd700073e40 0x7fd700074250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55678/0 (socket says 172.21.15.83:55678) 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fed9d640 1 -- 172.21.15.83:0/3741925500 learned_addr learned my addr 172.21.15.83:0/3741925500 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:24.409 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fdd9b640 1 --2- 172.21.15.83:0/3741925500 >> v2:172.21.15.155:3300/0 conn(0x7fd700161050 0x7fd70015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.411 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458115739 0 0) 0x7fd7000d2280 con 0x7fd700073e40 2024-05-17T21:57:24.411 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6e8003530 con 0x7fd700073e40 2024-05-17T21:57:24.411 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443078846 0 0) 0x7fd6e8003530 con 0x7fd700073e40 2024-05-17T21:57:24.411 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 >> v2:172.21.15.155:3300/0 conn(0x7fd700161050 msgr2=0x7fd70015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 --2- 172.21.15.83:0/3741925500 >> v2:172.21.15.155:3300/0 conn(0x7fd700161050 0x7fd70015ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 >> v1:172.21.15.155:6789/0 conn(0x7fd7000d1e70 legacy=0x7fd7000ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7000d1390 con 0x7fd700073e40 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fd6f8001a80 con 0x7fd700073e40 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 >> v1:172.21.15.83:6789/0 conn(0x7fd700073e40 legacy=0x7fd700074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6e8003a60 0x7fd6e8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.418+0000 7fd6fd59a640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6e8004810 0x7fd6e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd6e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000cecb0 con 0x7fd6e8005260 2024-05-17T21:57:24.412 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e800c1e0 con 0x7fd6e8003a60 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e800c3e0 con 0x7fd6e8004810 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd6e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fe59c640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6e8003a60 0x7fd6e8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fdd9b640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6e8004810 0x7fd6e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 -- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6e8004810 msgr2=0x7fd6e8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6e8004810 0x7fd6e8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 -- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6e8003a60 msgr2=0x7fd6e8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6e8003a60 0x7fd6e8003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.413 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 -- 172.21.15.83:0/3741925500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e8003530 con 0x7fd6e8005260 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 -- 172.21.15.83:0/3741925500 reap_dead start 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fed9d640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd6e800b9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80013e0 tx=0x7fd6f801d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d526dd69808c6820 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800a480 con 0x7fd6e8005260 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f80ac660 con 0x7fd6e8005260 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd6fd59a640 1 -- 172.21.15.83:0/3741925500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f80aca50 con 0x7fd6e8005260 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 -- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 msgr2=0x7fd6e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd6e800b9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80013e0 tx=0x7fd6f801d240 comp rx=0 tx=0).stop 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 -- 172.21.15.83:0/3741925500 shutdown_connections 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3741925500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd6e800b9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 -- 172.21.15.83:0/3741925500 >> 172.21.15.83:0/3741925500 conn(0x7fd7000d2510 msgr2=0x7fd7000d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 -- 172.21.15.83:0/3741925500 shutdown_connections 2024-05-17T21:57:24.415 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.422+0000 7fd704fd6640 1 -- 172.21.15.83:0/3741925500 wait complete. 2024-05-17T21:57:24.416 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.426+0000 7fd704fd6640 1 Processor -- start 2024-05-17T21:57:24.416 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.426+0000 7fd704fd6640 1 -- start start 2024-05-17T21:57:24.419 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.419 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd70015aa10 0x7fd70015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7001e9500 0x7fd7001e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd700161460 con 0x7fd6e8005260 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd7000d1950 con 0x7fd7001e9500 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd70015ca60 con 0x7fd70015aa10 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41902/0 (socket says 172.21.15.83:41902) 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 -- 172.21.15.83:0/3700610346 learned_addr learned my addr 172.21.15.83:0/3700610346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fdd9b640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd70015aa10 0x7fd70015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fed9d640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7001e9500 0x7fd7001e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 -- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd70015aa10 msgr2=0x7fd70015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd70015aa10 0x7fd70015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 -- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7001e9500 msgr2=0x7fd7001e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7001e9500 0x7fd7001e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.420 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 -- 172.21.15.83:0/3700610346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70015b360 con 0x7fd6e8005260 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fe59c640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e4d0 tx=0x7fd6f000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00052e0 con 0x7fd6e8005260 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f0018070 con 0x7fd6e8005260 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7001484a0 con 0x7fd6e8005260 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0003840 con 0x7fd6e8005260 2024-05-17T21:57:24.421 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7001e6c30 con 0x7fd6e8005260 2024-05-17T21:57:24.422 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70015cdf0 con 0x7fd6e8005260 2024-05-17T21:57:24.422 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6f00039e0 con 0x7fd6e8005260 2024-05-17T21:57:24.422 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 0x7fd6e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.422 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd6f008f250 con 0x7fd6e8005260 2024-05-17T21:57:24.423 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.430+0000 7fd6fdd9b640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 0x7fd6e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.423 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.434+0000 7fd6fdd9b640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 0x7fd6e0071940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7001f7330 tx=0x7fd6f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.425 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.434+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd6f005c600 con 0x7fd6e8005260 2024-05-17T21:57:24.589 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.598+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7000b15b0 con 0x7fd6e8005260 2024-05-17T21:57:24.591 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd6eeffd640 1 -- 172.21.15.83:0/3700610346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12860 (secure 0 0 0) 0x7fd6f005bf00 con 0x7fd6e8005260 2024-05-17T21:57:24.591 INFO:teuthology.misc.health.smithi083.stdout: 2024-05-17T21:57:24.591 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":16,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:21.199148+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 msgr2=0x7fd6e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 0x7fd6e0071940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7001f7330 tx=0x7fd6f4008040 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 msgr2=0x7fd700147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000e4d0 tx=0x7fd6f000e990 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 shutdown_connections 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd6e006f430 0x7fd6e0071940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7001e9500 0x7fd7001e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd70015aa10 0x7fd70015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 --2- 172.21.15.83:0/3700610346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6e8005260 0x7fd700147f60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 >> 172.21.15.83:0/3700610346 conn(0x7fd7000d2510 msgr2=0x7fd70015f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 shutdown_connections 2024-05-17T21:57:24.594 INFO:teuthology.misc.health.smithi083.stderr:2024-05-17T21:57:24.602+0000 7fd704fd6640 1 -- 172.21.15.83:0/3700610346 wait complete. 2024-05-17T21:57:24.611 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-17T21:57:24.611 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-17T21:57:24.612 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T21:57:24.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T21:57:24.614 INFO:tasks.ceph.mds.a:Started 2024-05-17T21:57:24.614 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T21:57:24.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T21:57:24.616 INFO:tasks.ceph.mds.c:Started 2024-05-17T21:57:24.617 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T21:57:24.617 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T21:57:24.619 INFO:tasks.ceph.mds.b:Started 2024-05-17T21:57:24.620 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T21:57:24.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T21:57:24.622 INFO:tasks.ceph.mds.d:Started 2024-05-17T21:57:24.622 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-17T21:57:24.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:24.734 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T21:57:24.735 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T21:57:24.736 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:57:24.737 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T21:57:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 Processor -- start 2024-05-17T21:57:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 -- start start 2024-05-17T21:57:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5480d02f0 0x7fe54815fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54806d5a0 con 0x7fe54815c370 2024-05-17T21:57:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe548160b00 con 0x7fe5480d02f0 2024-05-17T21:57:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.786+0000 7fe54e738640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe54806d880 con 0x7fe5481601b0 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe5477fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5480d02f0 0x7fe54815fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5480d02f0 0x7fe54815fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:49916/0 (socket says 172.21.15.83:49916) 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe5477fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:46562/0 (socket says 172.21.15.83:46562) 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 -- 172.21.15.83:0/4211817205 learned_addr learned my addr 172.21.15.83:0/4211817205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54ccd5640 1 --1- 172.21.15.83:0/4211817205 >> v1:172.21.15.155:6789/0 conn(0x7fe54815c370 0x7fe54815c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:50122/0 (socket says 172.21.15.83:50122) 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 -- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 msgr2=0x7fe5481605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 -- 172.21.15.83:0/4211817205 >> v1:172.21.15.155:6789/0 conn(0x7fe54815c370 legacy=0x7fe54815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe546ffd640 1 -- 172.21.15.83:0/4211817205 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485701027 0 0) 0x7fe54806d5a0 con 0x7fe54815c370 2024-05-17T21:57:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 -- 172.21.15.83:0/4211817205 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54806db60 con 0x7fe5480d02f0 2024-05-17T21:57:24.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe5477fe640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe547fff640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5480d02f0 0x7fe54815fc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00bb10 tx=0x7fe53c01eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a0f42d4b7b475e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe546ffd640 1 -- 172.21.15.83:0/4211817205 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0b4070 con 0x7fe5480d02f0 2024-05-17T21:57:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe546ffd640 1 -- 172.21.15.83:0/4211817205 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c0a7b00 con 0x7fe5480d02f0 2024-05-17T21:57:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe546ffd640 1 -- 172.21.15.83:0/4211817205 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0af440 con 0x7fe5480d02f0 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 -- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5480d02f0 msgr2=0x7fe54815fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5480d02f0 0x7fe54815fc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00bb10 tx=0x7fe53c01eba0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 -- 172.21.15.83:0/4211817205 shutdown_connections 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5481601b0 0x7fe5481605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 --2- 172.21.15.83:0/4211817205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5480d02f0 0x7fe54815fc70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 -- 172.21.15.83:0/4211817205 >> 172.21.15.83:0/4211817205 conn(0x7fe5480d1ca0 msgr2=0x7fe5480d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 -- 172.21.15.83:0/4211817205 shutdown_connections 2024-05-17T21:57:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.790+0000 7fe54e738640 1 -- 172.21.15.83:0/4211817205 wait complete. 2024-05-17T21:57:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 Processor -- start 2024-05-17T21:57:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- start start 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5480d02f0 0x7fe54814c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54815c370 0x7fe548161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe54815d220 con 0x7fe5481601b0 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe54815d0a0 con 0x7fe54815c370 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe54815d3a0 con 0x7fe5480d02f0 2024-05-17T21:57:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41938/0 (socket says 172.21.15.83:41938) 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 -- 172.21.15.83:0/259682658 learned_addr learned my addr 172.21.15.83:0/259682658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe5477fe640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54815c370 0x7fe548161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe547fff640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5480d02f0 0x7fe54814c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 -- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5480d02f0 msgr2=0x7fe54814c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5480d02f0 0x7fe54814c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 -- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54815c370 msgr2=0x7fe548161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54815c370 0x7fe548161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 -- 172.21.15.83:0/259682658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe548164a00 con 0x7fe5481601b0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54ccd5640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe54000ca30 tx=0x7fe54000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540017070 con 0x7fe5481601b0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54000abd0 con 0x7fe5481601b0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe548164ce0 con 0x7fe5481601b0 2024-05-17T21:57:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540012400 con 0x7fe5481601b0 2024-05-17T21:57:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.794+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481652a0 con 0x7fe5481601b0 2024-05-17T21:57:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548164770 con 0x7fe5481601b0 2024-05-17T21:57:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe54001a070 con 0x7fe5481601b0 2024-05-17T21:57:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe544ff9640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 0x7fe534071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe547fff640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 0x7fe534071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe54008ec10 con 0x7fe5481601b0 2024-05-17T21:57:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.798+0000 7fe547fff640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 0x7fe534071940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b1000 tx=0x7fe53c0a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.802+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe54005be50 con 0x7fe5481601b0 2024-05-17T21:57:24.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.982+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5481669a0 con 0x7fe5481601b0 2024-05-17T21:57:24.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.982+0000 7fe544ff9640 1 -- 172.21.15.83:0/259682658 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fe54005b750 con 0x7fe5481601b0 2024-05-17T21:57:24.973 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-17T21:57:24.974 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-05-17T21:57:24.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 msgr2=0x7fe534071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 0x7fe534071940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0b1000 tx=0x7fe53c0a9070 comp rx=0 tx=0).stop 2024-05-17T21:57:24.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 msgr2=0x7fe548161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:24.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe54000ca30 tx=0x7fe54000cef0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 shutdown_connections 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5481601b0 0x7fe548161b60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe53406f430 0x7fe534071940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe54815c370 0x7fe548161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 --2- 172.21.15.83:0/259682658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5480d02f0 0x7fe54814c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 >> 172.21.15.83:0/259682658 conn(0x7fe5480d1ca0 msgr2=0x7fe5480b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 shutdown_connections 2024-05-17T21:57:24.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:24.986+0000 7fe54e738640 1 -- 172.21.15.83:0/259682658 wait complete. 2024-05-17T21:57:26.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 Processor -- start 2024-05-17T21:57:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 -- start start 2024-05-17T21:57:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c640b67d0 0x7f8c640aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6406deb0 0x7f8c640b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c640714b0 con 0x7f8c640b5940 2024-05-17T21:57:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8c640b4c10 con 0x7f8c640b67d0 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c6b521640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640b6d20 con 0x7f8c6406deb0 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c692bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c640b67d0 0x7f8c640aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c692bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8c640b67d0 0x7f8c640aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38660/0 (socket says 172.21.15.83:38660) 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c692bd640 1 -- 172.21.15.83:0/221440681 learned_addr learned my addr 172.21.15.83:0/221440681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.162+0000 7f8c69abe640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6406deb0 0x7f8c640b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c692bd640 1 -- 172.21.15.83:0/221440681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6406deb0 msgr2=0x7f8c640b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c692bd640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6406deb0 0x7f8c640b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c692bd640 1 -- 172.21.15.83:0/221440681 >> v1:172.21.15.155:6789/0 conn(0x7f8c640b5940 legacy=0x7f8c6406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c692bd640 1 -- 172.21.15.83:0/221440681 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c640b4e60 con 0x7f8c640b67d0 2024-05-17T21:57:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c692bd640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b67d0 0x7f8c640aebf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009870 tx=0x7f8c5c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85a3e29f7e94706 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c63fff640 1 -- 172.21.15.83:0/221440681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0ab650 con 0x7f8c640b67d0 2024-05-17T21:57:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c63fff640 1 -- 172.21.15.83:0/221440681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0b0070 con 0x7f8c640b67d0 2024-05-17T21:57:27.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c63fff640 1 -- 172.21.15.83:0/221440681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0aba10 con 0x7f8c640b67d0 2024-05-17T21:57:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 -- 172.21.15.83:0/221440681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b67d0 msgr2=0x7f8c640aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b67d0 0x7f8c640aebf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009870 tx=0x7f8c5c01c560 comp rx=0 tx=0).stop 2024-05-17T21:57:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 -- 172.21.15.83:0/221440681 shutdown_connections 2024-05-17T21:57:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c6406deb0 0x7f8c640b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 --2- 172.21.15.83:0/221440681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b67d0 0x7f8c640aebf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.166+0000 7f8c6b521640 1 -- 172.21.15.83:0/221440681 >> 172.21.15.83:0/221440681 conn(0x7f8c64073670 msgr2=0x7f8c64073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:27.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- 172.21.15.83:0/221440681 shutdown_connections 2024-05-17T21:57:27.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- 172.21.15.83:0/221440681 wait complete. 2024-05-17T21:57:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 Processor -- start 2024-05-17T21:57:27.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- start start 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 0x7f8c64158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 0x7f8c64158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641696c0 con 0x7f8c640b5940 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64169540 con 0x7f8c64158b40 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c6b521640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8c64169840 con 0x7f8c640b67d0 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38672/0 (socket says 172.21.15.83:38672) 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 -- 172.21.15.83:0/1294679091 learned_addr learned my addr 172.21.15.83:0/1294679091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c68abc640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 0x7f8c64158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c69abe640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 0x7f8c64158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 -- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 msgr2=0x7f8c64158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 0x7f8c64158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 -- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 msgr2=0x7f8c64158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 0x7f8c64158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c692bd640 1 -- 172.21.15.83:0/1294679091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64157af0 con 0x7f8c640b5940 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c68abc640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 0x7f8c64158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.170+0000 7f8c69abe640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 0x7f8c64158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c692bd640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0ad040 tx=0x7f8c5c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0be5f0 con 0x7f8c640b5940 2024-05-17T21:57:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0be790 con 0x7f8c640b5940 2024-05-17T21:57:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c64159490 con 0x7f8c640b5940 2024-05-17T21:57:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0beb80 con 0x7f8c640b5940 2024-05-17T21:57:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6415ed40 con 0x7f8c640b5940 2024-05-17T21:57:27.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c640b0480 con 0x7f8c640b5940 2024-05-17T21:57:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c5c0b43f0 con 0x7f8c640b5940 2024-05-17T21:57:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 0x7f8c50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8c5c1299c0 con 0x7f8c640b5940 2024-05-17T21:57:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c68abc640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 0x7f8c50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.174+0000 7f8c68abc640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 0x7f8c50071940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c58007fd0 tx=0x7f8c58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.178+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8c5c12be20 con 0x7f8c640b5940 2024-05-17T21:57:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.374+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c640d1400 con 0x7f8c640b5940 2024-05-17T21:57:27.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c61ffb640 1 -- 172.21.15.83:0/1294679091 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f8c5c016150 con 0x7f8c640b5940 2024-05-17T21:57:27.367 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:27.367 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-17T21:57:27.367 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-05-17T21:57:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 msgr2=0x7f8c50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 0x7f8c50071940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c58007fd0 tx=0x7f8c58009040 comp rx=0 tx=0).stop 2024-05-17T21:57:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 msgr2=0x7f8c6415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0ad040 tx=0x7f8c5c01ada0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 shutdown_connections 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8c5006f430 0x7f8c50071940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c64158b40 0x7f8c64158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8c640b67d0 0x7f8c64158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 --2- 172.21.15.83:0/1294679091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c640b5940 0x7f8c6415ab60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 >> 172.21.15.83:0/1294679091 conn(0x7f8c64073670 msgr2=0x7f8c640b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 shutdown_connections 2024-05-17T21:57:27.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.378+0000 7f8c6b521640 1 -- 172.21.15.83:0/1294679091 wait complete. 2024-05-17T21:57:27.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T21:57:27.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 Processor -- start 2024-05-17T21:57:27.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 -- start start 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff94c144080 0x7ff94c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94c073780 con 0x7ff94c0d1040 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94c15e790 con 0x7ff94c073370 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff95233d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c15ea40 con 0x7ff94c144080 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff94c0d1040 0x7ff94c15e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34894/0 (socket says 172.21.15.83:34894) 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94bfff640 1 -- 172.21.15.83:0/1401854714 learned_addr learned my addr 172.21.15.83:0/1401854714 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285008210 0 0) 0x7ff93c001120 con 0x7ff94c0d1040 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff934003530 con 0x7ff94c0d1040 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/1401854714 >> v2:172.21.15.155:3300/0 conn(0x7ff94c144080 0x7ff94c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 53081571 0 0) 0x7ff934003530 con 0x7ff94c0d1040 2024-05-17T21:57:27.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 >> v2:172.21.15.155:3300/0 conn(0x7ff94c144080 msgr2=0x7ff94c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 --2- 172.21.15.83:0/1401854714 >> v2:172.21.15.155:3300/0 conn(0x7ff94c144080 0x7ff94c072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 >> v1:172.21.15.155:6789/0 conn(0x7ff94c073370 legacy=0x7ff94c06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c15ed20 con 0x7ff94c0d1040 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ff93c0018d0 con 0x7ff94c0d1040 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/1401854714 >> v2:172.21.15.155:3300/0 conn(0x7ff94c144080 0x7ff94c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 >> v1:172.21.15.83:6789/0 conn(0x7ff94c0d1040 legacy=0x7ff94c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff934003a60 0x7ff934003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934004810 0x7ff934004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 0x7ff93400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c15ea40 con 0x7ff934005310 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff93400c2b0 con 0x7ff934004810 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.550+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff93400c4b0 con 0x7ff934003a60 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 0x7ff93400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934004810 0x7ff934004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94bfff640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff934003a60 0x7ff934003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 -- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff934003a60 msgr2=0x7ff934003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff934003a60 0x7ff934003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 -- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934004810 msgr2=0x7ff934004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934004810 0x7ff934004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 -- 172.21.15.83:0/1401854714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff934003530 con 0x7ff934005310 2024-05-17T21:57:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934004810 0x7ff934004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 -- 172.21.15.83:0/1401854714 reap_dead start 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff9508da640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 0x7ff93400b9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0ce920 tx=0x7ff94401c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154431b10b2447a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944001ae0 con 0x7ff934005310 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff944003ba0 con 0x7ff934005310 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff94affd640 1 -- 172.21.15.83:0/1401854714 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440050d0 con 0x7ff934005310 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 -- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 msgr2=0x7ff93400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 0x7ff93400b9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0ce920 tx=0x7ff94401c750 comp rx=0 tx=0).stop 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 -- 172.21.15.83:0/1401854714 shutdown_connections 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 --2- 172.21.15.83:0/1401854714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff934005310 0x7ff93400b9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 -- 172.21.15.83:0/1401854714 >> 172.21.15.83:0/1401854714 conn(0x7ff94c06cd90 msgr2=0x7ff94c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 -- 172.21.15.83:0/1401854714 shutdown_connections 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.554+0000 7ff95233d640 1 -- 172.21.15.83:0/1401854714 wait complete. 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 Processor -- start 2024-05-17T21:57:27.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- start start 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934005310 0x7ff94c135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff94c15cd70 0x7ff94c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c06f210 con 0x7ff94c0aecf0 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c06ef30 con 0x7ff934005310 2024-05-17T21:57:27.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c144490 con 0x7ff94c15cd70 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38686/0 (socket says 172.21.15.83:38686) 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 -- 172.21.15.83:0/3287663500 learned_addr learned my addr 172.21.15.83:0/3287663500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94bfff640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934005310 0x7ff94c135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff9508da640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff94c15cd70 0x7ff94c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 -- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff94c15cd70 msgr2=0x7ff94c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff94c15cd70 0x7ff94c15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 -- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934005310 msgr2=0x7ff94c135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934005310 0x7ff94c135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 -- 172.21.15.83:0/3287663500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c15d6c0 con 0x7ff94c0aecf0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff94b7fe640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff940006c10 tx=0x7ff940010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94001a070 con 0x7ff94c0aecf0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff940004220 con 0x7ff94c0aecf0 2024-05-17T21:57:27.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f040 con 0x7ff94c0aecf0 2024-05-17T21:57:27.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c0afa20 con 0x7ff94c0aecf0 2024-05-17T21:57:27.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.558+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c0affe0 con 0x7ff94c0aecf0 2024-05-17T21:57:27.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94c0af770 con 0x7ff94c0aecf0 2024-05-17T21:57:27.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff94000c940 con 0x7ff94c0aecf0 2024-05-17T21:57:27.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff948ff9640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 0x7ff938071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff94008f490 con 0x7ff94c0aecf0 2024-05-17T21:57:27.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff94bfff640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 0x7ff938071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.562+0000 7ff94bfff640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 0x7ff938071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000fc0 tx=0x7ff93c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.566+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff94c15d6c0 con 0x7ff94c0aecf0 2024-05-17T21:57:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff94c072e30 con 0x7ff94c0aecf0 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff948ff9640 1 -- 172.21.15.83:0/3287663500 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7ff94005c760 con 0x7ff94c0aecf0 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T21:57:27.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:57:27.720 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T21:57:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 msgr2=0x7ff938071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 0x7ff938071940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000fc0 tx=0x7ff93c009040 comp rx=0 tx=0).stop 2024-05-17T21:57:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 msgr2=0x7ff94c0af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff940006c10 tx=0x7ff940010cf0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 shutdown_connections 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff94c15cd70 0x7ff94c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94c0aecf0 0x7ff94c0af100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff93806f430 0x7ff938071940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 --2- 172.21.15.83:0/3287663500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff934005310 0x7ff94c135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 >> 172.21.15.83:0/3287663500 conn(0x7ff94c06cd90 msgr2=0x7ff94c15f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 shutdown_connections 2024-05-17T21:57:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.730+0000 7ff95233d640 1 -- 172.21.15.83:0/3287663500 wait complete. 2024-05-17T21:57:27.740 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T21:57:27.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T21:57:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.906+0000 7f14f2521640 1 Processor -- start 2024-05-17T21:57:27.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.906+0000 7f14f2521640 1 -- start start 2024-05-17T21:57:27.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f2521640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14ec1570e0 0x7f14ec0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f2521640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14ec160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f2521640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14ec160d70 con 0x7f14ec15cde0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f2521640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14ec0b4a90 con 0x7f14ec1570e0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f2521640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec0d0820 con 0x7f14ec160420 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f0abe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14ec15cde0 0x7f14ec15d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34898/0 (socket says 172.21.15.83:34898) 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f0abe640 1 -- 172.21.15.83:0/2585830346 learned_addr learned my addr 172.21.15.83:0/2585830346 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14ec160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14ebfff640 1 --2- 172.21.15.83:0/2585830346 >> v2:172.21.15.155:3300/0 conn(0x7f14ec1570e0 0x7f14ec0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412033507 0 0) 0x7f14dc0014a0 con 0x7f14ec15cde0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14d4003530 con 0x7f14ec15cde0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2552119708 0 0) 0x7f14d4003530 con 0x7f14ec15cde0 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 msgr2=0x7f14ec160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14ec160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 >> v2:172.21.15.155:3300/0 conn(0x7f14ec1570e0 msgr2=0x7f14ec0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 --2- 172.21.15.83:0/2585830346 >> v2:172.21.15.155:3300/0 conn(0x7f14ec1570e0 0x7f14ec0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec0d0b00 con 0x7f14ec15cde0 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f14dc001c50 con 0x7f14ec15cde0 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 >> v1:172.21.15.83:6789/0 conn(0x7f14ec15cde0 legacy=0x7f14ec15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14d4003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4003fe0 0x7f14d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec0b4a90 con 0x7f14d4003fe0 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec0d0820 con 0x7f14d4004d50 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14d400bad0 con 0x7f14ec160420 2024-05-17T21:57:27.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4003fe0 0x7f14d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14ebfff640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14d4003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14f0abe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 -- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 msgr2=0x7f14d4003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14ec160420 0x7f14d4003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 -- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d4004d50 msgr2=0x7f14d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d4004d50 0x7f14d4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.910+0000 7f14eb7fe640 1 -- 172.21.15.83:0/2585830346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d4003530 con 0x7f14d4003fe0 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14eb7fe640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4003fe0 0x7f14d4004810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ae0 tx=0x7f14e001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cc6a8ed3a05d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00b2070 con 0x7f14d4003fe0 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e00a5640 con 0x7f14d4003fe0 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14eaffd640 1 -- 172.21.15.83:0/2585830346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00096b0 con 0x7f14d4003fe0 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- 172.21.15.83:0/2585830346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4003fe0 msgr2=0x7f14d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 --2- 172.21.15.83:0/2585830346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4003fe0 0x7f14d4004810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ae0 tx=0x7f14e001ca50 comp rx=0 tx=0).stop 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f0abe640 1 -- 172.21.15.83:0/2585830346 reap_dead start 2024-05-17T21:57:27.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- 172.21.15.83:0/2585830346 shutdown_connections 2024-05-17T21:57:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- 172.21.15.83:0/2585830346 >> 172.21.15.83:0/2585830346 conn(0x7f14ec06cd90 msgr2=0x7f14ec0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- 172.21.15.83:0/2585830346 shutdown_connections 2024-05-17T21:57:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- 172.21.15.83:0/2585830346 wait complete. 2024-05-17T21:57:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 Processor -- start 2024-05-17T21:57:27.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.914+0000 7f14f2521640 1 -- start start 2024-05-17T21:57:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4003fe0 0x7f14ec1f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec154340 0x7f14ec0aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec0b3fa0 con 0x7f14ec15bd60 2024-05-17T21:57:27.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec0d10c0 con 0x7f14ec154340 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec1f22b0 con 0x7f14d4003fe0 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38706/0 (socket says 172.21.15.83:38706) 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 -- 172.21.15.83:0/551198867 learned_addr learned my addr 172.21.15.83:0/551198867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14ebfff640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4003fe0 0x7f14ec1f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 -- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4003fe0 msgr2=0x7f14ec1f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4003fe0 0x7f14ec1f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 -- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec154340 msgr2=0x7f14ec0aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec154340 0x7f14ec0aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 -- 172.21.15.83:0/551198867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec0aa850 con 0x7f14ec15bd60 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f0abe640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14ec071280 tx=0x7f14dc0108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc014810 con 0x7f14ec15bd60 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14dc01a070 con 0x7f14ec15bd60 2024-05-17T21:57:27.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc014bf0 con 0x7f14ec15bd60 2024-05-17T21:57:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec15c6b0 con 0x7f14ec15bd60 2024-05-17T21:57:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec15afa0 con 0x7f14ec15bd60 2024-05-17T21:57:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec0d03b0 con 0x7f14ec15bd60 2024-05-17T21:57:27.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.918+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14dc013070 con 0x7f14ec15bd60 2024-05-17T21:57:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.922+0000 7f14e8ff9640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 0x7f14d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:27.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.922+0000 7f14ebfff640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 0x7f14d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.922+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f14dc091a90 con 0x7f14ec15bd60 2024-05-17T21:57:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.922+0000 7f14ebfff640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 0x7f14d8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009970 tx=0x7f14e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:27.926+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f14ec160d70 con 0x7f14ec15bd60 2024-05-17T21:57:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.086+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f14ec06d6f0 con 0x7f14ec15bd60 2024-05-17T21:57:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.238+0000 7f14e8ff9640 1 -- 172.21.15.83:0/551198867 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f14dc05ed60 con 0x7f14ec15bd60 2024-05-17T21:57:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T21:57:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 msgr2=0x7f14d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 0x7f14d8071910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009970 tx=0x7f14e4008040 comp rx=0 tx=0).stop 2024-05-17T21:57:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 msgr2=0x7f14ec15c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f14ec071280 tx=0x7f14dc0108a0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 shutdown_connections 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ec15bd60 0x7f14ec15c170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14d806f400 0x7f14d8071910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14ec154340 0x7f14ec0aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 --2- 172.21.15.83:0/551198867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4003fe0 0x7f14ec1f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 >> 172.21.15.83:0/551198867 conn(0x7f14ec06cd90 msgr2=0x7f14ec06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 shutdown_connections 2024-05-17T21:57:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.242+0000 7f14f2521640 1 -- 172.21.15.83:0/551198867 wait complete. 2024-05-17T21:57:28.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T21:57:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.426+0000 7f4040f42640 1 Processor -- start 2024-05-17T21:57:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.426+0000 7f4040f42640 1 -- start start 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4040f42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f403c0d0e00 0x7f403c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4040f42640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c0d1210 con 0x7f403c0d2050 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4040f42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c160e50 con 0x7f403c1600f0 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4040f42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f403c160c10 con 0x7f403c0d0e00 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4039d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f403c0d2050 0x7f403c0d2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34914/0 (socket says 172.21.15.83:34914) 2024-05-17T21:57:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4039d9b640 1 -- 172.21.15.83:0/2419019465 learned_addr learned my addr 172.21.15.83:0/2419019465 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767964987 0 0) 0x7f403c0d1210 con 0x7f403c0d2050 2024-05-17T21:57:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 --2- 172.21.15.83:0/2419019465 >> v2:172.21.15.155:3300/0 conn(0x7f403c0d0e00 0x7f403c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4024003530 con 0x7f403c0d2050 2024-05-17T21:57:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3786588731 0 0) 0x7f4024003530 con 0x7f403c0d2050 2024-05-17T21:57:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 >> v2:172.21.15.155:3300/0 conn(0x7f403c0d0e00 msgr2=0x7f403c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 --2- 172.21.15.83:0/2419019465 >> v2:172.21.15.155:3300/0 conn(0x7f403c0d0e00 0x7f403c15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 >> v1:172.21.15.155:6789/0 conn(0x7f403c1600f0 legacy=0x7f403c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0d0320 con 0x7f403c0d2050 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f4030001700 con 0x7f403c0d2050 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 >> v1:172.21.15.83:6789/0 conn(0x7f403c0d2050 legacy=0x7f403c0d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f4024003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4024004810 0x7f4024004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4024005360 0x7f402400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f403c160c10 con 0x7f4024003a60 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f402400c290 con 0x7f4024005360 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f402400c490 con 0x7f4024004810 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f4024003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f4039d9b640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4024004810 0x7f4024004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 -- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4024004810 msgr2=0x7f4024004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4024004810 0x7f4024004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 -- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4024005360 msgr2=0x7f402400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403ad9d640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4024005360 0x7f402400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4024005360 0x7f402400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403ad9d640 1 -- 172.21.15.83:0/2419019465 reap_dead start 2024-05-17T21:57:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.430+0000 7f403a59c640 1 -- 172.21.15.83:0/2419019465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4024003530 con 0x7f4024003a60 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f403a59c640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f4024003ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ae0 tx=0x7f402c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e707307275e257b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0096b0 con 0x7f4024003a60 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c0a6450 con 0x7f4024003a60 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f403959a640 1 -- 172.21.15.83:0/2419019465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0a6840 con 0x7f4024003a60 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 msgr2=0x7f4024003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f4024003ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ae0 tx=0x7f402c01ca70 comp rx=0 tx=0).stop 2024-05-17T21:57:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- 172.21.15.83:0/2419019465 shutdown_connections 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 --2- 172.21.15.83:0/2419019465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f4024003ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- 172.21.15.83:0/2419019465 >> 172.21.15.83:0/2419019465 conn(0x7f403c06dfd0 msgr2=0x7f403c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- 172.21.15.83:0/2419019465 shutdown_connections 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- 172.21.15.83:0/2419019465 wait complete. 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 Processor -- start 2024-05-17T21:57:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.434+0000 7f4040f42640 1 -- start start 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f403c0cd290 0x7f403c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 0x7f403c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0b42e0 con 0x7f4024003a60 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0d0950 con 0x7f403c0cd6a0 2024-05-17T21:57:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f403c0b4460 con 0x7f403c0cd290 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38728/0 (socket says 172.21.15.83:38728) 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 -- 172.21.15.83:0/1337463583 learned_addr learned my addr 172.21.15.83:0/1337463583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403ad9d640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 0x7f403c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4039d9b640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f403c0cd290 0x7f403c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 -- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f403c0cd290 msgr2=0x7f403c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f403c0cd290 0x7f403c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 -- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 msgr2=0x7f403c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 0x7f403c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 -- 172.21.15.83:0/1337463583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0b6350 con 0x7f4024003a60 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403ad9d640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 0x7f403c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f403a59c640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ae0 tx=0x7f402c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0a6d00 con 0x7f4024003a60 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c0b9440 con 0x7f4024003a60 2024-05-17T21:57:28.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0b7c40 con 0x7f4024003a60 2024-05-17T21:57:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0b9810 con 0x7f4024003a60 2024-05-17T21:57:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c0b47f0 con 0x7f4024003a60 2024-05-17T21:57:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c0d3120 con 0x7f4024003a60 2024-05-17T21:57:28.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.438+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f402c0a7430 con 0x7f4024003a60 2024-05-17T21:57:28.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.442+0000 7f402affd640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 0x7f401c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:28.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.442+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f402c129250 con 0x7f4024003a60 2024-05-17T21:57:28.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.442+0000 7f4039d9b640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 0x7f401c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:28.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.442+0000 7f4039d9b640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 0x7f401c071940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4030001250 tx=0x7f4030009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:28.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.446+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f402c0bf060 con 0x7f4024003a60 2024-05-17T21:57:28.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:28.606+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f403c0d64a0 con 0x7f4024003a60 2024-05-17T21:57:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.249+0000 7f402affd640 1 -- 172.21.15.83:0/1337463583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f402c0ac020 con 0x7f4024003a60 2024-05-17T21:57:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T21:57:29.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 msgr2=0x7f401c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 0x7f401c071940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4030001250 tx=0x7f4030009040 comp rx=0 tx=0).stop 2024-05-17T21:57:29.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 msgr2=0x7f403c152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f402c009ae0 tx=0x7f402c019810 comp rx=0 tx=0).stop 2024-05-17T21:57:29.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 shutdown_connections 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f401c06f430 0x7f401c071940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f403c0cd6a0 0x7f403c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f403c0cd290 0x7f403c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 --2- 172.21.15.83:0/1337463583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4024003a60 0x7f403c152c40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 >> 172.21.15.83:0/1337463583 conn(0x7f403c06dfd0 msgr2=0x7f403c15dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 shutdown_connections 2024-05-17T21:57:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.253+0000 7f4040f42640 1 -- 172.21.15.83:0/1337463583 wait complete. 2024-05-17T21:57:29.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T21:57:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.437+0000 7fe691586640 1 Processor -- start 2024-05-17T21:57:29.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.437+0000 7fe691586640 1 -- start start 2024-05-17T21:57:29.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:57:29.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe691586640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe68c073590 0x7fe68c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:29.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe691586640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68c0d10d0 con 0x7fe68c143f20 2024-05-17T21:57:29.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe691586640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68c070120 con 0x7fe68c0b3d00 2024-05-17T21:57:29.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe691586640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe68c0b4110 con 0x7fe68c073590 2024-05-17T21:57:29.433 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:57:29.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe68c073590 0x7fe68c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:29.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe68c143f20 0x7fe68c1447f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34916/0 (socket says 172.21.15.83:34916) 2024-05-17T21:57:29.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68b7fe640 1 -- 172.21.15.83:0/2928122905 learned_addr learned my addr 172.21.15.83:0/2928122905 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:29.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68a7fc640 1 -- 172.21.15.83:0/2928122905 >> v1:172.21.15.155:6789/0 conn(0x7fe68c0b3d00 legacy=0x7fe68c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:29.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68a7fc640 1 -- 172.21.15.83:0/2928122905 >> v1:172.21.15.83:6789/0 conn(0x7fe68c143f20 legacy=0x7fe68c1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.441+0000 7fe68a7fc640 1 -- 172.21.15.83:0/2928122905 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c070400 con 0x7fe68c073590 2024-05-17T21:57:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe68a7fc640 1 --2- 172.21.15.83:0/2928122905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c073590 0x7fe68c0d0a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6800097c0 tx=0x7fe68001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5948cb520f192b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe689ffb640 1 -- 172.21.15.83:0/2928122905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800b0070 con 0x7fe68c073590 2024-05-17T21:57:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe689ffb640 1 -- 172.21.15.83:0/2928122905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6800ab600 con 0x7fe68c073590 2024-05-17T21:57:29.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe689ffb640 1 -- 172.21.15.83:0/2928122905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800ab9d0 con 0x7fe68c073590 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- 172.21.15.83:0/2928122905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c073590 msgr2=0x7fe68c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 --2- 172.21.15.83:0/2928122905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c073590 0x7fe68c0d0a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6800097c0 tx=0x7fe68001c4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- 172.21.15.83:0/2928122905 shutdown_connections 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 --2- 172.21.15.83:0/2928122905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c073590 0x7fe68c0d0a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- 172.21.15.83:0/2928122905 >> 172.21.15.83:0/2928122905 conn(0x7fe68c06cd90 msgr2=0x7fe68c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- 172.21.15.83:0/2928122905 shutdown_connections 2024-05-17T21:57:29.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- 172.21.15.83:0/2928122905 wait complete. 2024-05-17T21:57:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 Processor -- start 2024-05-17T21:57:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.445+0000 7fe691586640 1 -- start start 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe68c073590 0x7fe68c143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68c0b3d00 0x7fe68c158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c16b820 con 0x7fe68c143f20 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c16b6a0 con 0x7fe68c0b3d00 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe68c16b9a0 con 0x7fe68c073590 2024-05-17T21:57:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38744/0 (socket says 172.21.15.83:38744) 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 -- 172.21.15.83:0/2081977921 learned_addr learned my addr 172.21.15.83:0/2081977921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68affd640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe68c073590 0x7fe68c143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68a7fc640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68c0b3d00 0x7fe68c158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 -- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe68c073590 msgr2=0x7fe68c143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe68c073590 0x7fe68c143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 -- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68c0b3d00 msgr2=0x7fe68c158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68c0b3d00 0x7fe68c158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 -- 172.21.15.83:0/2081977921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c0aa310 con 0x7fe68c143f20 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe68b7fe640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe68400cb10 tx=0x7fe68400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684017070 con 0x7fe68c143f20 2024-05-17T21:57:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68400ac10 con 0x7fe68c143f20 2024-05-17T21:57:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c143580 con 0x7fe68c143f20 2024-05-17T21:57:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684012400 con 0x7fe68c143f20 2024-05-17T21:57:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c0aaa60 con 0x7fe68c143f20 2024-05-17T21:57:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c0aa4e0 con 0x7fe68c143f20 2024-05-17T21:57:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.449+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe68401a070 con 0x7fe68c143f20 2024-05-17T21:57:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.453+0000 7fe6737fe640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 0x7fe678071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.453+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fe68408ee90 con 0x7fe68c143f20 2024-05-17T21:57:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.453+0000 7fe68affd640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 0x7fe678071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.453+0000 7fe68affd640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 0x7fe678071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00b810 tx=0x7fe67c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.457+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe68c0aa4e0 con 0x7fe68c143f20 2024-05-17T21:57:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:29.653+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe68c0d0a10 con 0x7fe68c143f20 2024-05-17T21:57:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.249+0000 7fe6737fe640 1 -- 172.21.15.83:0/2081977921 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe68405b790 con 0x7fe68c143f20 2024-05-17T21:57:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-05-17T21:57:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 msgr2=0x7fe678071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 0x7fe678071910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00b810 tx=0x7fe67c008040 comp rx=0 tx=0).stop 2024-05-17T21:57:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 msgr2=0x7fe68c13e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe68400cb10 tx=0x7fe68400a060 comp rx=0 tx=0).stop 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 shutdown_connections 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe68c143f20 0x7fe68c13e790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe67806f400 0x7fe678071910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe68c0b3d00 0x7fe68c158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 --2- 172.21.15.83:0/2081977921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe68c073590 0x7fe68c143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 >> 172.21.15.83:0/2081977921 conn(0x7fe68c06cd90 msgr2=0x7fe68c13fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 shutdown_connections 2024-05-17T21:57:30.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.253+0000 7fe691586640 1 -- 172.21.15.83:0/2081977921 wait complete. 2024-05-17T21:57:30.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:30.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 Processor -- start 2024-05-17T21:57:30.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 -- start start 2024-05-17T21:57:30.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12981605a0 0x7f12981609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12981465d0 con 0x7f129806ffd0 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1298160ef0 con 0x7f12981461c0 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129cdbb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f129806ef30 con 0x7f12981605a0 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f129806ffd0 0x7f12980703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34922/0 (socket says 172.21.15.83:34922) 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 -- 172.21.15.83:0/2579973405 learned_addr learned my addr 172.21.15.83:0/2579973405 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1296d9d640 1 --2- 172.21.15.83:0/2579973405 >> v2:172.21.15.155:3300/0 conn(0x7f12981605a0 0x7f12981609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184262472 0 0) 0x7f12981465d0 con 0x7f129806ffd0 2024-05-17T21:57:30.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1284005740 con 0x7f129806ffd0 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 531684144 0 0) 0x7f1284005740 con 0x7f129806ffd0 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 >> v2:172.21.15.155:3300/0 conn(0x7f12981605a0 msgr2=0x7f12981609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 --2- 172.21.15.83:0/2579973405 >> v2:172.21.15.155:3300/0 conn(0x7f12981605a0 0x7f12981609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 >> v1:172.21.15.155:6789/0 conn(0x7f12981461c0 legacy=0x7f129815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129806f210 con 0x7f129806ffd0 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1280001700 con 0x7f129806ffd0 2024-05-17T21:57:30.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 >> v1:172.21.15.83:6789/0 conn(0x7f129806ffd0 legacy=0x7f12980703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284005c70 0x7f12840060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 0x7f1284006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1284007470 0x7f128400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f129806ef30 con 0x7f1284006a20 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f128400e400 con 0x7f1284007470 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f128400e600 con 0x7f1284005c70 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 0x7f1284006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f129659c640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284005c70 0x7f12840060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1296d9d640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1284007470 0x7f128400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 -- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284005c70 msgr2=0x7f12840060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284005c70 0x7f12840060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 -- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1284007470 msgr2=0x7f128400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1284007470 0x7f128400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1295d9b640 1 -- 172.21.15.83:0/2579973405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1284005740 con 0x7f1284006a20 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1296d9d640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1284007470 0x7f128400dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:30.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.441+0000 7f1296d9d640 1 -- 172.21.15.83:0/2579973405 reap_dead start 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f1295d9b640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 0x7f1284006e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1280001060 tx=0x7f128001f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b5f24f8d9a859a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280007d20 con 0x7f1284006a20 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280005ae0 con 0x7f1284006a20 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129559a640 1 -- 172.21.15.83:0/2579973405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280007070 con 0x7f1284006a20 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 -- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 msgr2=0x7f1284006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 0x7f1284006e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1280001060 tx=0x7f128001f290 comp rx=0 tx=0).stop 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 -- 172.21.15.83:0/2579973405 shutdown_connections 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 --2- 172.21.15.83:0/2579973405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1284006a20 0x7f1284006e30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 -- 172.21.15.83:0/2579973405 >> 172.21.15.83:0/2579973405 conn(0x7f129806cd90 msgr2=0x7f12980b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 -- 172.21.15.83:0/2579973405 shutdown_connections 2024-05-17T21:57:30.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.445+0000 7f129cdbb640 1 -- 172.21.15.83:0/2579973405 wait complete. 2024-05-17T21:57:30.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 Processor -- start 2024-05-17T21:57:30.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- start start 2024-05-17T21:57:30.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284006a20 0x7f12980b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 0x7f12980d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f129806fab0 con 0x7f129813f560 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f129806f7d0 con 0x7f12980d0890 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f12981609b0 con 0x7f1284006a20 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38758/0 (socket says 172.21.15.83:38758) 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 -- 172.21.15.83:0/1261021848 learned_addr learned my addr 172.21.15.83:0/1261021848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1296d9d640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 0x7f12980d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129659c640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284006a20 0x7f12980b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 -- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284006a20 msgr2=0x7f12980b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284006a20 0x7f12980b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 -- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 msgr2=0x7f12980d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 0x7f12980d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 -- 172.21.15.83:0/1261021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12980d11e0 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1296d9d640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 0x7f12980d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f1295d9b640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f128001d990 tx=0x7f1284007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800047f0 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280004990 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12980aa3b0 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280004d40 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.449+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12980b3300 con 0x7f129813f560 2024-05-17T21:57:30.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12980703e0 con 0x7f129813f560 2024-05-17T21:57:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f12800070e0 con 0x7f129813f560 2024-05-17T21:57:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f128effd640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 0x7f1278071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f128012ca90 con 0x7f129813f560 2024-05-17T21:57:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f129659c640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 0x7f1278071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.453+0000 7f129659c640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 0x7f1278071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f12980b11f0 tx=0x7f1288009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.457+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f12800ca080 con 0x7f129813f560 2024-05-17T21:57:30.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.621+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12980b41a0 con 0x7f129813f560 2024-05-17T21:57:30.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f128effd640 1 -- 172.21.15.83:0/1261021848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16594 (secure 0 0 0) 0x7f128012c020 con 0x7f129813f560 2024-05-17T21:57:30.616 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:30.616 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":19,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:29.658235+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 msgr2=0x7f1278071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 0x7f1278071940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f12980b11f0 tx=0x7f1288009040 comp rx=0 tx=0).stop 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 msgr2=0x7f12981fd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f128001d990 tx=0x7f1284007a50 comp rx=0 tx=0).stop 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 shutdown_connections 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f127806f430 0x7f1278071940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12980d0890 0x7f12980d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f129813f560 0x7f12981fd2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 --2- 172.21.15.83:0/1261021848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1284006a20 0x7f12980b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 >> 172.21.15.83:0/1261021848 conn(0x7f129806cd90 msgr2=0x7f1298142650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 shutdown_connections 2024-05-17T21:57:30.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.625+0000 7f129cdbb640 1 -- 172.21.15.83:0/1261021848 wait complete. 2024-05-17T21:57:30.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 Processor -- start 2024-05-17T21:57:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 -- start start 2024-05-17T21:57:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9ba40d1040 0x7f9ba4144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba415ea40 con 0x7f9ba4073540 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba415ed20 con 0x7f9ba415e080 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9baab1f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba415f000 con 0x7f9ba40d1040 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9ba90bc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9ba4073540 0x7f9ba406f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34930/0 (socket says 172.21.15.83:34930) 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9ba90bc640 1 -- 172.21.15.83:0/1235275720 learned_addr learned my addr 172.21.15.83:0/1235275720 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.793+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40d1040 0x7f9ba4144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166657179 0 0) 0x7f9ba415ea40 con 0x7f9ba4073540 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c003530 con 0x7f9ba4073540 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767004510 0 0) 0x7f9ba415ed20 con 0x7f9ba415e080 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ba415ea40 con 0x7f9ba415e080 2024-05-17T21:57:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1632823830 0 0) 0x7f9b8c003530 con 0x7f9ba4073540 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40d1040 msgr2=0x7f9ba4144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1235275720 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40d1040 0x7f9ba4144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 >> v1:172.21.15.155:6789/0 conn(0x7f9ba415e080 legacy=0x7f9ba4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba406ef30 con 0x7f9ba4073540 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9b9c001a80 con 0x7f9ba4073540 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40d1040 0x7f9ba4144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 >> v1:172.21.15.83:6789/0 conn(0x7f9ba4073540 legacy=0x7f9ba406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 0x7f9b8c004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b8c005290 0x7f9b8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba415f000 con 0x7f9b8c003c40 2024-05-17T21:57:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c00c240 con 0x7f9b8c004810 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c00c440 con 0x7f9b8c005290 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 0x7f9b8c004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba90bc640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b8c005290 0x7f9b8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 -- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b8c005290 msgr2=0x7f9b8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b8c005290 0x7f9b8c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 -- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c004810 msgr2=0x7f9b8c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c004810 0x7f9b8c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 -- 172.21.15.83:0/1235275720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c003530 con 0x7f9b8c003c40 2024-05-17T21:57:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba90bc640 1 -- 172.21.15.83:0/1235275720 reap_dead start 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 0x7f9b8c004090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009cb0 tx=0x7f9b9401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ad53f89e53ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b2070 con 0x7f9b8c003c40 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b940a58a0 con 0x7f9b8c003c40 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.797+0000 7f9ba37fe640 1 -- 172.21.15.83:0/1235275720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94009880 con 0x7f9b8c003c40 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 msgr2=0x7f9b8c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 0x7f9b8c004090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009cb0 tx=0x7f9b9401cbe0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- 172.21.15.83:0/1235275720 shutdown_connections 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 --2- 172.21.15.83:0/1235275720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b8c003c40 0x7f9b8c004090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- 172.21.15.83:0/1235275720 >> 172.21.15.83:0/1235275720 conn(0x7f9ba406cd90 msgr2=0x7f9ba40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- 172.21.15.83:0/1235275720 shutdown_connections 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- 172.21.15.83:0/1235275720 wait complete. 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 Processor -- start 2024-05-17T21:57:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- start start 2024-05-17T21:57:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c003c40 0x7f9ba413d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba41fadf0 0x7f9ba41fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4161840 con 0x7f9ba41fa790 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba41616c0 con 0x7f9b8c003c40 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9baab1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba41619c0 con 0x7f9ba41fadf0 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38780/0 (socket says 172.21.15.83:38780) 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 -- 172.21.15.83:0/308133164 learned_addr learned my addr 172.21.15.83:0/308133164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 -- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba41fadf0 msgr2=0x7f9ba41fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba41fadf0 0x7f9ba41fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 -- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c003c40 msgr2=0x7f9ba413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c003c40 0x7f9ba413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.801+0000 7f9ba90bc640 1 -- 172.21.15.83:0/308133164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba413d690 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba90bc640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0012f0 tx=0x7f9b9c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00de70 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c00bb80 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba41faba0 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c007070 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba4140740 con 0x7f9ba41fa790 2024-05-17T21:57:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba40ab8c0 con 0x7f9ba41fa790 2024-05-17T21:57:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b9c01b020 con 0x7f9ba41fa790 2024-05-17T21:57:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c08f430 con 0x7f9ba41fa790 2024-05-17T21:57:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.805+0000 7f9ba88bb640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b940af000 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.809+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9ba41faba0 con 0x7f9ba41fa790 2024-05-17T21:57:30.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.973+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ba4161cf0 con 0x7f9ba41fa790 2024-05-17T21:57:30.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.973+0000 7f9ba17fa640 1 -- 172.21.15.83:0/308133164 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16594 (secure 0 0 0) 0x7f9b9c05c440 con 0x7f9ba41fa790 2024-05-17T21:57:30.965 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:30.965 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":19,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:29.658235+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 msgr2=0x7f9b90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b940af000 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 msgr2=0x7f9ba413dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0012f0 tx=0x7f9b9c00b4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 shutdown_connections 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba41fa790 0x7f9ba413dd10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba41fadf0 0x7f9ba41fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 --2- 172.21.15.83:0/308133164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8c003c40 0x7f9ba413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 >> 172.21.15.83:0/308133164 conn(0x7f9ba406cd90 msgr2=0x7f9ba40b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:30.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 shutdown_connections 2024-05-17T21:57:30.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:30.977+0000 7f9baab1f640 1 -- 172.21.15.83:0/308133164 wait complete. 2024-05-17T21:57:30.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T21:57:31.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.133+0000 7f07d7e05640 1 Processor -- start 2024-05-17T21:57:31.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.133+0000 7f07d7e05640 1 -- start start 2024-05-17T21:57:31.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d7e05640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 0x7f07d015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d7e05640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d00b6180 con 0x7f07d015ebc0 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d7e05640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d00b6460 con 0x7f07d0158160 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d7e05640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07d00b6740 con 0x7f07d00b5c70 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d63a2640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f07d015ebc0 0x7f07d006c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34946/0 (socket says 172.21.15.83:34946) 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d63a2640 1 -- 172.21.15.83:0/897627154 learned_addr learned my addr 172.21.15.83:0/897627154 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 0x7f07d015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689099557 0 0) 0x7f07d00b6180 con 0x7f07d015ebc0 2024-05-17T21:57:31.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07bc003530 con 0x7f07d015ebc0 2024-05-17T21:57:31.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 500023783 0 0) 0x7f07bc003530 con 0x7f07d015ebc0 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 msgr2=0x7f07d015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 0x7f07d015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 >> v1:172.21.15.155:6789/0 conn(0x7f07d0158160 legacy=0x7f07d015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d00b6a20 con 0x7f07d015ebc0 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f07c8001a80 con 0x7f07d015ebc0 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 >> v1:172.21.15.83:6789/0 conn(0x7f07d015ebc0 legacy=0x7f07d006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07bc003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc004810 0x7f07bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 0x7f07bc0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07d00b6740 con 0x7f07bc003a40 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc00bb90 con 0x7f07bc004810 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc00bd90 con 0x7f07d00b5c70 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07bc003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:31.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d53a0640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc004810 0x7f07bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 -- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 msgr2=0x7f07bc0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d00b5c70 0x7f07bc0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 -- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc004810 msgr2=0x7f07bc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc004810 0x7f07bc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d5ba1640 1 -- 172.21.15.83:0/897627154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc003530 con 0x7f07bc003a40 2024-05-17T21:57:31.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.137+0000 7f07d53a0640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc004810 0x7f07bc004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07bc003eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73cdb187bb89de9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0096b0 con 0x7f07bc003a40 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a6490 con 0x7f07bc003a40 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d4b9f640 1 -- 172.21.15.83:0/897627154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a6880 con 0x7f07bc003a40 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- 172.21.15.83:0/897627154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 msgr2=0x7f07bc003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 --2- 172.21.15.83:0/897627154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07bc003eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07cc01cab0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d63a2640 1 -- 172.21.15.83:0/897627154 reap_dead start 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- 172.21.15.83:0/897627154 shutdown_connections 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- 172.21.15.83:0/897627154 >> 172.21.15.83:0/897627154 conn(0x7f07d00d1ed0 msgr2=0x7f07d00d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- 172.21.15.83:0/897627154 shutdown_connections 2024-05-17T21:57:31.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- 172.21.15.83:0/897627154 wait complete. 2024-05-17T21:57:31.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 Processor -- start 2024-05-17T21:57:31.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.141+0000 7f07d7e05640 1 -- start start 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07d01679f0 0x7f07d015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d0167e00 0x7f07d015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07d006dad0 con 0x7f07bc003a40 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07d006d950 con 0x7f07d01679f0 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07d01696d0 con 0x7f07d0167e00 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38808/0 (socket says 172.21.15.83:38808) 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 -- 172.21.15.83:0/806484864 learned_addr learned my addr 172.21.15.83:0/806484864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 -- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d0167e00 msgr2=0x7f07d015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d0167e00 0x7f07d015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 -- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07d01679f0 msgr2=0x7f07d015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:31.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07d01679f0 0x7f07d015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 -- 172.21.15.83:0/806484864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d01692e0 con 0x7f07bc003a40 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d5ba1640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0096b0 con 0x7f07bc003a40 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a6e40 con 0x7f07bc003a40 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d006f780 con 0x7f07bc003a40 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a7670 con 0x7f07bc003a40 2024-05-17T21:57:31.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d006fd40 con 0x7f07bc003a40 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d006c820 con 0x7f07bc003a40 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f07cc0c2060 con 0x7f07bc003a40 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 0x7f07b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f07cc1294d0 con 0x7f07bc003a40 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d53a0640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 0x7f07b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:31.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.145+0000 7f07d53a0640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 0x7f07b4071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07d0160760 tx=0x7f07c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:31.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.149+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f07d01692e0 con 0x7f07bc003a40 2024-05-17T21:57:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:31.345+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f07d00d2580 con 0x7f07bc003a40 2024-05-17T21:57:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.289+0000 7f07c67fc640 1 -- 172.21.15.83:0/806484864 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f07cc0b6070 con 0x7f07bc003a40 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 msgr2=0x7f07b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 0x7f07b4071940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07d0160760 tx=0x7f07c0009040 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 msgr2=0x7f07d0157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ae0 tx=0x7f07bc005840 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 shutdown_connections 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07d0167e00 0x7f07d015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f07b406f430 0x7f07b4071940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07d01679f0 0x7f07d015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 --2- 172.21.15.83:0/806484864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc003a40 0x7f07d0157860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 >> 172.21.15.83:0/806484864 conn(0x7f07d00d1ed0 msgr2=0x7f07d0160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 shutdown_connections 2024-05-17T21:57:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.293+0000 7f07d7e05640 1 -- 172.21.15.83:0/806484864 wait complete. 2024-05-17T21:57:32.309 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T21:57:32.309 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T21:57:32.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 Processor -- start 2024-05-17T21:57:32.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 -- start start 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220073590 0x7f12200d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1220143f20 0x7f12201447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12200d10d0 con 0x7f12200b3d00 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f12200b4110 con 0x7f1220143f20 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12269ef640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1220070770 con 0x7f1220073590 2024-05-17T21:57:32.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f12177fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220073590 0x7f12200d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1220143f20 0x7f12201447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1220143f20 0x7f12201447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:45328/0 (socket says 172.21.15.83:45328) 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 -- 172.21.15.83:0/1781452895 learned_addr learned my addr 172.21.15.83:0/1781452895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f121ffff640 1 --1- 172.21.15.83:0/1781452895 >> v1:172.21.15.155:6789/0 conn(0x7f12200b3d00 0x7f1220072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:51954/0 (socket says 172.21.15.83:51954) 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 -- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220073590 msgr2=0x7f12200d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:32.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 --2- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220073590 0x7f12200d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 -- 172.21.15.83:0/1781452895 >> v1:172.21.15.155:6789/0 conn(0x7f12200b3d00 legacy=0x7f1220072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.477+0000 7f1224f8c640 1 -- 172.21.15.83:0/1781452895 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1220070120 con 0x7f1220143f20 2024-05-17T21:57:32.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f1224f8c640 1 --2- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f12201447f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1218009e50 tx=0x7f121801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c70e56dde0af0ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:32.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f121f7fe640 1 -- 172.21.15.83:0/1781452895 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12180a5790 con 0x7f1220143f20 2024-05-17T21:57:32.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f121f7fe640 1 -- 172.21.15.83:0/1781452895 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12180a5930 con 0x7f1220143f20 2024-05-17T21:57:32.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f121f7fe640 1 -- 172.21.15.83:0/1781452895 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12180a5d00 con 0x7f1220143f20 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 -- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 msgr2=0x7f12201447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 --2- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f12201447f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1218009e50 tx=0x7f121801ca30 comp rx=0 tx=0).stop 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 -- 172.21.15.83:0/1781452895 shutdown_connections 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 --2- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f12201447f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 --2- 172.21.15.83:0/1781452895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220073590 0x7f12200d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 -- 172.21.15.83:0/1781452895 >> 172.21.15.83:0/1781452895 conn(0x7f122006cd90 msgr2=0x7f12200b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:32.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 -- 172.21.15.83:0/1781452895 shutdown_connections 2024-05-17T21:57:32.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.481+0000 7f12269ef640 1 -- 172.21.15.83:0/1781452895 wait complete. 2024-05-17T21:57:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 Processor -- start 2024-05-17T21:57:32.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- start start 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 0x7f122013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f122015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1220070c00 con 0x7f1220073590 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1220145540 con 0x7f1220143f20 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f122015be90 con 0x7f12200b3d00 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38814/0 (socket says 172.21.15.83:38814) 2024-05-17T21:57:32.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 -- 172.21.15.83:0/2508513798 learned_addr learned my addr 172.21.15.83:0/2508513798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12177fe640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 0x7f122013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f1224f8c640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f122015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 -- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 msgr2=0x7f122013f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 0x7f122013f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 -- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 msgr2=0x7f122015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f122015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 -- 172.21.15.83:0/2508513798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122013e5d0 con 0x7f1220073590 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12177fe640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 0x7f122013f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121ffff640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f121000c430 tx=0x7f121000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12100074e0 con 0x7f1220073590 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1210007680 con 0x7f1220073590 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210004a10 con 0x7f1220073590 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122013e8b0 con 0x7f1220073590 2024-05-17T21:57:32.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.485+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122013ee70 con 0x7f1220073590 2024-05-17T21:57:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12201447f0 con 0x7f1220073590 2024-05-17T21:57:32.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f121001a020 con 0x7f1220073590 2024-05-17T21:57:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f121d7fa640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 0x7f1200071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f121008eb00 con 0x7f1220073590 2024-05-17T21:57:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f12177fe640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 0x7f1200071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:32.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.489+0000 7f12177fe640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 0x7f1200071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1220160c50 tx=0x7f1208008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:32.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.493+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f121005b9f0 con 0x7f1220073590 2024-05-17T21:57:32.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:32.673+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f12200b07b0 con 0x7f1220073590 2024-05-17T21:57:33.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.297+0000 7f121d7fa640 1 -- 172.21.15.83:0/2508513798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f121005b2f0 con 0x7f1220073590 2024-05-17T21:57:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 msgr2=0x7f1200071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 0x7f1200071910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1220160c50 tx=0x7f1208008040 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 msgr2=0x7f12200b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f121000c430 tx=0x7f121000c8f0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 shutdown_connections 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f120006f400 0x7f1200071910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220143f20 0x7f122015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12200b3d00 0x7f122013f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 --2- 172.21.15.83:0/2508513798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220073590 0x7f12200b5500 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 >> 172.21.15.83:0/2508513798 conn(0x7f122006cd90 msgr2=0x7f12200d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 shutdown_connections 2024-05-17T21:57:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.301+0000 7f12269ef640 1 -- 172.21.15.83:0/2508513798 wait complete. 2024-05-17T21:57:33.316 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T21:57:33.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:33.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 Processor -- start 2024-05-17T21:57:33.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 -- start start 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c06c790 0x7f585c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f585c156c00 0x7f585c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c0af9c0 con 0x7f585c157010 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f585c0afca0 con 0x7f585c156c00 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f5861efb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f585c0aff80 con 0x7f585c06c790 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f585c156c00 0x7f585c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f585c156c00 0x7f585c0b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38828/0 (socket says 172.21.15.83:38828) 2024-05-17T21:57:33.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 -- 172.21.15.83:0/1118946762 learned_addr learned my addr 172.21.15.83:0/1118946762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585b7fe640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c06c790 0x7f585c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 -- 172.21.15.83:0/1118946762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c06c790 msgr2=0x7f585c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c06c790 0x7f585c0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 -- 172.21.15.83:0/1118946762 >> v1:172.21.15.155:6789/0 conn(0x7f585c157010 legacy=0x7f585c161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.477+0000 7f585affd640 1 -- 172.21.15.83:0/1118946762 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c0b0260 con 0x7f585c156c00 2024-05-17T21:57:33.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f585affd640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c156c00 0x7f585c0b0a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58440098a0 tx=0x7f584401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981a7d9005eb6330 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f585a7fc640 1 -- 172.21.15.83:0/1118946762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440a95f0 con 0x7f585c156c00 2024-05-17T21:57:33.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f585a7fc640 1 -- 172.21.15.83:0/1118946762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58440a9790 con 0x7f585c156c00 2024-05-17T21:57:33.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f585a7fc640 1 -- 172.21.15.83:0/1118946762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440a9b60 con 0x7f585c156c00 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 -- 172.21.15.83:0/1118946762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c156c00 msgr2=0x7f585c0b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c156c00 0x7f585c0b0a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58440098a0 tx=0x7f584401c510 comp rx=0 tx=0).stop 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 -- 172.21.15.83:0/1118946762 shutdown_connections 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c156c00 0x7f585c0b0a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 --2- 172.21.15.83:0/1118946762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c06c790 0x7f585c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 -- 172.21.15.83:0/1118946762 >> 172.21.15.83:0/1118946762 conn(0x7f585c09b8b0 msgr2=0x7f585c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 -- 172.21.15.83:0/1118946762 shutdown_connections 2024-05-17T21:57:33.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.481+0000 7f5861efb640 1 -- 172.21.15.83:0/1118946762 wait complete. 2024-05-17T21:57:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 Processor -- start 2024-05-17T21:57:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- start start 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c156c00 0x7f585c15f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c157010 0x7f585c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0d1820 con 0x7f585c06c790 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f585c161d80 con 0x7f585c156c00 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f585c0d19a0 con 0x7f585c157010 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38832/0 (socket says 172.21.15.83:38832) 2024-05-17T21:57:33.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 -- 172.21.15.83:0/137537610 learned_addr learned my addr 172.21.15.83:0/137537610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585affd640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c156c00 0x7f585c15f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585bfff640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c157010 0x7f585c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 -- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c157010 msgr2=0x7f585c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c157010 0x7f585c15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 -- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c156c00 msgr2=0x7f585c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c156c00 0x7f585c15f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 -- 172.21.15.83:0/137537610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c14b080 con 0x7f585c06c790 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f585b7fe640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5854007e70 tx=0x7f585400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854014620 con 0x7f585c06c790 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5854019070 con 0x7f585c06c790 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c15e4b0 con 0x7f585c06c790 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854014a00 con 0x7f585c06c790 2024-05-17T21:57:33.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.485+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c15ea70 con 0x7f585c06c790 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c0af3e0 con 0x7f585c06c790 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5854014ba0 con 0x7f585c06c790 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f5853fff640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 0x7f583c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f585408fb60 con 0x7f585c06c790 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f585affd640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 0x7f583c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.489+0000 7f585affd640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 0x7f583c071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58440ae000 tx=0x7f58440a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.493+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f585405ca40 con 0x7f585c06c790 2024-05-17T21:57:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.685+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f585c076820 con 0x7f585c06c790 2024-05-17T21:57:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.685+0000 7f5853fff640 1 -- 172.21.15.83:0/137537610 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f585405c340 con 0x7f585c06c790 2024-05-17T21:57:33.677 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:33.678 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:57:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 msgr2=0x7f583c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 0x7f583c071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58440ae000 tx=0x7f58440a7000 comp rx=0 tx=0).stop 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 msgr2=0x7f585c14ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5854007e70 tx=0x7f585400e4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 shutdown_connections 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585c157010 0x7f585c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f583c06f400 0x7f583c071910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f585c156c00 0x7f585c15f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 --2- 172.21.15.83:0/137537610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f585c06c790 0x7f585c14ab40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 >> 172.21.15.83:0/137537610 conn(0x7f585c09b8b0 msgr2=0x7f585c075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 shutdown_connections 2024-05-17T21:57:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.689+0000 7f5861efb640 1 -- 172.21.15.83:0/137537610 wait complete. 2024-05-17T21:57:33.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:33.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 Processor -- start 2024-05-17T21:57:33.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 -- start start 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcbb415ffa0 0x7fcbb40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb40d1150 0x7fcbb406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb406e610 con 0x7fcbb406e200 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb40d34e0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbbc7ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb40d3780 con 0x7fcbb40d1150 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcbb415ffa0 0x7fcbb40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcbb415ffa0 0x7fcbb40d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38836/0 (socket says 172.21.15.83:38836) 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 -- 172.21.15.83:0/183518005 learned_addr learned my addr 172.21.15.83:0/183518005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbb9d87640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb40d1150 0x7fcbb406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 -- 172.21.15.83:0/183518005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb40d1150 msgr2=0x7fcbb406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:33.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb40d1150 0x7fcbb406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 -- 172.21.15.83:0/183518005 >> v1:172.21.15.155:6789/0 conn(0x7fcbb406e200 legacy=0x7fcbb4163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.841+0000 7fcbba588640 1 -- 172.21.15.83:0/183518005 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb40d3a60 con 0x7fcbb415ffa0 2024-05-17T21:57:33.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.845+0000 7fcbba588640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40d0c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009e50 tx=0x7fcba401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bfdc1b6518b9d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.845+0000 7fcbb9586640 1 -- 172.21.15.83:0/183518005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40a5700 con 0x7fcbb415ffa0 2024-05-17T21:57:33.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.845+0000 7fcbb9586640 1 -- 172.21.15.83:0/183518005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba40a58a0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.845+0000 7fcbb9586640 1 -- 172.21.15.83:0/183518005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40a5c70 con 0x7fcbb415ffa0 2024-05-17T21:57:33.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/183518005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 msgr2=0x7fcbb40d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40d0c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009e50 tx=0x7fcba401ca30 comp rx=0 tx=0).stop 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/183518005 shutdown_connections 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb40d1150 0x7fcbb406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/183518005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40d0c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/183518005 >> 172.21.15.83:0/183518005 conn(0x7fcbb40d28f0 msgr2=0x7fcbb40d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/183518005 shutdown_connections 2024-05-17T21:57:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/183518005 wait complete. 2024-05-17T21:57:33.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 Processor -- start 2024-05-17T21:57:33.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- start start 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb406e200 0x7fcbb40c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb40d1150 0x7fcbb4158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb416cd00 con 0x7fcbb415ffa0 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb416cb80 con 0x7fcbb40d1150 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbc7ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb416ce80 con 0x7fcbb406e200 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38850/0 (socket says 172.21.15.83:38850) 2024-05-17T21:57:33.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 -- 172.21.15.83:0/1879921082 learned_addr learned my addr 172.21.15.83:0/1879921082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbba588640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb406e200 0x7fcbb40c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbb9d87640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb40d1150 0x7fcbb4158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 -- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb406e200 msgr2=0x7fcbb40c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb406e200 0x7fcbb40c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 -- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb40d1150 msgr2=0x7fcbb4158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb40d1150 0x7fcbb4158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 -- 172.21.15.83:0/1879921082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb4159270 con 0x7fcbb415ffa0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbbad89640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c370 tx=0x7fcbac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.849+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac017070 con 0x7fcbb415ffa0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac0052d0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41619b0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac003820 con 0x7fcbb415ffa0 2024-05-17T21:57:33.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb4161f70 con 0x7fcbb415ffa0 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb40ca5d0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcbac01a070 con 0x7fcbb415ffa0 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbb2ffd640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 0x7fcba0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbba588640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 0x7fcba0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcbac08e5e0 con 0x7fcbb415ffa0 2024-05-17T21:57:33.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.853+0000 7fcbba588640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 0x7fcba0071940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcba40ae000 tx=0x7fcba40a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:33.857+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcbb406e610 con 0x7fcbb415ffa0 2024-05-17T21:57:33.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.009+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbb406dcc0 con 0x7fcbb415ffa0 2024-05-17T21:57:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.009+0000 7fcbb2ffd640 1 -- 172.21.15.83:0/1879921082 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fcbac005480 con 0x7fcbb415ffa0 2024-05-17T21:57:34.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:34.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 msgr2=0x7fcba0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 0x7fcba0071940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcba40ae000 tx=0x7fcba40a7070 comp rx=0 tx=0).stop 2024-05-17T21:57:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 msgr2=0x7fcbb40ba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c370 tx=0x7fcbac00c830 comp rx=0 tx=0).stop 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 shutdown_connections 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbb415ffa0 0x7fcbb40ba2d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcba006f430 0x7fcba0071940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbb40d1150 0x7fcbb4158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 --2- 172.21.15.83:0/1879921082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbb406e200 0x7fcbb40c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 >> 172.21.15.83:0/1879921082 conn(0x7fcbb40d28f0 msgr2=0x7fcbb40b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 shutdown_connections 2024-05-17T21:57:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.013+0000 7fcbbc7ec640 1 -- 172.21.15.83:0/1879921082 wait complete. 2024-05-17T21:57:34.021 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T21:57:34.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.161+0000 7f5adfcad640 1 Processor -- start 2024-05-17T21:57:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.161+0000 7f5adfcad640 1 -- start start 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adfcad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad81461c0 0x7f5ad815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adfcad640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ad815e6b0 0x7f5ad806ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adfcad640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad81465d0 con 0x7f5ad8070510 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adfcad640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad806ef30 con 0x7f5ad815e6b0 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adfcad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad806f210 con 0x7f5ad81461c0 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5add248640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ad815e6b0 0x7f5ad806ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5ade24a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5ad8070510 0x7f5ad81605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34950/0 (socket says 172.21.15.83:34950) 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5ade24a640 1 -- 172.21.15.83:0/307005345 learned_addr learned my addr 172.21.15.83:0/307005345 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adda49640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad81461c0 0x7f5ad815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5add248640 1 -- 172.21.15.83:0/307005345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad81461c0 msgr2=0x7f5ad815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5add248640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad81461c0 0x7f5ad815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5add248640 1 -- 172.21.15.83:0/307005345 >> v1:172.21.15.83:6789/0 conn(0x7f5ad8070510 legacy=0x7f5ad81605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5add248640 1 -- 172.21.15.83:0/307005345 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad8070920 con 0x7f5ad815e6b0 2024-05-17T21:57:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.165+0000 7f5adca47640 1 -- 172.21.15.83:0/307005345 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197121924 0 0) 0x7f5ad81465d0 con 0x7f5ad8070510 2024-05-17T21:57:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5add248640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad815e6b0 0x7f5ad806ffd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1d88103b2c4570 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adca47640 1 -- 172.21.15.83:0/307005345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40a7070 con 0x7f5ad815e6b0 2024-05-17T21:57:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adca47640 1 -- 172.21.15.83:0/307005345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad40b0070 con 0x7f5ad815e6b0 2024-05-17T21:57:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adca47640 1 -- 172.21.15.83:0/307005345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40ab890 con 0x7f5ad815e6b0 2024-05-17T21:57:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- 172.21.15.83:0/307005345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad815e6b0 msgr2=0x7f5ad806ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad815e6b0 0x7f5ad806ffd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401c470 comp rx=0 tx=0).stop 2024-05-17T21:57:34.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- 172.21.15.83:0/307005345 shutdown_connections 2024-05-17T21:57:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad815e6b0 0x7f5ad806ffd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 --2- 172.21.15.83:0/307005345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad81461c0 0x7f5ad815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- 172.21.15.83:0/307005345 >> 172.21.15.83:0/307005345 conn(0x7f5ad806cd90 msgr2=0x7f5ad80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- 172.21.15.83:0/307005345 shutdown_connections 2024-05-17T21:57:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- 172.21.15.83:0/307005345 wait complete. 2024-05-17T21:57:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 Processor -- start 2024-05-17T21:57:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.169+0000 7f5adfcad640 1 -- start start 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8070510 0x7f5ad80af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad815e6b0 0x7f5ad80b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8160e30 con 0x7f5ad81461c0 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8160cb0 con 0x7f5ad8070510 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad80d10a0 con 0x7f5ad815e6b0 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38876/0 (socket says 172.21.15.83:38876) 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 -- 172.21.15.83:0/4013039237 learned_addr learned my addr 172.21.15.83:0/4013039237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adda49640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8070510 0x7f5ad80af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5ade24a640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad815e6b0 0x7f5ad80b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 -- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad815e6b0 msgr2=0x7f5ad80b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad815e6b0 0x7f5ad80b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 -- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8070510 msgr2=0x7f5ad80af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8070510 0x7f5ad80af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 -- 172.21.15.83:0/4013039237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad80b31c0 con 0x7f5ad81461c0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5add248640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4006e20 con 0x7f5ad81461c0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4004220 con 0x7f5ad81461c0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad80b0030 con 0x7f5ad81461c0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4003620 con 0x7f5ad81461c0 2024-05-17T21:57:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad80b2ee0 con 0x7f5ad81461c0 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad815e170 con 0x7f5ad81461c0 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.173+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ad40037c0 con 0x7f5ad81461c0 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.177+0000 7f5aca7fc640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 0x7f5abc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.177+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad412a060 con 0x7f5ad81461c0 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.177+0000 7f5adda49640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 0x7f5abc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.177+0000 7f5adda49640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 0x7f5abc071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80b11f0 tx=0x7f5acc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.177+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5ad4016090 con 0x7f5ad81461c0 2024-05-17T21:57:34.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.337+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ad80b0310 con 0x7f5abc06f400 2024-05-17T21:57:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.349+0000 7f5aca7fc640 1 -- 172.21.15.83:0/4013039237 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240030 (secure 0 0 0) 0x7f5ad80b0310 con 0x7f5abc06f400 2024-05-17T21:57:34.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 msgr2=0x7f5abc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 0x7f5abc071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80b11f0 tx=0x7f5acc008040 comp rx=0 tx=0).stop 2024-05-17T21:57:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 msgr2=0x7f5ad815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401acb0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 shutdown_connections 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ad815e6b0 0x7f5ad80b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad81461c0 0x7f5ad815c100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5abc06f400 0x7f5abc071910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 --2- 172.21.15.83:0/4013039237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8070510 0x7f5ad80af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 >> 172.21.15.83:0/4013039237 conn(0x7f5ad806cd90 msgr2=0x7f5ad8075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 shutdown_connections 2024-05-17T21:57:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.353+0000 7f5adfcad640 1 -- 172.21.15.83:0/4013039237 wait complete. 2024-05-17T21:57:34.363 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-17T21:57:32.965713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218632,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756088,"statfs":{"total":773094113280,"available":772870234112,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.724573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244080+0000","last_change":"2024-05-17T21:57:28.244080+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244080+0000","last_undegraded":"2024-05-17T21:57:28.244080+0000","last_fullsized":"2024-05-17T21:57:28.244080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252538+0000","last_change":"2024-05-17T21:57:28.252538+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252538+0000","last_undegraded":"2024-05-17T21:57:28.252538+0000","last_fullsized":"2024-05-17T21:57:28.252538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244228+0000","last_change":"2024-05-17T21:57:28.244228+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244228+0000","last_undegraded":"2024-05-17T21:57:28.244228+0000","last_fullsized":"2024-05-17T21:57:28.244228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251942+0000","last_change":"2024-05-17T21:57:28.251942+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251942+0000","last_undegraded":"2024-05-17T21:57:28.251942+0000","last_fullsized":"2024-05-17T21:57:28.251942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254328+0000","last_change":"2024-05-17T21:57:28.254328+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254328+0000","last_undegraded":"2024-05-17T21:57:28.254328+0000","last_fullsized":"2024-05-17T21:57:28.254328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251351+0000","last_change":"2024-05-17T21:57:28.251351+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251351+0000","last_undegraded":"2024-05-17T21:57:28.251351+0000","last_fullsized":"2024-05-17T21:57:28.251351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250744+0000","last_change":"2024-05-17T21:57:28.250744+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250744+0000","last_undegraded":"2024-05-17T21:57:28.250744+0000","last_fullsized":"2024-05-17T21:57:28.250744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250075+0000","last_change":"2024-05-17T21:57:28.250075+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250075+0000","last_undegraded":"2024-05-17T21:57:28.250075+0000","last_fullsized":"2024-05-17T21:57:28.250075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.246353+0000","last_change":"2024-05-17T21:57:28.246353+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.246353+0000","last_undegraded":"2024-05-17T21:57:28.246353+0000","last_fullsized":"2024-05-17T21:57:28.246353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252773+0000","last_change":"2024-05-17T21:57:28.252773+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252773+0000","last_undegraded":"2024-05-17T21:57:28.252773+0000","last_fullsized":"2024-05-17T21:57:28.252773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251002+0000","last_change":"2024-05-17T21:57:28.251002+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251002+0000","last_undegraded":"2024-05-17T21:57:28.251002+0000","last_fullsized":"2024-05-17T21:57:28.251002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254587+0000","last_change":"2024-05-17T21:57:28.254587+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254587+0000","last_undegraded":"2024-05-17T21:57:28.254587+0000","last_fullsized":"2024-05-17T21:57:28.254587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254154+0000","last_change":"2024-05-17T21:57:28.254154+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254154+0000","last_undegraded":"2024-05-17T21:57:28.254154+0000","last_fullsized":"2024-05-17T21:57:28.254154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254491+0000","last_change":"2024-05-17T21:57:28.254491+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254491+0000","last_undegraded":"2024-05-17T21:57:28.254491+0000","last_fullsized":"2024-05-17T21:57:28.254491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249824+0000","last_change":"2024-05-17T21:57:28.249824+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249824+0000","last_undegraded":"2024-05-17T21:57:28.249824+0000","last_fullsized":"2024-05-17T21:57:28.249824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254499+0000","last_change":"2024-05-17T21:57:28.254499+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254499+0000","last_undegraded":"2024-05-17T21:57:28.254499+0000","last_fullsized":"2024-05-17T21:57:28.254499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-17T21:57:28.251581+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:28.251581+0000","last_peered":"2024-05-17T21:57:28.251581+0000","last_clean":"2024-05-17T21:57:28.251581+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:28.251581+0000","last_undegraded":"2024-05-17T21:57:28.251581+0000","last_fullsized":"2024-05-17T21:57:28.251581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250489+0000","last_change":"2024-05-17T21:57:28.250489+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250489+0000","last_undegraded":"2024-05-17T21:57:28.250489+0000","last_fullsized":"2024-05-17T21:57:28.250489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.245922+0000","last_change":"2024-05-17T21:57:28.245922+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.245922+0000","last_undegraded":"2024-05-17T21:57:28.245922+0000","last_fullsized":"2024-05-17T21:57:28.245922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254557+0000","last_change":"2024-05-17T21:57:28.254557+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254557+0000","last_undegraded":"2024-05-17T21:57:28.254557+0000","last_fullsized":"2024-05-17T21:57:28.254557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249759+0000","last_change":"2024-05-17T21:57:28.249759+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249759+0000","last_undegraded":"2024-05-17T21:57:28.249759+0000","last_fullsized":"2024-05-17T21:57:28.249759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249904+0000","last_change":"2024-05-17T21:57:28.249904+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249904+0000","last_undegraded":"2024-05-17T21:57:28.249904+0000","last_fullsized":"2024-05-17T21:57:28.249904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244465+0000","last_change":"2024-05-17T21:57:28.244465+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244465+0000","last_undegraded":"2024-05-17T21:57:28.244465+0000","last_fullsized":"2024-05-17T21:57:28.244465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250594+0000","last_change":"2024-05-17T21:57:28.250594+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250594+0000","last_undegraded":"2024-05-17T21:57:28.250594+0000","last_fullsized":"2024-05-17T21:57:28.250594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252798+0000","last_change":"2024-05-17T21:57:28.252798+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252798+0000","last_undegraded":"2024-05-17T21:57:28.252798+0000","last_fullsized":"2024-05-17T21:57:28.252798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.248014+0000","last_change":"2024-05-17T21:57:28.248014+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.248014+0000","last_undegraded":"2024-05-17T21:57:28.248014+0000","last_fullsized":"2024-05-17T21:57:28.248014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249996+0000","last_change":"2024-05-17T21:57:28.249996+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249996+0000","last_undegraded":"2024-05-17T21:57:28.249996+0000","last_fullsized":"2024-05-17T21:57:28.249996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249903+0000","last_change":"2024-05-17T21:57:28.249903+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249903+0000","last_undegraded":"2024-05-17T21:57:28.249903+0000","last_fullsized":"2024-05-17T21:57:28.249903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244427+0000","last_change":"2024-05-17T21:57:28.244427+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244427+0000","last_undegraded":"2024-05-17T21:57:28.244427+0000","last_fullsized":"2024-05-17T21:57:28.244427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244300+0000","last_change":"2024-05-17T21:57:28.244300+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244300+0000","last_undegraded":"2024-05-17T21:57:28.244300+0000","last_fullsized":"2024-05-17T21:57:28.244300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.247840+0000","last_change":"2024-05-17T21:57:28.247840+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.247840+0000","last_undegraded":"2024-05-17T21:57:28.247840+0000","last_fullsized":"2024-05-17T21:57:28.247840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244079+0000","last_change":"2024-05-17T21:57:28.244079+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244079+0000","last_undegraded":"2024-05-17T21:57:28.244079+0000","last_fullsized":"2024-05-17T21:57:28.244079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.246402+0000","last_change":"2024-05-17T21:57:28.246402+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.246402+0000","last_undegraded":"2024-05-17T21:57:28.246402+0000","last_fullsized":"2024-05-17T21:57:28.246402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:34.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:34.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.565+0000 7ff4f5130640 1 Processor -- start 2024-05-17T21:57:34.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.565+0000 7ff4f5130640 1 -- start start 2024-05-17T21:57:34.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4f5130640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4f006f3c0 0x7ff4f00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4f5130640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f0071a20 con 0x7ff4f01608c0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4f5130640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f0160cd0 con 0x7ff4f0071610 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4f5130640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f006e690 con 0x7ff4f006f3c0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ef59e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff4f01608c0 0x7ff4f00b69d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34960/0 (socket says 172.21.15.83:34960) 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ef59e640 1 -- 172.21.15.83:0/1757430961 learned_addr learned my addr 172.21.15.83:0/1757430961 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4eed9d640 1 --2- 172.21.15.83:0/1757430961 >> v2:172.21.15.155:3300/0 conn(0x7ff4f006f3c0 0x7ff4f00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797409004 0 0) 0x7ff4e80014a0 con 0x7ff4f01608c0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4dc003530 con 0x7ff4f01608c0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059735845 0 0) 0x7ff4dc003530 con 0x7ff4f01608c0 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 >> v2:172.21.15.155:3300/0 conn(0x7ff4f006f3c0 msgr2=0x7ff4f00710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 --2- 172.21.15.83:0/1757430961 >> v2:172.21.15.155:3300/0 conn(0x7ff4f006f3c0 0x7ff4f00710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 >> v1:172.21.15.155:6789/0 conn(0x7ff4f0071610 legacy=0x7ff4f01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f0160f50 con 0x7ff4f01608c0 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ff4e8001c50 con 0x7ff4f01608c0 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 >> v1:172.21.15.83:6789/0 conn(0x7ff4f01608c0 legacy=0x7ff4f00b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc003a60 0x7ff4dc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 0x7ff4dc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4dc005260 0x7ff4dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f006e690 con 0x7ff4dc004810 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc00c1f0 con 0x7ff4dc005260 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4dc00c3f0 con 0x7ff4dc003a60 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 0x7ff4dc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4eed9d640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc003a60 0x7ff4dc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ef59e640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4dc005260 0x7ff4dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 -- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc003a60 msgr2=0x7ff4dc003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc003a60 0x7ff4dc003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 -- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4dc005260 msgr2=0x7ff4dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4dc005260 0x7ff4dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4ef59e640 1 -- 172.21.15.83:0/1757430961 reap_dead start 2024-05-17T21:57:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.569+0000 7ff4e659c640 1 -- 172.21.15.83:0/1757430961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc003530 con 0x7ff4dc004810 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4e659c640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 0x7ff4dc004c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8001790 tx=0x7ff4d801c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fcdbef1fa42e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80013e0 con 0x7ff4dc004810 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d80052d0 con 0x7ff4dc004810 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4ee59c640 1 -- 172.21.15.83:0/1757430961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8003840 con 0x7ff4dc004810 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 msgr2=0x7ff4dc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 0x7ff4dc004c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8001790 tx=0x7ff4d801c1f0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- 172.21.15.83:0/1757430961 shutdown_connections 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 --2- 172.21.15.83:0/1757430961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4dc004810 0x7ff4dc004c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- 172.21.15.83:0/1757430961 >> 172.21.15.83:0/1757430961 conn(0x7ff4f0068730 msgr2=0x7ff4f0068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- 172.21.15.83:0/1757430961 shutdown_connections 2024-05-17T21:57:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- 172.21.15.83:0/1757430961 wait complete. 2024-05-17T21:57:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 Processor -- start 2024-05-17T21:57:34.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.573+0000 7ff4f5130640 1 -- start start 2024-05-17T21:57:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc004810 0x7ff4f01556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 0x7ff4f00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f006f7d0 con 0x7ff4f01ea610 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f006ebc0 con 0x7ff4f00cf780 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f0156e50 con 0x7ff4dc004810 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38892/0 (socket says 172.21.15.83:38892) 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 -- 172.21.15.83:0/2695340893 learned_addr learned my addr 172.21.15.83:0/2695340893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e659c640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 0x7ff4f00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4eed9d640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc004810 0x7ff4f01556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 -- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc004810 msgr2=0x7ff4f01556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc004810 0x7ff4f01556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 -- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 msgr2=0x7ff4f00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 0x7ff4f00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 -- 172.21.15.83:0/2695340893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01eaf60 con 0x7ff4f01ea610 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e659c640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 0x7ff4f00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4ef59e640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00b6790 tx=0x7ff4e80108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e801a070 con 0x7ff4f01ea610 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e8007070 con 0x7ff4f01ea610 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f00b2e80 con 0x7ff4f01ea610 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007440 con 0x7ff4f01ea610 2024-05-17T21:57:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f00b3400 con 0x7ff4f01ea610 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f00710d0 con 0x7ff4f01ea610 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4e801d070 con 0x7ff4f01ea610 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.577+0000 7ff4e7fff640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 0x7ff4d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.581+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e8091230 con 0x7ff4f01ea610 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.581+0000 7ff4eed9d640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 0x7ff4d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.581+0000 7ff4eed9d640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 0x7ff4d0071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00099f0 tx=0x7ff4e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.585+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff4f01eaf60 con 0x7ff4f01ea610 2024-05-17T21:57:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.741+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4f00d33a0 con 0x7ff4d006f400 2024-05-17T21:57:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.745+0000 7ff4e7fff640 1 -- 172.21.15.83:0/2695340893 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240030 (secure 0 0 0) 0x7ff4f00d33a0 con 0x7ff4d006f400 2024-05-17T21:57:34.736 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 msgr2=0x7ff4d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 0x7ff4d0071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4e00099f0 tx=0x7ff4e0008040 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 msgr2=0x7ff4f01eaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00b6790 tx=0x7ff4e80108a0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 shutdown_connections 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f01ea610 0x7ff4f01eaa20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4d006f400 0x7ff4d0071910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f00cf780 0x7ff4f00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 --2- 172.21.15.83:0/2695340893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4dc004810 0x7ff4f01556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 >> 172.21.15.83:0/2695340893 conn(0x7ff4f0068730 msgr2=0x7ff4f00b67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 shutdown_connections 2024-05-17T21:57:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.749+0000 7ff4f5130640 1 -- 172.21.15.83:0/2695340893 wait complete. 2024-05-17T21:57:34.759 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-17T21:57:32.965713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218632,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756088,"statfs":{"total":773094113280,"available":772870234112,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.724573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244080+0000","last_change":"2024-05-17T21:57:28.244080+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244080+0000","last_undegraded":"2024-05-17T21:57:28.244080+0000","last_fullsized":"2024-05-17T21:57:28.244080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252538+0000","last_change":"2024-05-17T21:57:28.252538+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252538+0000","last_undegraded":"2024-05-17T21:57:28.252538+0000","last_fullsized":"2024-05-17T21:57:28.252538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244228+0000","last_change":"2024-05-17T21:57:28.244228+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244228+0000","last_undegraded":"2024-05-17T21:57:28.244228+0000","last_fullsized":"2024-05-17T21:57:28.244228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251942+0000","last_change":"2024-05-17T21:57:28.251942+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251942+0000","last_undegraded":"2024-05-17T21:57:28.251942+0000","last_fullsized":"2024-05-17T21:57:28.251942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254328+0000","last_change":"2024-05-17T21:57:28.254328+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254328+0000","last_undegraded":"2024-05-17T21:57:28.254328+0000","last_fullsized":"2024-05-17T21:57:28.254328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251351+0000","last_change":"2024-05-17T21:57:28.251351+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251351+0000","last_undegraded":"2024-05-17T21:57:28.251351+0000","last_fullsized":"2024-05-17T21:57:28.251351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250744+0000","last_change":"2024-05-17T21:57:28.250744+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250744+0000","last_undegraded":"2024-05-17T21:57:28.250744+0000","last_fullsized":"2024-05-17T21:57:28.250744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250075+0000","last_change":"2024-05-17T21:57:28.250075+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250075+0000","last_undegraded":"2024-05-17T21:57:28.250075+0000","last_fullsized":"2024-05-17T21:57:28.250075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.246353+0000","last_change":"2024-05-17T21:57:28.246353+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.246353+0000","last_undegraded":"2024-05-17T21:57:28.246353+0000","last_fullsized":"2024-05-17T21:57:28.246353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252773+0000","last_change":"2024-05-17T21:57:28.252773+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252773+0000","last_undegraded":"2024-05-17T21:57:28.252773+0000","last_fullsized":"2024-05-17T21:57:28.252773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.251002+0000","last_change":"2024-05-17T21:57:28.251002+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.251002+0000","last_undegraded":"2024-05-17T21:57:28.251002+0000","last_fullsized":"2024-05-17T21:57:28.251002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254587+0000","last_change":"2024-05-17T21:57:28.254587+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254587+0000","last_undegraded":"2024-05-17T21:57:28.254587+0000","last_fullsized":"2024-05-17T21:57:28.254587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254154+0000","last_change":"2024-05-17T21:57:28.254154+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254154+0000","last_undegraded":"2024-05-17T21:57:28.254154+0000","last_fullsized":"2024-05-17T21:57:28.254154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254491+0000","last_change":"2024-05-17T21:57:28.254491+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254491+0000","last_undegraded":"2024-05-17T21:57:28.254491+0000","last_fullsized":"2024-05-17T21:57:28.254491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249824+0000","last_change":"2024-05-17T21:57:28.249824+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249824+0000","last_undegraded":"2024-05-17T21:57:28.249824+0000","last_fullsized":"2024-05-17T21:57:28.249824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254499+0000","last_change":"2024-05-17T21:57:28.254499+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254499+0000","last_undegraded":"2024-05-17T21:57:28.254499+0000","last_fullsized":"2024-05-17T21:57:28.254499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":104,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-17T21:57:28.251581+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:28.251581+0000","last_peered":"2024-05-17T21:57:28.251581+0000","last_clean":"2024-05-17T21:57:28.251581+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:28.251581+0000","last_undegraded":"2024-05-17T21:57:28.251581+0000","last_fullsized":"2024-05-17T21:57:28.251581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250489+0000","last_change":"2024-05-17T21:57:28.250489+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250489+0000","last_undegraded":"2024-05-17T21:57:28.250489+0000","last_fullsized":"2024-05-17T21:57:28.250489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.245922+0000","last_change":"2024-05-17T21:57:28.245922+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.245922+0000","last_undegraded":"2024-05-17T21:57:28.245922+0000","last_fullsized":"2024-05-17T21:57:28.245922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.254557+0000","last_change":"2024-05-17T21:57:28.254557+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.254557+0000","last_undegraded":"2024-05-17T21:57:28.254557+0000","last_fullsized":"2024-05-17T21:57:28.254557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249759+0000","last_change":"2024-05-17T21:57:28.249759+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249759+0000","last_undegraded":"2024-05-17T21:57:28.249759+0000","last_fullsized":"2024-05-17T21:57:28.249759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249904+0000","last_change":"2024-05-17T21:57:28.249904+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249904+0000","last_undegraded":"2024-05-17T21:57:28.249904+0000","last_fullsized":"2024-05-17T21:57:28.249904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244465+0000","last_change":"2024-05-17T21:57:28.244465+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244465+0000","last_undegraded":"2024-05-17T21:57:28.244465+0000","last_fullsized":"2024-05-17T21:57:28.244465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.250594+0000","last_change":"2024-05-17T21:57:28.250594+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.250594+0000","last_undegraded":"2024-05-17T21:57:28.250594+0000","last_fullsized":"2024-05-17T21:57:28.250594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.252798+0000","last_change":"2024-05-17T21:57:28.252798+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.252798+0000","last_undegraded":"2024-05-17T21:57:28.252798+0000","last_fullsized":"2024-05-17T21:57:28.252798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.248014+0000","last_change":"2024-05-17T21:57:28.248014+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.248014+0000","last_undegraded":"2024-05-17T21:57:28.248014+0000","last_fullsized":"2024-05-17T21:57:28.248014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249996+0000","last_change":"2024-05-17T21:57:28.249996+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249996+0000","last_undegraded":"2024-05-17T21:57:28.249996+0000","last_fullsized":"2024-05-17T21:57:28.249996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.249903+0000","last_change":"2024-05-17T21:57:28.249903+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.249903+0000","last_undegraded":"2024-05-17T21:57:28.249903+0000","last_fullsized":"2024-05-17T21:57:28.249903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244427+0000","last_change":"2024-05-17T21:57:28.244427+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244427+0000","last_undegraded":"2024-05-17T21:57:28.244427+0000","last_fullsized":"2024-05-17T21:57:28.244427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244300+0000","last_change":"2024-05-17T21:57:28.244300+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244300+0000","last_undegraded":"2024-05-17T21:57:28.244300+0000","last_fullsized":"2024-05-17T21:57:28.244300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.247840+0000","last_change":"2024-05-17T21:57:28.247840+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.247840+0000","last_undegraded":"2024-05-17T21:57:28.247840+0000","last_fullsized":"2024-05-17T21:57:28.247840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.244079+0000","last_change":"2024-05-17T21:57:28.244079+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.244079+0000","last_undegraded":"2024-05-17T21:57:28.244079+0000","last_fullsized":"2024-05-17T21:57:28.244079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:57:29.240170+0000","last_change":"2024-05-17T21:57:29.240170+0000","last_active":"2024-05-17T21:57:29.240170+0000","last_peered":"2024-05-17T21:57:29.240170+0000","last_clean":"2024-05-17T21:57:29.240170+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:29.240170+0000","last_undegraded":"2024-05-17T21:57:29.240170+0000","last_fullsized":"2024-05-17T21:57:29.240170+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-17T21:57:28.246402+0000","last_change":"2024-05-17T21:57:28.246402+0000","last_active":"2024-05-17T21:57:28.228873+0000","last_peered":"2024-05-17T21:57:28.228873+0000","last_clean":"2024-05-17T21:57:28.228873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:57:28.246402+0000","last_undegraded":"2024-05-17T21:57:28.246402+0000","last_fullsized":"2024-05-17T21:57:28.246402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:34.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 Processor -- start 2024-05-17T21:57:34.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 -- start start 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4740d1040 0x7fb474144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47415ea40 con 0x7fb47415e080 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47415ed20 con 0x7fb474073540 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb47a63e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb47415f000 con 0x7fb4740d1040 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb473fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4740d1040 0x7fb474144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb4737fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb47415e080 0x7fb474072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34976/0 (socket says 172.21.15.83:34976) 2024-05-17T21:57:34.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb4737fe640 1 -- 172.21.15.83:0/1938044784 learned_addr learned my addr 172.21.15.83:0/1938044784 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb473fff640 1 -- 172.21.15.83:0/1938044784 >> v1:172.21.15.155:6789/0 conn(0x7fb474073540 legacy=0x7fb47406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb473fff640 1 -- 172.21.15.83:0/1938044784 >> v1:172.21.15.83:6789/0 conn(0x7fb47415e080 legacy=0x7fb474072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb472ffd640 1 -- 172.21.15.83:0/1938044784 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207567976 0 0) 0x7fb47415ea40 con 0x7fb47415e080 2024-05-17T21:57:34.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.961+0000 7fb473fff640 1 -- 172.21.15.83:0/1938044784 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47406ef30 con 0x7fb4740d1040 2024-05-17T21:57:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb473fff640 1 --2- 172.21.15.83:0/1938044784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4740d1040 0x7fb474144080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb464009d90 tx=0x7fb46401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6200cadf3c0c6e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb472ffd640 1 -- 172.21.15.83:0/1938044784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4640b2070 con 0x7fb4740d1040 2024-05-17T21:57:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb472ffd640 1 -- 172.21.15.83:0/1938044784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4640a5940 con 0x7fb4740d1040 2024-05-17T21:57:34.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb472ffd640 1 -- 172.21.15.83:0/1938044784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4640a5d10 con 0x7fb4740d1040 2024-05-17T21:57:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- 172.21.15.83:0/1938044784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4740d1040 msgr2=0x7fb474144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 --2- 172.21.15.83:0/1938044784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4740d1040 0x7fb474144080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb464009d90 tx=0x7fb46401c970 comp rx=0 tx=0).stop 2024-05-17T21:57:34.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- 172.21.15.83:0/1938044784 shutdown_connections 2024-05-17T21:57:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 --2- 172.21.15.83:0/1938044784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4740d1040 0x7fb474144080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- 172.21.15.83:0/1938044784 >> 172.21.15.83:0/1938044784 conn(0x7fb47406cd90 msgr2=0x7fb4740b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- 172.21.15.83:0/1938044784 shutdown_connections 2024-05-17T21:57:34.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- 172.21.15.83:0/1938044784 wait complete. 2024-05-17T21:57:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 Processor -- start 2024-05-17T21:57:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.965+0000 7fb47a63e640 1 -- start start 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4740d1040 0x7fb47413af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47415e080 0x7fb47415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4740709e0 con 0x7fb47415d6f0 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb474070860 con 0x7fb47415e080 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb47416b860 con 0x7fb4740d1040 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38916/0 (socket says 172.21.15.83:38916) 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 -- 172.21.15.83:0/764806527 learned_addr learned my addr 172.21.15.83:0/764806527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb473fff640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4740d1040 0x7fb47413af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 -- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4740d1040 msgr2=0x7fb47413af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb4737fe640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47415e080 0x7fb47415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4740d1040 0x7fb47413af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 -- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47415e080 msgr2=0x7fb47415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47415e080 0x7fb47415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 -- 172.21.15.83:0/764806527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47415de70 con 0x7fb47415d6f0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb478bdb640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00b4c0 tx=0x7fb46c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c015020 con 0x7fb47415d6f0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb46c011830 con 0x7fb47415d6f0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47413b4c0 con 0x7fb47415d6f0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c011c20 con 0x7fb47415d6f0 2024-05-17T21:57:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741f6cd0 con 0x7fb47415d6f0 2024-05-17T21:57:34.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.969+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47415dbe0 con 0x7fb47415d6f0 2024-05-17T21:57:34.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.973+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb46c018020 con 0x7fb47415d6f0 2024-05-17T21:57:34.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.973+0000 7fb470ff9640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 0x7fb454071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:34.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.973+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb46c08f4e0 con 0x7fb47415d6f0 2024-05-17T21:57:34.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.973+0000 7fb473fff640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 0x7fb454071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.973+0000 7fb473fff640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 0x7fb454071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb4640af000 tx=0x7fb4640a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:34.977+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb46c01f020 con 0x7fb47415d6f0 2024-05-17T21:57:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.121+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb474144080 con 0x7fb45406f400 2024-05-17T21:57:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.129+0000 7fb470ff9640 1 -- 172.21.15.83:0/764806527 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7fb474144080 con 0x7fb45406f400 2024-05-17T21:57:35.121 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.129+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 msgr2=0x7fb454071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.129+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 0x7fb454071910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb4640af000 tx=0x7fb4640a7070 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.129+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 msgr2=0x7fb4740b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.129+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00b4c0 tx=0x7fb46c00b980 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 shutdown_connections 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb47415d6f0 0x7fb4740b1790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb45406f400 0x7fb454071910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb47415e080 0x7fb47415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 --2- 172.21.15.83:0/764806527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4740d1040 0x7fb47413af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 >> 172.21.15.83:0/764806527 conn(0x7fb47406cd90 msgr2=0x7fb4740cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 shutdown_connections 2024-05-17T21:57:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.133+0000 7fb47a63e640 1 -- 172.21.15.83:0/764806527 wait complete. 2024-05-17T21:57:35.140 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-17T21:57:34.966304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001960"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200427+0000","last_change":"2024-05-17T21:57:30.334206+0000","last_active":"2024-05-17T21:57:32.200427+0000","last_peered":"2024-05-17T21:57:32.200427+0000","last_clean":"2024-05-17T21:57:32.200427+0000","last_became_active":"2024-05-17T21:57:30.333625+0000","last_became_peered":"2024-05-17T21:57:30.333625+0000","last_unstale":"2024-05-17T21:57:32.200427+0000","last_undegraded":"2024-05-17T21:57:32.200427+0000","last_fullsized":"2024-05-17T21:57:32.200427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268827+0000","last_change":"2024-05-17T21:57:30.282372+0000","last_active":"2024-05-17T21:57:31.268827+0000","last_peered":"2024-05-17T21:57:31.268827+0000","last_clean":"2024-05-17T21:57:31.268827+0000","last_became_active":"2024-05-17T21:57:30.282085+0000","last_became_peered":"2024-05-17T21:57:30.282085+0000","last_unstale":"2024-05-17T21:57:31.268827+0000","last_undegraded":"2024-05-17T21:57:31.268827+0000","last_fullsized":"2024-05-17T21:57:31.268827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201211+0000","last_change":"2024-05-17T21:57:30.274388+0000","last_active":"2024-05-17T21:57:32.201211+0000","last_peered":"2024-05-17T21:57:32.201211+0000","last_clean":"2024-05-17T21:57:32.201211+0000","last_became_active":"2024-05-17T21:57:30.274079+0000","last_became_peered":"2024-05-17T21:57:30.274079+0000","last_unstale":"2024-05-17T21:57:32.201211+0000","last_undegraded":"2024-05-17T21:57:32.201211+0000","last_fullsized":"2024-05-17T21:57:32.201211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:10.441970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262187+0000","last_change":"2024-05-17T21:57:30.299694+0000","last_active":"2024-05-17T21:57:31.262187+0000","last_peered":"2024-05-17T21:57:31.262187+0000","last_clean":"2024-05-17T21:57:31.262187+0000","last_became_active":"2024-05-17T21:57:30.298884+0000","last_became_peered":"2024-05-17T21:57:30.298884+0000","last_unstale":"2024-05-17T21:57:31.262187+0000","last_undegraded":"2024-05-17T21:57:31.262187+0000","last_fullsized":"2024-05-17T21:57:31.262187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199587+0000","last_change":"2024-05-17T21:57:30.316073+0000","last_active":"2024-05-17T21:57:32.199587+0000","last_peered":"2024-05-17T21:57:32.199587+0000","last_clean":"2024-05-17T21:57:32.199587+0000","last_became_active":"2024-05-17T21:57:30.315439+0000","last_became_peered":"2024-05-17T21:57:30.315439+0000","last_unstale":"2024-05-17T21:57:32.199587+0000","last_undegraded":"2024-05-17T21:57:32.199587+0000","last_fullsized":"2024-05-17T21:57:32.199587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261267+0000","last_change":"2024-05-17T21:57:30.295598+0000","last_active":"2024-05-17T21:57:31.261267+0000","last_peered":"2024-05-17T21:57:31.261267+0000","last_clean":"2024-05-17T21:57:31.261267+0000","last_became_active":"2024-05-17T21:57:30.294722+0000","last_became_peered":"2024-05-17T21:57:30.294722+0000","last_unstale":"2024-05-17T21:57:31.261267+0000","last_undegraded":"2024-05-17T21:57:31.261267+0000","last_fullsized":"2024-05-17T21:57:31.261267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199251+0000","last_change":"2024-05-17T21:57:30.324100+0000","last_active":"2024-05-17T21:57:32.199251+0000","last_peered":"2024-05-17T21:57:32.199251+0000","last_clean":"2024-05-17T21:57:32.199251+0000","last_became_active":"2024-05-17T21:57:30.320444+0000","last_became_peered":"2024-05-17T21:57:30.320444+0000","last_unstale":"2024-05-17T21:57:32.199251+0000","last_undegraded":"2024-05-17T21:57:32.199251+0000","last_fullsized":"2024-05-17T21:57:32.199251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200972+0000","last_change":"2024-05-17T21:57:30.302526+0000","last_active":"2024-05-17T21:57:32.200972+0000","last_peered":"2024-05-17T21:57:32.200972+0000","last_clean":"2024-05-17T21:57:32.200972+0000","last_became_active":"2024-05-17T21:57:30.302136+0000","last_became_peered":"2024-05-17T21:57:30.302136+0000","last_unstale":"2024-05-17T21:57:32.200972+0000","last_undegraded":"2024-05-17T21:57:32.200972+0000","last_fullsized":"2024-05-17T21:57:32.200972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200607+0000","last_change":"2024-05-17T21:57:30.332944+0000","last_active":"2024-05-17T21:57:32.200607+0000","last_peered":"2024-05-17T21:57:32.200607+0000","last_clean":"2024-05-17T21:57:32.200607+0000","last_became_active":"2024-05-17T21:57:30.329174+0000","last_became_peered":"2024-05-17T21:57:30.329174+0000","last_unstale":"2024-05-17T21:57:32.200607+0000","last_undegraded":"2024-05-17T21:57:32.200607+0000","last_fullsized":"2024-05-17T21:57:32.200607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:10.454507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272598+0000","last_change":"2024-05-17T21:57:30.314980+0000","last_active":"2024-05-17T21:57:31.272598+0000","last_peered":"2024-05-17T21:57:31.272598+0000","last_clean":"2024-05-17T21:57:31.272598+0000","last_became_active":"2024-05-17T21:57:30.314085+0000","last_became_peered":"2024-05-17T21:57:30.314085+0000","last_unstale":"2024-05-17T21:57:31.272598+0000","last_undegraded":"2024-05-17T21:57:31.272598+0000","last_fullsized":"2024-05-17T21:57:31.272598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272161+0000","last_change":"2024-05-17T21:57:30.316071+0000","last_active":"2024-05-17T21:57:31.272161+0000","last_peered":"2024-05-17T21:57:31.272161+0000","last_clean":"2024-05-17T21:57:31.272161+0000","last_became_active":"2024-05-17T21:57:30.314772+0000","last_became_peered":"2024-05-17T21:57:30.314772+0000","last_unstale":"2024-05-17T21:57:31.272161+0000","last_undegraded":"2024-05-17T21:57:31.272161+0000","last_fullsized":"2024-05-17T21:57:31.272161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260695+0000","last_change":"2024-05-17T21:57:30.319379+0000","last_active":"2024-05-17T21:57:31.260695+0000","last_peered":"2024-05-17T21:57:31.260695+0000","last_clean":"2024-05-17T21:57:31.260695+0000","last_became_active":"2024-05-17T21:57:30.318510+0000","last_became_peered":"2024-05-17T21:57:30.318510+0000","last_unstale":"2024-05-17T21:57:31.260695+0000","last_undegraded":"2024-05-17T21:57:31.260695+0000","last_fullsized":"2024-05-17T21:57:31.260695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:22.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203239+0000","last_change":"2024-05-17T21:57:30.315381+0000","last_active":"2024-05-17T21:57:32.203239+0000","last_peered":"2024-05-17T21:57:32.203239+0000","last_clean":"2024-05-17T21:57:32.203239+0000","last_became_active":"2024-05-17T21:57:30.315076+0000","last_became_peered":"2024-05-17T21:57:30.315076+0000","last_unstale":"2024-05-17T21:57:32.203239+0000","last_undegraded":"2024-05-17T21:57:32.203239+0000","last_fullsized":"2024-05-17T21:57:32.203239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200631+0000","last_change":"2024-05-17T21:57:30.274361+0000","last_active":"2024-05-17T21:57:32.200631+0000","last_peered":"2024-05-17T21:57:32.200631+0000","last_clean":"2024-05-17T21:57:32.200631+0000","last_became_active":"2024-05-17T21:57:30.273916+0000","last_became_peered":"2024-05-17T21:57:30.273916+0000","last_unstale":"2024-05-17T21:57:32.200631+0000","last_undegraded":"2024-05-17T21:57:32.200631+0000","last_fullsized":"2024-05-17T21:57:32.200631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:47.494853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268312+0000","last_change":"2024-05-17T21:57:30.283163+0000","last_active":"2024-05-17T21:57:31.268312+0000","last_peered":"2024-05-17T21:57:31.268312+0000","last_clean":"2024-05-17T21:57:31.268312+0000","last_became_active":"2024-05-17T21:57:30.282704+0000","last_became_peered":"2024-05-17T21:57:30.282704+0000","last_unstale":"2024-05-17T21:57:31.268312+0000","last_undegraded":"2024-05-17T21:57:31.268312+0000","last_fullsized":"2024-05-17T21:57:31.268312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260004+0000","last_change":"2024-05-17T21:57:30.296030+0000","last_active":"2024-05-17T21:57:31.260004+0000","last_peered":"2024-05-17T21:57:31.260004+0000","last_clean":"2024-05-17T21:57:31.260004+0000","last_became_active":"2024-05-17T21:57:30.294696+0000","last_became_peered":"2024-05-17T21:57:30.294696+0000","last_unstale":"2024-05-17T21:57:31.260004+0000","last_undegraded":"2024-05-17T21:57:31.260004+0000","last_fullsized":"2024-05-17T21:57:31.260004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263697+0000","last_change":"2024-05-17T21:57:30.323553+0000","last_active":"2024-05-17T21:57:31.263697+0000","last_peered":"2024-05-17T21:57:31.263697+0000","last_clean":"2024-05-17T21:57:31.263697+0000","last_became_active":"2024-05-17T21:57:30.323019+0000","last_became_peered":"2024-05-17T21:57:30.323019+0000","last_unstale":"2024-05-17T21:57:31.263697+0000","last_undegraded":"2024-05-17T21:57:31.263697+0000","last_fullsized":"2024-05-17T21:57:31.263697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262779+0000","last_change":"2024-05-17T21:57:30.319719+0000","last_active":"2024-05-17T21:57:31.262779+0000","last_peered":"2024-05-17T21:57:31.262779+0000","last_clean":"2024-05-17T21:57:31.262779+0000","last_became_active":"2024-05-17T21:57:30.319335+0000","last_became_peered":"2024-05-17T21:57:30.319335+0000","last_unstale":"2024-05-17T21:57:31.262779+0000","last_undegraded":"2024-05-17T21:57:31.262779+0000","last_fullsized":"2024-05-17T21:57:31.262779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200408+0000","last_change":"2024-05-17T21:57:30.321777+0000","last_active":"2024-05-17T21:57:32.200408+0000","last_peered":"2024-05-17T21:57:32.200408+0000","last_clean":"2024-05-17T21:57:32.200408+0000","last_became_active":"2024-05-17T21:57:30.321011+0000","last_became_peered":"2024-05-17T21:57:30.321011+0000","last_unstale":"2024-05-17T21:57:32.200408+0000","last_undegraded":"2024-05-17T21:57:32.200408+0000","last_fullsized":"2024-05-17T21:57:32.200408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272173+0000","last_change":"2024-05-17T21:57:30.314965+0000","last_active":"2024-05-17T21:57:31.272173+0000","last_peered":"2024-05-17T21:57:31.272173+0000","last_clean":"2024-05-17T21:57:31.272173+0000","last_became_active":"2024-05-17T21:57:30.314249+0000","last_became_peered":"2024-05-17T21:57:30.314249+0000","last_unstale":"2024-05-17T21:57:31.272173+0000","last_undegraded":"2024-05-17T21:57:31.272173+0000","last_fullsized":"2024-05-17T21:57:31.272173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201572+0000","last_change":"2024-05-17T21:57:30.318172+0000","last_active":"2024-05-17T21:57:32.201572+0000","last_peered":"2024-05-17T21:57:32.201572+0000","last_clean":"2024-05-17T21:57:32.201572+0000","last_became_active":"2024-05-17T21:57:30.317761+0000","last_became_peered":"2024-05-17T21:57:30.317761+0000","last_unstale":"2024-05-17T21:57:32.201572+0000","last_undegraded":"2024-05-17T21:57:32.201572+0000","last_fullsized":"2024-05-17T21:57:32.201572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262717+0000","last_change":"2024-05-17T21:57:30.320885+0000","last_active":"2024-05-17T21:57:31.262717+0000","last_peered":"2024-05-17T21:57:31.262717+0000","last_clean":"2024-05-17T21:57:31.262717+0000","last_became_active":"2024-05-17T21:57:30.320268+0000","last_became_peered":"2024-05-17T21:57:30.320268+0000","last_unstale":"2024-05-17T21:57:31.262717+0000","last_undegraded":"2024-05-17T21:57:31.262717+0000","last_fullsized":"2024-05-17T21:57:31.262717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201655+0000","last_change":"2024-05-17T21:57:30.306877+0000","last_active":"2024-05-17T21:57:32.201655+0000","last_peered":"2024-05-17T21:57:32.201655+0000","last_clean":"2024-05-17T21:57:32.201655+0000","last_became_active":"2024-05-17T21:57:30.306541+0000","last_became_peered":"2024-05-17T21:57:30.306541+0000","last_unstale":"2024-05-17T21:57:32.201655+0000","last_undegraded":"2024-05-17T21:57:32.201655+0000","last_fullsized":"2024-05-17T21:57:32.201655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260191+0000","last_change":"2024-05-17T21:57:30.285039+0000","last_active":"2024-05-17T21:57:31.260191+0000","last_peered":"2024-05-17T21:57:31.260191+0000","last_clean":"2024-05-17T21:57:31.260191+0000","last_became_active":"2024-05-17T21:57:30.284132+0000","last_became_peered":"2024-05-17T21:57:30.284132+0000","last_unstale":"2024-05-17T21:57:31.260191+0000","last_undegraded":"2024-05-17T21:57:31.260191+0000","last_fullsized":"2024-05-17T21:57:31.260191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202612+0000","last_change":"2024-05-17T21:57:30.285554+0000","last_active":"2024-05-17T21:57:32.202612+0000","last_peered":"2024-05-17T21:57:32.202612+0000","last_clean":"2024-05-17T21:57:32.202612+0000","last_became_active":"2024-05-17T21:57:30.285106+0000","last_became_peered":"2024-05-17T21:57:30.285106+0000","last_unstale":"2024-05-17T21:57:32.202612+0000","last_undegraded":"2024-05-17T21:57:32.202612+0000","last_fullsized":"2024-05-17T21:57:32.202612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272595+0000","last_change":"2024-05-17T21:57:30.318707+0000","last_active":"2024-05-17T21:57:31.272595+0000","last_peered":"2024-05-17T21:57:31.272595+0000","last_clean":"2024-05-17T21:57:31.272595+0000","last_became_active":"2024-05-17T21:57:30.317977+0000","last_became_peered":"2024-05-17T21:57:30.317977+0000","last_unstale":"2024-05-17T21:57:31.272595+0000","last_undegraded":"2024-05-17T21:57:31.272595+0000","last_fullsized":"2024-05-17T21:57:31.272595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.269288+0000","last_change":"2024-05-17T21:57:30.279948+0000","last_active":"2024-05-17T21:57:31.269288+0000","last_peered":"2024-05-17T21:57:31.269288+0000","last_clean":"2024-05-17T21:57:31.269288+0000","last_became_active":"2024-05-17T21:57:30.279346+0000","last_became_peered":"2024-05-17T21:57:30.279346+0000","last_unstale":"2024-05-17T21:57:31.269288+0000","last_undegraded":"2024-05-17T21:57:31.269288+0000","last_fullsized":"2024-05-17T21:57:31.269288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203266+0000","last_change":"2024-05-17T21:57:30.289588+0000","last_active":"2024-05-17T21:57:32.203266+0000","last_peered":"2024-05-17T21:57:32.203266+0000","last_clean":"2024-05-17T21:57:32.203266+0000","last_became_active":"2024-05-17T21:57:30.289189+0000","last_became_peered":"2024-05-17T21:57:30.289189+0000","last_unstale":"2024-05-17T21:57:32.203266+0000","last_undegraded":"2024-05-17T21:57:32.203266+0000","last_fullsized":"2024-05-17T21:57:32.203266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202018+0000","last_change":"2024-05-17T21:57:30.282815+0000","last_active":"2024-05-17T21:57:32.202018+0000","last_peered":"2024-05-17T21:57:32.202018+0000","last_clean":"2024-05-17T21:57:32.202018+0000","last_became_active":"2024-05-17T21:57:30.282056+0000","last_became_peered":"2024-05-17T21:57:30.282056+0000","last_unstale":"2024-05-17T21:57:32.202018+0000","last_undegraded":"2024-05-17T21:57:32.202018+0000","last_fullsized":"2024-05-17T21:57:32.202018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261817+0000","last_change":"2024-05-17T21:57:30.281437+0000","last_active":"2024-05-17T21:57:31.261817+0000","last_peered":"2024-05-17T21:57:31.261817+0000","last_clean":"2024-05-17T21:57:31.261817+0000","last_became_active":"2024-05-17T21:57:30.280235+0000","last_became_peered":"2024-05-17T21:57:30.280235+0000","last_unstale":"2024-05-17T21:57:31.261817+0000","last_undegraded":"2024-05-17T21:57:31.261817+0000","last_fullsized":"2024-05-17T21:57:31.261817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262790+0000","last_change":"2024-05-17T21:57:30.274855+0000","last_active":"2024-05-17T21:57:31.262790+0000","last_peered":"2024-05-17T21:57:31.262790+0000","last_clean":"2024-05-17T21:57:31.262790+0000","last_became_active":"2024-05-17T21:57:30.274300+0000","last_became_peered":"2024-05-17T21:57:30.274300+0000","last_unstale":"2024-05-17T21:57:31.262790+0000","last_undegraded":"2024-05-17T21:57:31.262790+0000","last_fullsized":"2024-05-17T21:57:31.262790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:33.958058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271627+0000","last_change":"2024-05-17T21:57:30.305394+0000","last_active":"2024-05-17T21:57:31.271627+0000","last_peered":"2024-05-17T21:57:31.271627+0000","last_clean":"2024-05-17T21:57:31.271627+0000","last_became_active":"2024-05-17T21:57:30.304545+0000","last_became_peered":"2024-05-17T21:57:30.304545+0000","last_unstale":"2024-05-17T21:57:31.271627+0000","last_undegraded":"2024-05-17T21:57:31.271627+0000","last_fullsized":"2024-05-17T21:57:31.271627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260730+0000","last_change":"2024-05-17T21:57:30.271395+0000","last_active":"2024-05-17T21:57:31.260730+0000","last_peered":"2024-05-17T21:57:31.260730+0000","last_clean":"2024-05-17T21:57:31.260730+0000","last_became_active":"2024-05-17T21:57:30.270232+0000","last_became_peered":"2024-05-17T21:57:30.270232+0000","last_unstale":"2024-05-17T21:57:31.260730+0000","last_undegraded":"2024-05-17T21:57:31.260730+0000","last_fullsized":"2024-05-17T21:57:31.260730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:33.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203332+0000","last_change":"2024-05-17T21:57:29.306016+0000","last_active":"2024-05-17T21:57:32.203332+0000","last_peered":"2024-05-17T21:57:32.203332+0000","last_clean":"2024-05-17T21:57:32.203332+0000","last_became_active":"2024-05-17T21:57:29.305576+0000","last_became_peered":"2024-05-17T21:57:29.305576+0000","last_unstale":"2024-05-17T21:57:32.203332+0000","last_undegraded":"2024-05-17T21:57:32.203332+0000","last_fullsized":"2024-05-17T21:57:32.203332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202752+0000","last_change":"2024-05-17T21:57:30.283060+0000","last_active":"2024-05-17T21:57:32.202752+0000","last_peered":"2024-05-17T21:57:32.202752+0000","last_clean":"2024-05-17T21:57:32.202752+0000","last_became_active":"2024-05-17T21:57:30.282526+0000","last_became_peered":"2024-05-17T21:57:30.282526+0000","last_unstale":"2024-05-17T21:57:32.202752+0000","last_undegraded":"2024-05-17T21:57:32.202752+0000","last_fullsized":"2024-05-17T21:57:32.202752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271871+0000","last_change":"2024-05-17T21:57:29.299174+0000","last_active":"2024-05-17T21:57:31.271871+0000","last_peered":"2024-05-17T21:57:31.271871+0000","last_clean":"2024-05-17T21:57:31.271871+0000","last_became_active":"2024-05-17T21:57:29.297800+0000","last_became_peered":"2024-05-17T21:57:29.297800+0000","last_unstale":"2024-05-17T21:57:31.271871+0000","last_undegraded":"2024-05-17T21:57:31.271871+0000","last_fullsized":"2024-05-17T21:57:31.271871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202014+0000","last_change":"2024-05-17T21:57:29.290665+0000","last_active":"2024-05-17T21:57:32.202014+0000","last_peered":"2024-05-17T21:57:32.202014+0000","last_clean":"2024-05-17T21:57:32.202014+0000","last_became_active":"2024-05-17T21:57:29.290298+0000","last_became_peered":"2024-05-17T21:57:29.290298+0000","last_unstale":"2024-05-17T21:57:32.202014+0000","last_undegraded":"2024-05-17T21:57:32.202014+0000","last_fullsized":"2024-05-17T21:57:32.202014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201127+0000","last_change":"2024-05-17T21:57:30.283851+0000","last_active":"2024-05-17T21:57:32.201127+0000","last_peered":"2024-05-17T21:57:32.201127+0000","last_clean":"2024-05-17T21:57:32.201127+0000","last_became_active":"2024-05-17T21:57:30.283547+0000","last_became_peered":"2024-05-17T21:57:30.283547+0000","last_unstale":"2024-05-17T21:57:32.201127+0000","last_undegraded":"2024-05-17T21:57:32.201127+0000","last_fullsized":"2024-05-17T21:57:32.201127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260166+0000","last_change":"2024-05-17T21:57:29.297693+0000","last_active":"2024-05-17T21:57:31.260166+0000","last_peered":"2024-05-17T21:57:31.260166+0000","last_clean":"2024-05-17T21:57:31.260166+0000","last_became_active":"2024-05-17T21:57:29.296843+0000","last_became_peered":"2024-05-17T21:57:29.296843+0000","last_unstale":"2024-05-17T21:57:31.260166+0000","last_undegraded":"2024-05-17T21:57:31.260166+0000","last_fullsized":"2024-05-17T21:57:31.260166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268162+0000","last_change":"2024-05-17T21:57:30.278227+0000","last_active":"2024-05-17T21:57:31.268162+0000","last_peered":"2024-05-17T21:57:31.268162+0000","last_clean":"2024-05-17T21:57:31.268162+0000","last_became_active":"2024-05-17T21:57:30.277784+0000","last_became_peered":"2024-05-17T21:57:30.277784+0000","last_unstale":"2024-05-17T21:57:31.268162+0000","last_undegraded":"2024-05-17T21:57:31.268162+0000","last_fullsized":"2024-05-17T21:57:31.268162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271960+0000","last_change":"2024-05-17T21:57:29.290141+0000","last_active":"2024-05-17T21:57:31.271960+0000","last_peered":"2024-05-17T21:57:31.271960+0000","last_clean":"2024-05-17T21:57:31.271960+0000","last_became_active":"2024-05-17T21:57:29.288975+0000","last_became_peered":"2024-05-17T21:57:29.288975+0000","last_unstale":"2024-05-17T21:57:31.271960+0000","last_undegraded":"2024-05-17T21:57:31.271960+0000","last_fullsized":"2024-05-17T21:57:31.271960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262047+0000","last_change":"2024-05-17T21:57:30.322244+0000","last_active":"2024-05-17T21:57:31.262047+0000","last_peered":"2024-05-17T21:57:31.262047+0000","last_clean":"2024-05-17T21:57:31.262047+0000","last_became_active":"2024-05-17T21:57:30.321701+0000","last_became_peered":"2024-05-17T21:57:30.321701+0000","last_unstale":"2024-05-17T21:57:31.262047+0000","last_undegraded":"2024-05-17T21:57:31.262047+0000","last_fullsized":"2024-05-17T21:57:31.262047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262204+0000","last_change":"2024-05-17T21:57:29.293457+0000","last_active":"2024-05-17T21:57:31.262204+0000","last_peered":"2024-05-17T21:57:31.262204+0000","last_clean":"2024-05-17T21:57:31.262204+0000","last_became_active":"2024-05-17T21:57:29.292418+0000","last_became_peered":"2024-05-17T21:57:29.292418+0000","last_unstale":"2024-05-17T21:57:31.262204+0000","last_undegraded":"2024-05-17T21:57:31.262204+0000","last_fullsized":"2024-05-17T21:57:31.262204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202960+0000","last_change":"2024-05-17T21:57:30.277503+0000","last_active":"2024-05-17T21:57:32.202960+0000","last_peered":"2024-05-17T21:57:32.202960+0000","last_clean":"2024-05-17T21:57:32.202960+0000","last_became_active":"2024-05-17T21:57:30.276765+0000","last_became_peered":"2024-05-17T21:57:30.276765+0000","last_unstale":"2024-05-17T21:57:32.202960+0000","last_undegraded":"2024-05-17T21:57:32.202960+0000","last_fullsized":"2024-05-17T21:57:32.202960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268442+0000","last_change":"2024-05-17T21:57:29.284139+0000","last_active":"2024-05-17T21:57:31.268442+0000","last_peered":"2024-05-17T21:57:31.268442+0000","last_clean":"2024-05-17T21:57:31.268442+0000","last_became_active":"2024-05-17T21:57:29.283786+0000","last_became_peered":"2024-05-17T21:57:29.283786+0000","last_unstale":"2024-05-17T21:57:31.268442+0000","last_undegraded":"2024-05-17T21:57:31.268442+0000","last_fullsized":"2024-05-17T21:57:31.268442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260839+0000","last_change":"2024-05-17T21:57:30.322236+0000","last_active":"2024-05-17T21:57:31.260839+0000","last_peered":"2024-05-17T21:57:31.260839+0000","last_clean":"2024-05-17T21:57:31.260839+0000","last_became_active":"2024-05-17T21:57:30.321651+0000","last_became_peered":"2024-05-17T21:57:30.321651+0000","last_unstale":"2024-05-17T21:57:31.260839+0000","last_undegraded":"2024-05-17T21:57:31.260839+0000","last_fullsized":"2024-05-17T21:57:31.260839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:11.129074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271878+0000","last_change":"2024-05-17T21:57:29.286191+0000","last_active":"2024-05-17T21:57:31.271878+0000","last_peered":"2024-05-17T21:57:31.271878+0000","last_clean":"2024-05-17T21:57:31.271878+0000","last_became_active":"2024-05-17T21:57:29.285202+0000","last_became_peered":"2024-05-17T21:57:29.285202+0000","last_unstale":"2024-05-17T21:57:31.271878+0000","last_undegraded":"2024-05-17T21:57:31.271878+0000","last_fullsized":"2024-05-17T21:57:31.271878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199606+0000","last_change":"2024-05-17T21:57:30.323488+0000","last_active":"2024-05-17T21:57:32.199606+0000","last_peered":"2024-05-17T21:57:32.199606+0000","last_clean":"2024-05-17T21:57:32.199606+0000","last_became_active":"2024-05-17T21:57:30.322790+0000","last_became_peered":"2024-05-17T21:57:30.322790+0000","last_unstale":"2024-05-17T21:57:32.199606+0000","last_undegraded":"2024-05-17T21:57:32.199606+0000","last_fullsized":"2024-05-17T21:57:32.199606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202865+0000","last_change":"2024-05-17T21:57:29.298078+0000","last_active":"2024-05-17T21:57:32.202865+0000","last_peered":"2024-05-17T21:57:32.202865+0000","last_clean":"2024-05-17T21:57:32.202865+0000","last_became_active":"2024-05-17T21:57:29.297807+0000","last_became_peered":"2024-05-17T21:57:29.297807+0000","last_unstale":"2024-05-17T21:57:32.202865+0000","last_undegraded":"2024-05-17T21:57:32.202865+0000","last_fullsized":"2024-05-17T21:57:32.202865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:24:33.244279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201096+0000","last_change":"2024-05-17T21:57:30.318056+0000","last_active":"2024-05-17T21:57:32.201096+0000","last_peered":"2024-05-17T21:57:32.201096+0000","last_clean":"2024-05-17T21:57:32.201096+0000","last_became_active":"2024-05-17T21:57:30.317771+0000","last_became_peered":"2024-05-17T21:57:30.317771+0000","last_unstale":"2024-05-17T21:57:32.201096+0000","last_undegraded":"2024-05-17T21:57:32.201096+0000","last_fullsized":"2024-05-17T21:57:32.201096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:42.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261966+0000","last_change":"2024-05-17T21:57:29.284218+0000","last_active":"2024-05-17T21:57:31.261966+0000","last_peered":"2024-05-17T21:57:31.261966+0000","last_clean":"2024-05-17T21:57:31.261966+0000","last_became_active":"2024-05-17T21:57:29.283386+0000","last_became_peered":"2024-05-17T21:57:29.283386+0000","last_unstale":"2024-05-17T21:57:31.261966+0000","last_undegraded":"2024-05-17T21:57:31.261966+0000","last_fullsized":"2024-05-17T21:57:31.261966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200000+0000","last_change":"2024-05-17T21:57:30.321543+0000","last_active":"2024-05-17T21:57:32.200000+0000","last_peered":"2024-05-17T21:57:32.200000+0000","last_clean":"2024-05-17T21:57:32.200000+0000","last_became_active":"2024-05-17T21:57:30.320566+0000","last_became_peered":"2024-05-17T21:57:30.320566+0000","last_unstale":"2024-05-17T21:57:32.200000+0000","last_undegraded":"2024-05-17T21:57:32.200000+0000","last_fullsized":"2024-05-17T21:57:32.200000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271412+0000","last_change":"2024-05-17T21:57:29.297525+0000","last_active":"2024-05-17T21:57:31.271412+0000","last_peered":"2024-05-17T21:57:31.271412+0000","last_clean":"2024-05-17T21:57:31.271412+0000","last_became_active":"2024-05-17T21:57:29.296129+0000","last_became_peered":"2024-05-17T21:57:29.296129+0000","last_unstale":"2024-05-17T21:57:31.271412+0000","last_undegraded":"2024-05-17T21:57:31.271412+0000","last_fullsized":"2024-05-17T21:57:31.271412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271732+0000","last_change":"2024-05-17T21:57:30.306121+0000","last_active":"2024-05-17T21:57:31.271732+0000","last_peered":"2024-05-17T21:57:31.271732+0000","last_clean":"2024-05-17T21:57:31.271732+0000","last_became_active":"2024-05-17T21:57:30.304496+0000","last_became_peered":"2024-05-17T21:57:30.304496+0000","last_unstale":"2024-05-17T21:57:31.271732+0000","last_undegraded":"2024-05-17T21:57:31.271732+0000","last_fullsized":"2024-05-17T21:57:31.271732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262030+0000","last_change":"2024-05-17T21:57:29.273347+0000","last_active":"2024-05-17T21:57:31.262030+0000","last_peered":"2024-05-17T21:57:31.262030+0000","last_clean":"2024-05-17T21:57:31.262030+0000","last_became_active":"2024-05-17T21:57:29.272913+0000","last_became_peered":"2024-05-17T21:57:29.272913+0000","last_unstale":"2024-05-17T21:57:31.262030+0000","last_undegraded":"2024-05-17T21:57:31.262030+0000","last_fullsized":"2024-05-17T21:57:31.262030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:34:49.617602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199328+0000","last_change":"2024-05-17T21:57:30.322675+0000","last_active":"2024-05-17T21:57:32.199328+0000","last_peered":"2024-05-17T21:57:32.199328+0000","last_clean":"2024-05-17T21:57:32.199328+0000","last_became_active":"2024-05-17T21:57:30.321924+0000","last_became_peered":"2024-05-17T21:57:30.321924+0000","last_unstale":"2024-05-17T21:57:32.199328+0000","last_undegraded":"2024-05-17T21:57:32.199328+0000","last_fullsized":"2024-05-17T21:57:32.199328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262737+0000","last_change":"2024-05-17T21:57:29.294110+0000","last_active":"2024-05-17T21:57:31.262737+0000","last_peered":"2024-05-17T21:57:31.262737+0000","last_clean":"2024-05-17T21:57:31.262737+0000","last_became_active":"2024-05-17T21:57:29.293170+0000","last_became_peered":"2024-05-17T21:57:29.293170+0000","last_unstale":"2024-05-17T21:57:31.262737+0000","last_undegraded":"2024-05-17T21:57:31.262737+0000","last_fullsized":"2024-05-17T21:57:31.262737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:28:34.887309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200251+0000","last_change":"2024-05-17T21:57:30.315778+0000","last_active":"2024-05-17T21:57:32.200251+0000","last_peered":"2024-05-17T21:57:32.200251+0000","last_clean":"2024-05-17T21:57:32.200251+0000","last_became_active":"2024-05-17T21:57:30.314758+0000","last_became_peered":"2024-05-17T21:57:30.314758+0000","last_unstale":"2024-05-17T21:57:32.200251+0000","last_undegraded":"2024-05-17T21:57:32.200251+0000","last_fullsized":"2024-05-17T21:57:32.200251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263385+0000","last_change":"2024-05-17T21:57:29.281454+0000","last_active":"2024-05-17T21:57:31.263385+0000","last_peered":"2024-05-17T21:57:31.263385+0000","last_clean":"2024-05-17T21:57:31.263385+0000","last_became_active":"2024-05-17T21:57:29.280983+0000","last_became_peered":"2024-05-17T21:57:29.280983+0000","last_unstale":"2024-05-17T21:57:31.263385+0000","last_undegraded":"2024-05-17T21:57:31.263385+0000","last_fullsized":"2024-05-17T21:57:31.263385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200703+0000","last_change":"2024-05-17T21:57:30.334810+0000","last_active":"2024-05-17T21:57:32.200703+0000","last_peered":"2024-05-17T21:57:32.200703+0000","last_clean":"2024-05-17T21:57:32.200703+0000","last_became_active":"2024-05-17T21:57:30.332951+0000","last_became_peered":"2024-05-17T21:57:30.332951+0000","last_unstale":"2024-05-17T21:57:32.200703+0000","last_undegraded":"2024-05-17T21:57:32.200703+0000","last_fullsized":"2024-05-17T21:57:32.200703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:52.659720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268772+0000","last_change":"2024-05-17T21:57:29.296523+0000","last_active":"2024-05-17T21:57:31.268772+0000","last_peered":"2024-05-17T21:57:31.268772+0000","last_clean":"2024-05-17T21:57:31.268772+0000","last_became_active":"2024-05-17T21:57:29.296297+0000","last_became_peered":"2024-05-17T21:57:29.296297+0000","last_unstale":"2024-05-17T21:57:31.268772+0000","last_undegraded":"2024-05-17T21:57:31.268772+0000","last_fullsized":"2024-05-17T21:57:31.268772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263237+0000","last_change":"2024-05-17T21:57:30.297086+0000","last_active":"2024-05-17T21:57:31.263237+0000","last_peered":"2024-05-17T21:57:31.263237+0000","last_clean":"2024-05-17T21:57:31.263237+0000","last_became_active":"2024-05-17T21:57:30.296276+0000","last_became_peered":"2024-05-17T21:57:30.296276+0000","last_unstale":"2024-05-17T21:57:31.263237+0000","last_undegraded":"2024-05-17T21:57:31.263237+0000","last_fullsized":"2024-05-17T21:57:31.263237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262701+0000","last_change":"2024-05-17T21:57:29.282987+0000","last_active":"2024-05-17T21:57:31.262701+0000","last_peered":"2024-05-17T21:57:31.262701+0000","last_clean":"2024-05-17T21:57:31.262701+0000","last_became_active":"2024-05-17T21:57:29.282594+0000","last_became_peered":"2024-05-17T21:57:29.282594+0000","last_unstale":"2024-05-17T21:57:31.262701+0000","last_undegraded":"2024-05-17T21:57:31.262701+0000","last_fullsized":"2024-05-17T21:57:31.262701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200040+0000","last_change":"2024-05-17T21:57:30.328305+0000","last_active":"2024-05-17T21:57:32.200040+0000","last_peered":"2024-05-17T21:57:32.200040+0000","last_clean":"2024-05-17T21:57:32.200040+0000","last_became_active":"2024-05-17T21:57:30.324651+0000","last_became_peered":"2024-05-17T21:57:30.324651+0000","last_unstale":"2024-05-17T21:57:32.200040+0000","last_undegraded":"2024-05-17T21:57:32.200040+0000","last_fullsized":"2024-05-17T21:57:32.200040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199220+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:32.199220+0000","last_peered":"2024-05-17T21:57:32.199220+0000","last_clean":"2024-05-17T21:57:32.199220+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:32.199220+0000","last_undegraded":"2024-05-17T21:57:32.199220+0000","last_fullsized":"2024-05-17T21:57:32.199220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261146+0000","last_change":"2024-05-17T21:57:29.277871+0000","last_active":"2024-05-17T21:57:31.261146+0000","last_peered":"2024-05-17T21:57:31.261146+0000","last_clean":"2024-05-17T21:57:31.261146+0000","last_became_active":"2024-05-17T21:57:29.276986+0000","last_became_peered":"2024-05-17T21:57:29.276986+0000","last_unstale":"2024-05-17T21:57:31.261146+0000","last_undegraded":"2024-05-17T21:57:31.261146+0000","last_fullsized":"2024-05-17T21:57:31.261146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201481+0000","last_change":"2024-05-17T21:57:30.281160+0000","last_active":"2024-05-17T21:57:32.201481+0000","last_peered":"2024-05-17T21:57:32.201481+0000","last_clean":"2024-05-17T21:57:32.201481+0000","last_became_active":"2024-05-17T21:57:30.280004+0000","last_became_peered":"2024-05-17T21:57:30.280004+0000","last_unstale":"2024-05-17T21:57:32.201481+0000","last_undegraded":"2024-05-17T21:57:32.201481+0000","last_fullsized":"2024-05-17T21:57:32.201481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198887+0000","last_change":"2024-05-17T21:57:29.308082+0000","last_active":"2024-05-17T21:57:32.198887+0000","last_peered":"2024-05-17T21:57:32.198887+0000","last_clean":"2024-05-17T21:57:32.198887+0000","last_became_active":"2024-05-17T21:57:29.306695+0000","last_became_peered":"2024-05-17T21:57:29.306695+0000","last_unstale":"2024-05-17T21:57:32.198887+0000","last_undegraded":"2024-05-17T21:57:32.198887+0000","last_fullsized":"2024-05-17T21:57:32.198887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200656+0000","last_change":"2024-05-17T21:57:30.281759+0000","last_active":"2024-05-17T21:57:32.200656+0000","last_peered":"2024-05-17T21:57:32.200656+0000","last_clean":"2024-05-17T21:57:32.200656+0000","last_became_active":"2024-05-17T21:57:30.281449+0000","last_became_peered":"2024-05-17T21:57:30.281449+0000","last_unstale":"2024-05-17T21:57:32.200656+0000","last_undegraded":"2024-05-17T21:57:32.200656+0000","last_fullsized":"2024-05-17T21:57:32.200656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:53:58.249441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262812+0000","last_change":"2024-05-17T21:57:29.280664+0000","last_active":"2024-05-17T21:57:31.262812+0000","last_peered":"2024-05-17T21:57:31.262812+0000","last_clean":"2024-05-17T21:57:31.262812+0000","last_became_active":"2024-05-17T21:57:29.280118+0000","last_became_peered":"2024-05-17T21:57:29.280118+0000","last_unstale":"2024-05-17T21:57:31.262812+0000","last_undegraded":"2024-05-17T21:57:31.262812+0000","last_fullsized":"2024-05-17T21:57:31.262812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268285+0000","last_change":"2024-05-17T21:57:30.287528+0000","last_active":"2024-05-17T21:57:31.268285+0000","last_peered":"2024-05-17T21:57:31.268285+0000","last_clean":"2024-05-17T21:57:31.268285+0000","last_became_active":"2024-05-17T21:57:30.285405+0000","last_became_peered":"2024-05-17T21:57:30.285405+0000","last_unstale":"2024-05-17T21:57:31.268285+0000","last_undegraded":"2024-05-17T21:57:31.268285+0000","last_fullsized":"2024-05-17T21:57:31.268285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262287+0000","last_change":"2024-05-17T21:57:29.281356+0000","last_active":"2024-05-17T21:57:31.262287+0000","last_peered":"2024-05-17T21:57:31.262287+0000","last_clean":"2024-05-17T21:57:31.262287+0000","last_became_active":"2024-05-17T21:57:29.279906+0000","last_became_peered":"2024-05-17T21:57:29.279906+0000","last_unstale":"2024-05-17T21:57:31.262287+0000","last_undegraded":"2024-05-17T21:57:31.262287+0000","last_fullsized":"2024-05-17T21:57:31.262287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261918+0000","last_change":"2024-05-17T21:57:30.277518+0000","last_active":"2024-05-17T21:57:31.261918+0000","last_peered":"2024-05-17T21:57:31.261918+0000","last_clean":"2024-05-17T21:57:31.261918+0000","last_became_active":"2024-05-17T21:57:30.275892+0000","last_became_peered":"2024-05-17T21:57:30.275892+0000","last_unstale":"2024-05-17T21:57:31.261918+0000","last_undegraded":"2024-05-17T21:57:31.261918+0000","last_fullsized":"2024-05-17T21:57:31.261918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268304+0000","last_change":"2024-05-17T21:57:29.270571+0000","last_active":"2024-05-17T21:57:31.268304+0000","last_peered":"2024-05-17T21:57:31.268304+0000","last_clean":"2024-05-17T21:57:31.268304+0000","last_became_active":"2024-05-17T21:57:29.270219+0000","last_became_peered":"2024-05-17T21:57:29.270219+0000","last_unstale":"2024-05-17T21:57:31.268304+0000","last_undegraded":"2024-05-17T21:57:31.268304+0000","last_fullsized":"2024-05-17T21:57:31.268304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262482+0000","last_change":"2024-05-17T21:57:30.281497+0000","last_active":"2024-05-17T21:57:31.262482+0000","last_peered":"2024-05-17T21:57:31.262482+0000","last_clean":"2024-05-17T21:57:31.262482+0000","last_became_active":"2024-05-17T21:57:30.280473+0000","last_became_peered":"2024-05-17T21:57:30.280473+0000","last_unstale":"2024-05-17T21:57:31.262482+0000","last_undegraded":"2024-05-17T21:57:31.262482+0000","last_fullsized":"2024-05-17T21:57:31.262482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200226+0000","last_change":"2024-05-17T21:57:29.310998+0000","last_active":"2024-05-17T21:57:32.200226+0000","last_peered":"2024-05-17T21:57:32.200226+0000","last_clean":"2024-05-17T21:57:32.200226+0000","last_became_active":"2024-05-17T21:57:29.310445+0000","last_became_peered":"2024-05-17T21:57:29.310445+0000","last_unstale":"2024-05-17T21:57:32.200226+0000","last_undegraded":"2024-05-17T21:57:32.200226+0000","last_fullsized":"2024-05-17T21:57:32.200226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.270156+0000","last_change":"2024-05-17T21:57:30.286938+0000","last_active":"2024-05-17T21:57:31.270156+0000","last_peered":"2024-05-17T21:57:31.270156+0000","last_clean":"2024-05-17T21:57:31.270156+0000","last_became_active":"2024-05-17T21:57:30.286141+0000","last_became_peered":"2024-05-17T21:57:30.286141+0000","last_unstale":"2024-05-17T21:57:31.270156+0000","last_undegraded":"2024-05-17T21:57:31.270156+0000","last_fullsized":"2024-05-17T21:57:31.270156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268967+0000","last_change":"2024-05-17T21:57:29.291111+0000","last_active":"2024-05-17T21:57:31.268967+0000","last_peered":"2024-05-17T21:57:31.268967+0000","last_clean":"2024-05-17T21:57:31.268967+0000","last_became_active":"2024-05-17T21:57:29.290818+0000","last_became_peered":"2024-05-17T21:57:29.290818+0000","last_unstale":"2024-05-17T21:57:31.268967+0000","last_undegraded":"2024-05-17T21:57:31.268967+0000","last_fullsized":"2024-05-17T21:57:31.268967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202605+0000","last_change":"2024-05-17T21:57:30.321709+0000","last_active":"2024-05-17T21:57:32.202605+0000","last_peered":"2024-05-17T21:57:32.202605+0000","last_clean":"2024-05-17T21:57:32.202605+0000","last_became_active":"2024-05-17T21:57:30.321312+0000","last_became_peered":"2024-05-17T21:57:30.321312+0000","last_unstale":"2024-05-17T21:57:32.202605+0000","last_undegraded":"2024-05-17T21:57:32.202605+0000","last_fullsized":"2024-05-17T21:57:32.202605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272058+0000","last_change":"2024-05-17T21:57:29.310186+0000","last_active":"2024-05-17T21:57:31.272058+0000","last_peered":"2024-05-17T21:57:31.272058+0000","last_clean":"2024-05-17T21:57:31.272058+0000","last_became_active":"2024-05-17T21:57:29.309480+0000","last_became_peered":"2024-05-17T21:57:29.309480+0000","last_unstale":"2024-05-17T21:57:31.272058+0000","last_undegraded":"2024-05-17T21:57:31.272058+0000","last_fullsized":"2024-05-17T21:57:31.272058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202172+0000","last_change":"2024-05-17T21:57:30.276347+0000","last_active":"2024-05-17T21:57:32.202172+0000","last_peered":"2024-05-17T21:57:32.202172+0000","last_clean":"2024-05-17T21:57:32.202172+0000","last_became_active":"2024-05-17T21:57:30.275932+0000","last_became_peered":"2024-05-17T21:57:30.275932+0000","last_unstale":"2024-05-17T21:57:32.202172+0000","last_undegraded":"2024-05-17T21:57:32.202172+0000","last_fullsized":"2024-05-17T21:57:32.202172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198846+0000","last_change":"2024-05-17T21:57:29.307483+0000","last_active":"2024-05-17T21:57:32.198846+0000","last_peered":"2024-05-17T21:57:32.198846+0000","last_clean":"2024-05-17T21:57:32.198846+0000","last_became_active":"2024-05-17T21:57:29.305855+0000","last_became_peered":"2024-05-17T21:57:29.305855+0000","last_unstale":"2024-05-17T21:57:32.198846+0000","last_undegraded":"2024-05-17T21:57:32.198846+0000","last_fullsized":"2024-05-17T21:57:32.198846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261305+0000","last_change":"2024-05-17T21:57:30.287920+0000","last_active":"2024-05-17T21:57:31.261305+0000","last_peered":"2024-05-17T21:57:31.261305+0000","last_clean":"2024-05-17T21:57:31.261305+0000","last_became_active":"2024-05-17T21:57:30.285382+0000","last_became_peered":"2024-05-17T21:57:30.285382+0000","last_unstale":"2024-05-17T21:57:31.261305+0000","last_undegraded":"2024-05-17T21:57:31.261305+0000","last_fullsized":"2024-05-17T21:57:31.261305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:06.097854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268342+0000","last_change":"2024-05-17T21:57:29.291983+0000","last_active":"2024-05-17T21:57:31.268342+0000","last_peered":"2024-05-17T21:57:31.268342+0000","last_clean":"2024-05-17T21:57:31.268342+0000","last_became_active":"2024-05-17T21:57:29.291695+0000","last_became_peered":"2024-05-17T21:57:29.291695+0000","last_unstale":"2024-05-17T21:57:31.268342+0000","last_undegraded":"2024-05-17T21:57:31.268342+0000","last_fullsized":"2024-05-17T21:57:31.268342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201964+0000","last_change":"2024-05-17T21:57:30.274417+0000","last_active":"2024-05-17T21:57:32.201964+0000","last_peered":"2024-05-17T21:57:32.201964+0000","last_clean":"2024-05-17T21:57:32.201964+0000","last_became_active":"2024-05-17T21:57:30.272167+0000","last_became_peered":"2024-05-17T21:57:30.272167+0000","last_unstale":"2024-05-17T21:57:32.201964+0000","last_undegraded":"2024-05-17T21:57:32.201964+0000","last_fullsized":"2024-05-17T21:57:32.201964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268209+0000","last_change":"2024-05-17T21:57:29.281250+0000","last_active":"2024-05-17T21:57:31.268209+0000","last_peered":"2024-05-17T21:57:31.268209+0000","last_clean":"2024-05-17T21:57:31.268209+0000","last_became_active":"2024-05-17T21:57:29.280913+0000","last_became_peered":"2024-05-17T21:57:29.280913+0000","last_unstale":"2024-05-17T21:57:31.268209+0000","last_undegraded":"2024-05-17T21:57:31.268209+0000","last_fullsized":"2024-05-17T21:57:31.268209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203516+0000","last_change":"2024-05-17T21:57:30.288741+0000","last_active":"2024-05-17T21:57:32.203516+0000","last_peered":"2024-05-17T21:57:32.203516+0000","last_clean":"2024-05-17T21:57:32.203516+0000","last_became_active":"2024-05-17T21:57:30.288383+0000","last_became_peered":"2024-05-17T21:57:30.288383+0000","last_unstale":"2024-05-17T21:57:32.203516+0000","last_undegraded":"2024-05-17T21:57:32.203516+0000","last_fullsized":"2024-05-17T21:57:32.203516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199760+0000","last_change":"2024-05-17T21:57:29.307532+0000","last_active":"2024-05-17T21:57:32.199760+0000","last_peered":"2024-05-17T21:57:32.199760+0000","last_clean":"2024-05-17T21:57:32.199760+0000","last_became_active":"2024-05-17T21:57:29.306783+0000","last_became_peered":"2024-05-17T21:57:29.306783+0000","last_unstale":"2024-05-17T21:57:32.199760+0000","last_undegraded":"2024-05-17T21:57:32.199760+0000","last_fullsized":"2024-05-17T21:57:32.199760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272292+0000","last_change":"2024-05-17T21:57:30.316509+0000","last_active":"2024-05-17T21:57:31.272292+0000","last_peered":"2024-05-17T21:57:31.272292+0000","last_clean":"2024-05-17T21:57:31.272292+0000","last_became_active":"2024-05-17T21:57:30.315773+0000","last_became_peered":"2024-05-17T21:57:30.315773+0000","last_unstale":"2024-05-17T21:57:31.272292+0000","last_undegraded":"2024-05-17T21:57:31.272292+0000","last_fullsized":"2024-05-17T21:57:31.272292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261905+0000","last_change":"2024-05-17T21:57:30.280585+0000","last_active":"2024-05-17T21:57:31.261905+0000","last_peered":"2024-05-17T21:57:31.261905+0000","last_clean":"2024-05-17T21:57:31.261905+0000","last_became_active":"2024-05-17T21:57:30.279214+0000","last_became_peered":"2024-05-17T21:57:30.279214+0000","last_unstale":"2024-05-17T21:57:31.261905+0000","last_undegraded":"2024-05-17T21:57:31.261905+0000","last_fullsized":"2024-05-17T21:57:31.261905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199406+0000","last_change":"2024-05-17T21:57:29.310082+0000","last_active":"2024-05-17T21:57:32.199406+0000","last_peered":"2024-05-17T21:57:32.199406+0000","last_clean":"2024-05-17T21:57:32.199406+0000","last_became_active":"2024-05-17T21:57:29.309288+0000","last_became_peered":"2024-05-17T21:57:29.309288+0000","last_unstale":"2024-05-17T21:57:32.199406+0000","last_undegraded":"2024-05-17T21:57:32.199406+0000","last_fullsized":"2024-05-17T21:57:32.199406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262800+0000","last_change":"2024-05-17T21:57:30.302983+0000","last_active":"2024-05-17T21:57:31.262800+0000","last_peered":"2024-05-17T21:57:31.262800+0000","last_clean":"2024-05-17T21:57:31.262800+0000","last_became_active":"2024-05-17T21:57:30.302146+0000","last_became_peered":"2024-05-17T21:57:30.302146+0000","last_unstale":"2024-05-17T21:57:31.262800+0000","last_undegraded":"2024-05-17T21:57:31.262800+0000","last_fullsized":"2024-05-17T21:57:31.262800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199774+0000","last_change":"2024-05-17T21:57:29.308164+0000","last_active":"2024-05-17T21:57:32.199774+0000","last_peered":"2024-05-17T21:57:32.199774+0000","last_clean":"2024-05-17T21:57:32.199774+0000","last_became_active":"2024-05-17T21:57:29.307552+0000","last_became_peered":"2024-05-17T21:57:29.307552+0000","last_unstale":"2024-05-17T21:57:32.199774+0000","last_undegraded":"2024-05-17T21:57:32.199774+0000","last_fullsized":"2024-05-17T21:57:32.199774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:46.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202699+0000","last_change":"2024-05-17T21:57:30.277781+0000","last_active":"2024-05-17T21:57:32.202699+0000","last_peered":"2024-05-17T21:57:32.202699+0000","last_clean":"2024-05-17T21:57:32.202699+0000","last_became_active":"2024-05-17T21:57:30.277206+0000","last_became_peered":"2024-05-17T21:57:30.277206+0000","last_unstale":"2024-05-17T21:57:32.202699+0000","last_undegraded":"2024-05-17T21:57:32.202699+0000","last_fullsized":"2024-05-17T21:57:32.202699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201872+0000","last_change":"2024-05-17T21:57:29.280172+0000","last_active":"2024-05-17T21:57:32.201872+0000","last_peered":"2024-05-17T21:57:32.201872+0000","last_clean":"2024-05-17T21:57:32.201872+0000","last_became_active":"2024-05-17T21:57:29.279827+0000","last_became_peered":"2024-05-17T21:57:29.279827+0000","last_unstale":"2024-05-17T21:57:32.201872+0000","last_undegraded":"2024-05-17T21:57:32.201872+0000","last_fullsized":"2024-05-17T21:57:32.201872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201468+0000","last_change":"2024-05-17T21:57:30.316828+0000","last_active":"2024-05-17T21:57:32.201468+0000","last_peered":"2024-05-17T21:57:32.201468+0000","last_clean":"2024-05-17T21:57:32.201468+0000","last_became_active":"2024-05-17T21:57:30.316471+0000","last_became_peered":"2024-05-17T21:57:30.316471+0000","last_unstale":"2024-05-17T21:57:32.201468+0000","last_undegraded":"2024-05-17T21:57:32.201468+0000","last_fullsized":"2024-05-17T21:57:32.201468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202822+0000","last_change":"2024-05-17T21:57:29.305595+0000","last_active":"2024-05-17T21:57:32.202822+0000","last_peered":"2024-05-17T21:57:32.202822+0000","last_clean":"2024-05-17T21:57:32.202822+0000","last_became_active":"2024-05-17T21:57:29.305272+0000","last_became_peered":"2024-05-17T21:57:29.305272+0000","last_unstale":"2024-05-17T21:57:32.202822+0000","last_undegraded":"2024-05-17T21:57:32.202822+0000","last_fullsized":"2024-05-17T21:57:32.202822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:33.665928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:35.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-17T21:57:35.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 Processor -- start 2024-05-17T21:57:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 -- start start 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fabac0cf2e0 0x7fabac09b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac0d1950 con 0x7fabac072d70 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabac0d1670 con 0x7fabac09c570 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb375f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fabac071280 con 0x7fabac0cf2e0 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb14fb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fabac0cf2e0 0x7fabac09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb1cfc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fabac072d70 0x7fabac0ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34990/0 (socket says 172.21.15.83:34990) 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb1cfc640 1 -- 172.21.15.83:0/1433200807 learned_addr learned my addr 172.21.15.83:0/1433200807 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb14fb640 1 -- 172.21.15.83:0/1433200807 >> v1:172.21.15.155:6789/0 conn(0x7fabac09c570 legacy=0x7fabac0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb14fb640 1 -- 172.21.15.83:0/1433200807 >> v1:172.21.15.83:6789/0 conn(0x7fabac072d70 legacy=0x7fabac0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.345+0000 7fabb14fb640 1 -- 172.21.15.83:0/1433200807 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac071560 con 0x7fabac0cf2e0 2024-05-17T21:57:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb14fb640 1 --2- 172.21.15.83:0/1433200807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac09b990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faba80097c0 tx=0x7faba801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be42904b0f1728ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fab9ffff640 1 -- 172.21.15.83:0/1433200807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80b5070 con 0x7fabac0cf2e0 2024-05-17T21:57:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fab9ffff640 1 -- 172.21.15.83:0/1433200807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba80a8990 con 0x7fabac0cf2e0 2024-05-17T21:57:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fab9ffff640 1 -- 172.21.15.83:0/1433200807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80a8d60 con 0x7fabac0cf2e0 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- 172.21.15.83:0/1433200807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 msgr2=0x7fabac09b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 --2- 172.21.15.83:0/1433200807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac09b990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faba80097c0 tx=0x7faba801ec90 comp rx=0 tx=0).stop 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- 172.21.15.83:0/1433200807 shutdown_connections 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 --2- 172.21.15.83:0/1433200807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac09b990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- 172.21.15.83:0/1433200807 >> 172.21.15.83:0/1433200807 conn(0x7fabac0b5120 msgr2=0x7fabac0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- 172.21.15.83:0/1433200807 shutdown_connections 2024-05-17T21:57:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- 172.21.15.83:0/1433200807 wait complete. 2024-05-17T21:57:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 Processor -- start 2024-05-17T21:57:35.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.349+0000 7fabb375f640 1 -- start start 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabac072d70 0x7fabac0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 0x7fabac15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0d1200 con 0x7fabac0cf2e0 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0cef60 con 0x7fabac09c570 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fabac0d1380 con 0x7fabac072d70 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38932/0 (socket says 172.21.15.83:38932) 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 -- 172.21.15.83:0/973155063 learned_addr learned my addr 172.21.15.83:0/973155063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb0cfa640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 0x7fabac15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb14fb640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabac072d70 0x7fabac0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 -- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabac072d70 msgr2=0x7fabac0c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabac072d70 0x7fabac0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 -- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 msgr2=0x7fabac15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 0x7fabac15fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 -- 172.21.15.83:0/973155063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac160430 con 0x7fabac0cf2e0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb0cfa640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 0x7fabac15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb1cfc640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faba400cad0 tx=0x7faba400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4017070 con 0x7fabac0cf2e0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400ac30 con 0x7fabac0cf2e0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac0c7e10 con 0x7fabac0cf2e0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4012400 con 0x7fabac0cf2e0 2024-05-17T21:57:35.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac0c83d0 con 0x7fabac0cf2e0 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabac0b8de0 con 0x7fabac0cf2e0 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faba401a070 con 0x7fabac0cf2e0 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 0x7fab90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.353+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faba408ee80 con 0x7fabac0cf2e0 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.357+0000 7fabb14fb640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 0x7fab90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.357+0000 7fabb14fb640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 0x7fab90071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faba80b2000 tx=0x7faba8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.361+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7faba405b7b0 con 0x7fabac0cf2e0 2024-05-17T21:57:35.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.581+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fabac0c7b60 con 0x7fabac0cf2e0 2024-05-17T21:57:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fab9dffb640 1 -- 172.21.15.83:0/973155063 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7faba405fab0 con 0x7fabac0cf2e0 2024-05-17T21:57:35.575 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:35.575 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1715983038,"num_in_osds":8,"osd_in_since":1715983028,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224256000,"bytes_avail":772869857280,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4269}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-05-17T21:57:06.321233+0000","services":{}},"progress_events":{}} 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 msgr2=0x7fab90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 0x7fab90071910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faba80b2000 tx=0x7faba8008040 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 msgr2=0x7fabac0b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faba400cad0 tx=0x7faba400cf90 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 shutdown_connections 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabac0cf2e0 0x7fabac0b7a50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab9006f400 0x7fab90071910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabac09c570 0x7fabac15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 --2- 172.21.15.83:0/973155063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fabac072d70 0x7fabac0c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 >> 172.21.15.83:0/973155063 conn(0x7fabac0b5120 msgr2=0x7fabac0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:35.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 shutdown_connections 2024-05-17T21:57:35.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.585+0000 7fabb375f640 1 -- 172.21.15.83:0/973155063 wait complete. 2024-05-17T21:57:35.595 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-17T21:57:35.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:35.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 Processor -- start 2024-05-17T21:57:35.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 -- start start 2024-05-17T21:57:35.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14d4144080 0x7f14d4072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14d4073370 0x7f14d406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d4073780 con 0x7f14d40d1040 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f14d415e790 con 0x7f14d4144080 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d979d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f14d415ea40 con 0x7f14d4073370 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14d4144080 0x7f14d4072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14d4144080 0x7f14d4072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38942/0 (socket says 172.21.15.83:38942) 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 -- 172.21.15.83:0/2105615818 learned_addr learned my addr 172.21.15.83:0/2105615818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d37fe640 1 --2- 172.21.15.83:0/2105615818 >> v2:172.21.15.155:3300/0 conn(0x7f14d4073370 0x7f14d406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 -- 172.21.15.83:0/2105615818 >> v2:172.21.15.155:3300/0 conn(0x7f14d4073370 msgr2=0x7f14d406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 --2- 172.21.15.83:0/2105615818 >> v2:172.21.15.155:3300/0 conn(0x7f14d4073370 0x7f14d406f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 -- 172.21.15.83:0/2105615818 >> v1:172.21.15.155:6789/0 conn(0x7f14d40d1040 legacy=0x7f14d415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:35.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.753+0000 7f14d27fc640 1 -- 172.21.15.83:0/2105615818 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d415ed20 con 0x7f14d4144080 2024-05-17T21:57:35.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d27fc640 1 --2- 172.21.15.83:0/2105615818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d4072e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14d4072c30 tx=0x7f14c801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70e9e9f65684191 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d1ffb640 1 -- 172.21.15.83:0/2105615818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b0070 con 0x7f14d4144080 2024-05-17T21:57:35.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d1ffb640 1 -- 172.21.15.83:0/2105615818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c80ab600 con 0x7f14d4144080 2024-05-17T21:57:35.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d1ffb640 1 -- 172.21.15.83:0/2105615818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80ab9d0 con 0x7f14d4144080 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 -- 172.21.15.83:0/2105615818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 msgr2=0x7f14d4072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 --2- 172.21.15.83:0/2105615818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d4072e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14d4072c30 tx=0x7f14c801c310 comp rx=0 tx=0).stop 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 -- 172.21.15.83:0/2105615818 shutdown_connections 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 --2- 172.21.15.83:0/2105615818 >> v2:172.21.15.155:3300/0 conn(0x7f14d4073370 0x7f14d406f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 --2- 172.21.15.83:0/2105615818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d4072e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 -- 172.21.15.83:0/2105615818 >> 172.21.15.83:0/2105615818 conn(0x7f14d406cd90 msgr2=0x7f14d40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 -- 172.21.15.83:0/2105615818 shutdown_connections 2024-05-17T21:57:35.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.757+0000 7f14d979d640 1 -- 172.21.15.83:0/2105615818 wait complete. 2024-05-17T21:57:35.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 Processor -- start 2024-05-17T21:57:35.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- start start 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4073370 0x7f14d40ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d40d1040 0x7f14d415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4070480 con 0x7f14d4144080 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4144490 con 0x7f14d40d1040 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14d4070600 con 0x7f14d4073370 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38946/0 (socket says 172.21.15.83:38946) 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 -- 172.21.15.83:0/2817568302 learned_addr learned my addr 172.21.15.83:0/2817568302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d27fc640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d40d1040 0x7f14d415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 -- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4073370 msgr2=0x7f14d40ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4073370 0x7f14d40ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 -- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d40d1040 msgr2=0x7f14d415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d40d1040 0x7f14d415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 -- 172.21.15.83:0/2817568302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41fa330 con 0x7f14d4144080 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d37fe640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fd0 tx=0x7f14cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc014420 con 0x7f14d4144080 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14cc019070 con 0x7f14d4144080 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41fa610 con 0x7f14d4144080 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc014800 con 0x7f14d4144080 2024-05-17T21:57:35.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41fabd0 con 0x7f14d4144080 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d40abbf0 con 0x7f14d4144080 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.761+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14cc0149a0 con 0x7f14d4144080 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.765+0000 7f14bb7fe640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 0x7f14c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.765+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14cc08fb30 con 0x7f14d4144080 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.765+0000 7f14d2ffd640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 0x7f14c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:35.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.765+0000 7f14d2ffd640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 0x7f14c0071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009670 tx=0x7f14c4006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:35.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.769+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f14cc05ca20 con 0x7f14d4144080 2024-05-17T21:57:35.765 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T21:57:35.773+0000 7f5dfd612640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:57:35.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.925+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14d4070990 con 0x7f14c006f400 2024-05-17T21:57:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.937+0000 7f14bb7fe640 1 -- 172.21.15.83:0/2817568302 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f14d4070990 con 0x7f14c006f400 2024-05-17T21:57:35.929 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 msgr2=0x7f14c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 0x7f14c0071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009670 tx=0x7f14c4006bd0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 msgr2=0x7f14d40b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fd0 tx=0x7f14cc00e4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 shutdown_connections 2024-05-17T21:57:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d4144080 0x7f14d40b2db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f14c006f400 0x7f14c0071910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14d40d1040 0x7f14d415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 --2- 172.21.15.83:0/2817568302 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14d4073370 0x7f14d40ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 >> 172.21.15.83:0/2817568302 conn(0x7f14d406cd90 msgr2=0x7f14d4144fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 shutdown_connections 2024-05-17T21:57:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:35.941+0000 7f14d979d640 1 -- 172.21.15.83:0/2817568302 wait complete. 2024-05-17T21:57:35.951 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-17T21:57:34.966304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001960"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200427+0000","last_change":"2024-05-17T21:57:30.334206+0000","last_active":"2024-05-17T21:57:32.200427+0000","last_peered":"2024-05-17T21:57:32.200427+0000","last_clean":"2024-05-17T21:57:32.200427+0000","last_became_active":"2024-05-17T21:57:30.333625+0000","last_became_peered":"2024-05-17T21:57:30.333625+0000","last_unstale":"2024-05-17T21:57:32.200427+0000","last_undegraded":"2024-05-17T21:57:32.200427+0000","last_fullsized":"2024-05-17T21:57:32.200427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268827+0000","last_change":"2024-05-17T21:57:30.282372+0000","last_active":"2024-05-17T21:57:31.268827+0000","last_peered":"2024-05-17T21:57:31.268827+0000","last_clean":"2024-05-17T21:57:31.268827+0000","last_became_active":"2024-05-17T21:57:30.282085+0000","last_became_peered":"2024-05-17T21:57:30.282085+0000","last_unstale":"2024-05-17T21:57:31.268827+0000","last_undegraded":"2024-05-17T21:57:31.268827+0000","last_fullsized":"2024-05-17T21:57:31.268827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201211+0000","last_change":"2024-05-17T21:57:30.274388+0000","last_active":"2024-05-17T21:57:32.201211+0000","last_peered":"2024-05-17T21:57:32.201211+0000","last_clean":"2024-05-17T21:57:32.201211+0000","last_became_active":"2024-05-17T21:57:30.274079+0000","last_became_peered":"2024-05-17T21:57:30.274079+0000","last_unstale":"2024-05-17T21:57:32.201211+0000","last_undegraded":"2024-05-17T21:57:32.201211+0000","last_fullsized":"2024-05-17T21:57:32.201211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:10.441970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262187+0000","last_change":"2024-05-17T21:57:30.299694+0000","last_active":"2024-05-17T21:57:31.262187+0000","last_peered":"2024-05-17T21:57:31.262187+0000","last_clean":"2024-05-17T21:57:31.262187+0000","last_became_active":"2024-05-17T21:57:30.298884+0000","last_became_peered":"2024-05-17T21:57:30.298884+0000","last_unstale":"2024-05-17T21:57:31.262187+0000","last_undegraded":"2024-05-17T21:57:31.262187+0000","last_fullsized":"2024-05-17T21:57:31.262187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199587+0000","last_change":"2024-05-17T21:57:30.316073+0000","last_active":"2024-05-17T21:57:32.199587+0000","last_peered":"2024-05-17T21:57:32.199587+0000","last_clean":"2024-05-17T21:57:32.199587+0000","last_became_active":"2024-05-17T21:57:30.315439+0000","last_became_peered":"2024-05-17T21:57:30.315439+0000","last_unstale":"2024-05-17T21:57:32.199587+0000","last_undegraded":"2024-05-17T21:57:32.199587+0000","last_fullsized":"2024-05-17T21:57:32.199587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261267+0000","last_change":"2024-05-17T21:57:30.295598+0000","last_active":"2024-05-17T21:57:31.261267+0000","last_peered":"2024-05-17T21:57:31.261267+0000","last_clean":"2024-05-17T21:57:31.261267+0000","last_became_active":"2024-05-17T21:57:30.294722+0000","last_became_peered":"2024-05-17T21:57:30.294722+0000","last_unstale":"2024-05-17T21:57:31.261267+0000","last_undegraded":"2024-05-17T21:57:31.261267+0000","last_fullsized":"2024-05-17T21:57:31.261267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199251+0000","last_change":"2024-05-17T21:57:30.324100+0000","last_active":"2024-05-17T21:57:32.199251+0000","last_peered":"2024-05-17T21:57:32.199251+0000","last_clean":"2024-05-17T21:57:32.199251+0000","last_became_active":"2024-05-17T21:57:30.320444+0000","last_became_peered":"2024-05-17T21:57:30.320444+0000","last_unstale":"2024-05-17T21:57:32.199251+0000","last_undegraded":"2024-05-17T21:57:32.199251+0000","last_fullsized":"2024-05-17T21:57:32.199251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200972+0000","last_change":"2024-05-17T21:57:30.302526+0000","last_active":"2024-05-17T21:57:32.200972+0000","last_peered":"2024-05-17T21:57:32.200972+0000","last_clean":"2024-05-17T21:57:32.200972+0000","last_became_active":"2024-05-17T21:57:30.302136+0000","last_became_peered":"2024-05-17T21:57:30.302136+0000","last_unstale":"2024-05-17T21:57:32.200972+0000","last_undegraded":"2024-05-17T21:57:32.200972+0000","last_fullsized":"2024-05-17T21:57:32.200972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200607+0000","last_change":"2024-05-17T21:57:30.332944+0000","last_active":"2024-05-17T21:57:32.200607+0000","last_peered":"2024-05-17T21:57:32.200607+0000","last_clean":"2024-05-17T21:57:32.200607+0000","last_became_active":"2024-05-17T21:57:30.329174+0000","last_became_peered":"2024-05-17T21:57:30.329174+0000","last_unstale":"2024-05-17T21:57:32.200607+0000","last_undegraded":"2024-05-17T21:57:32.200607+0000","last_fullsized":"2024-05-17T21:57:32.200607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:10.454507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272598+0000","last_change":"2024-05-17T21:57:30.314980+0000","last_active":"2024-05-17T21:57:31.272598+0000","last_peered":"2024-05-17T21:57:31.272598+0000","last_clean":"2024-05-17T21:57:31.272598+0000","last_became_active":"2024-05-17T21:57:30.314085+0000","last_became_peered":"2024-05-17T21:57:30.314085+0000","last_unstale":"2024-05-17T21:57:31.272598+0000","last_undegraded":"2024-05-17T21:57:31.272598+0000","last_fullsized":"2024-05-17T21:57:31.272598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272161+0000","last_change":"2024-05-17T21:57:30.316071+0000","last_active":"2024-05-17T21:57:31.272161+0000","last_peered":"2024-05-17T21:57:31.272161+0000","last_clean":"2024-05-17T21:57:31.272161+0000","last_became_active":"2024-05-17T21:57:30.314772+0000","last_became_peered":"2024-05-17T21:57:30.314772+0000","last_unstale":"2024-05-17T21:57:31.272161+0000","last_undegraded":"2024-05-17T21:57:31.272161+0000","last_fullsized":"2024-05-17T21:57:31.272161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260695+0000","last_change":"2024-05-17T21:57:30.319379+0000","last_active":"2024-05-17T21:57:31.260695+0000","last_peered":"2024-05-17T21:57:31.260695+0000","last_clean":"2024-05-17T21:57:31.260695+0000","last_became_active":"2024-05-17T21:57:30.318510+0000","last_became_peered":"2024-05-17T21:57:30.318510+0000","last_unstale":"2024-05-17T21:57:31.260695+0000","last_undegraded":"2024-05-17T21:57:31.260695+0000","last_fullsized":"2024-05-17T21:57:31.260695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:22.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203239+0000","last_change":"2024-05-17T21:57:30.315381+0000","last_active":"2024-05-17T21:57:32.203239+0000","last_peered":"2024-05-17T21:57:32.203239+0000","last_clean":"2024-05-17T21:57:32.203239+0000","last_became_active":"2024-05-17T21:57:30.315076+0000","last_became_peered":"2024-05-17T21:57:30.315076+0000","last_unstale":"2024-05-17T21:57:32.203239+0000","last_undegraded":"2024-05-17T21:57:32.203239+0000","last_fullsized":"2024-05-17T21:57:32.203239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200631+0000","last_change":"2024-05-17T21:57:30.274361+0000","last_active":"2024-05-17T21:57:32.200631+0000","last_peered":"2024-05-17T21:57:32.200631+0000","last_clean":"2024-05-17T21:57:32.200631+0000","last_became_active":"2024-05-17T21:57:30.273916+0000","last_became_peered":"2024-05-17T21:57:30.273916+0000","last_unstale":"2024-05-17T21:57:32.200631+0000","last_undegraded":"2024-05-17T21:57:32.200631+0000","last_fullsized":"2024-05-17T21:57:32.200631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:47.494853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268312+0000","last_change":"2024-05-17T21:57:30.283163+0000","last_active":"2024-05-17T21:57:31.268312+0000","last_peered":"2024-05-17T21:57:31.268312+0000","last_clean":"2024-05-17T21:57:31.268312+0000","last_became_active":"2024-05-17T21:57:30.282704+0000","last_became_peered":"2024-05-17T21:57:30.282704+0000","last_unstale":"2024-05-17T21:57:31.268312+0000","last_undegraded":"2024-05-17T21:57:31.268312+0000","last_fullsized":"2024-05-17T21:57:31.268312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260004+0000","last_change":"2024-05-17T21:57:30.296030+0000","last_active":"2024-05-17T21:57:31.260004+0000","last_peered":"2024-05-17T21:57:31.260004+0000","last_clean":"2024-05-17T21:57:31.260004+0000","last_became_active":"2024-05-17T21:57:30.294696+0000","last_became_peered":"2024-05-17T21:57:30.294696+0000","last_unstale":"2024-05-17T21:57:31.260004+0000","last_undegraded":"2024-05-17T21:57:31.260004+0000","last_fullsized":"2024-05-17T21:57:31.260004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263697+0000","last_change":"2024-05-17T21:57:30.323553+0000","last_active":"2024-05-17T21:57:31.263697+0000","last_peered":"2024-05-17T21:57:31.263697+0000","last_clean":"2024-05-17T21:57:31.263697+0000","last_became_active":"2024-05-17T21:57:30.323019+0000","last_became_peered":"2024-05-17T21:57:30.323019+0000","last_unstale":"2024-05-17T21:57:31.263697+0000","last_undegraded":"2024-05-17T21:57:31.263697+0000","last_fullsized":"2024-05-17T21:57:31.263697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262779+0000","last_change":"2024-05-17T21:57:30.319719+0000","last_active":"2024-05-17T21:57:31.262779+0000","last_peered":"2024-05-17T21:57:31.262779+0000","last_clean":"2024-05-17T21:57:31.262779+0000","last_became_active":"2024-05-17T21:57:30.319335+0000","last_became_peered":"2024-05-17T21:57:30.319335+0000","last_unstale":"2024-05-17T21:57:31.262779+0000","last_undegraded":"2024-05-17T21:57:31.262779+0000","last_fullsized":"2024-05-17T21:57:31.262779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200408+0000","last_change":"2024-05-17T21:57:30.321777+0000","last_active":"2024-05-17T21:57:32.200408+0000","last_peered":"2024-05-17T21:57:32.200408+0000","last_clean":"2024-05-17T21:57:32.200408+0000","last_became_active":"2024-05-17T21:57:30.321011+0000","last_became_peered":"2024-05-17T21:57:30.321011+0000","last_unstale":"2024-05-17T21:57:32.200408+0000","last_undegraded":"2024-05-17T21:57:32.200408+0000","last_fullsized":"2024-05-17T21:57:32.200408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272173+0000","last_change":"2024-05-17T21:57:30.314965+0000","last_active":"2024-05-17T21:57:31.272173+0000","last_peered":"2024-05-17T21:57:31.272173+0000","last_clean":"2024-05-17T21:57:31.272173+0000","last_became_active":"2024-05-17T21:57:30.314249+0000","last_became_peered":"2024-05-17T21:57:30.314249+0000","last_unstale":"2024-05-17T21:57:31.272173+0000","last_undegraded":"2024-05-17T21:57:31.272173+0000","last_fullsized":"2024-05-17T21:57:31.272173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201572+0000","last_change":"2024-05-17T21:57:30.318172+0000","last_active":"2024-05-17T21:57:32.201572+0000","last_peered":"2024-05-17T21:57:32.201572+0000","last_clean":"2024-05-17T21:57:32.201572+0000","last_became_active":"2024-05-17T21:57:30.317761+0000","last_became_peered":"2024-05-17T21:57:30.317761+0000","last_unstale":"2024-05-17T21:57:32.201572+0000","last_undegraded":"2024-05-17T21:57:32.201572+0000","last_fullsized":"2024-05-17T21:57:32.201572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262717+0000","last_change":"2024-05-17T21:57:30.320885+0000","last_active":"2024-05-17T21:57:31.262717+0000","last_peered":"2024-05-17T21:57:31.262717+0000","last_clean":"2024-05-17T21:57:31.262717+0000","last_became_active":"2024-05-17T21:57:30.320268+0000","last_became_peered":"2024-05-17T21:57:30.320268+0000","last_unstale":"2024-05-17T21:57:31.262717+0000","last_undegraded":"2024-05-17T21:57:31.262717+0000","last_fullsized":"2024-05-17T21:57:31.262717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201655+0000","last_change":"2024-05-17T21:57:30.306877+0000","last_active":"2024-05-17T21:57:32.201655+0000","last_peered":"2024-05-17T21:57:32.201655+0000","last_clean":"2024-05-17T21:57:32.201655+0000","last_became_active":"2024-05-17T21:57:30.306541+0000","last_became_peered":"2024-05-17T21:57:30.306541+0000","last_unstale":"2024-05-17T21:57:32.201655+0000","last_undegraded":"2024-05-17T21:57:32.201655+0000","last_fullsized":"2024-05-17T21:57:32.201655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260191+0000","last_change":"2024-05-17T21:57:30.285039+0000","last_active":"2024-05-17T21:57:31.260191+0000","last_peered":"2024-05-17T21:57:31.260191+0000","last_clean":"2024-05-17T21:57:31.260191+0000","last_became_active":"2024-05-17T21:57:30.284132+0000","last_became_peered":"2024-05-17T21:57:30.284132+0000","last_unstale":"2024-05-17T21:57:31.260191+0000","last_undegraded":"2024-05-17T21:57:31.260191+0000","last_fullsized":"2024-05-17T21:57:31.260191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202612+0000","last_change":"2024-05-17T21:57:30.285554+0000","last_active":"2024-05-17T21:57:32.202612+0000","last_peered":"2024-05-17T21:57:32.202612+0000","last_clean":"2024-05-17T21:57:32.202612+0000","last_became_active":"2024-05-17T21:57:30.285106+0000","last_became_peered":"2024-05-17T21:57:30.285106+0000","last_unstale":"2024-05-17T21:57:32.202612+0000","last_undegraded":"2024-05-17T21:57:32.202612+0000","last_fullsized":"2024-05-17T21:57:32.202612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272595+0000","last_change":"2024-05-17T21:57:30.318707+0000","last_active":"2024-05-17T21:57:31.272595+0000","last_peered":"2024-05-17T21:57:31.272595+0000","last_clean":"2024-05-17T21:57:31.272595+0000","last_became_active":"2024-05-17T21:57:30.317977+0000","last_became_peered":"2024-05-17T21:57:30.317977+0000","last_unstale":"2024-05-17T21:57:31.272595+0000","last_undegraded":"2024-05-17T21:57:31.272595+0000","last_fullsized":"2024-05-17T21:57:31.272595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.269288+0000","last_change":"2024-05-17T21:57:30.279948+0000","last_active":"2024-05-17T21:57:31.269288+0000","last_peered":"2024-05-17T21:57:31.269288+0000","last_clean":"2024-05-17T21:57:31.269288+0000","last_became_active":"2024-05-17T21:57:30.279346+0000","last_became_peered":"2024-05-17T21:57:30.279346+0000","last_unstale":"2024-05-17T21:57:31.269288+0000","last_undegraded":"2024-05-17T21:57:31.269288+0000","last_fullsized":"2024-05-17T21:57:31.269288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203266+0000","last_change":"2024-05-17T21:57:30.289588+0000","last_active":"2024-05-17T21:57:32.203266+0000","last_peered":"2024-05-17T21:57:32.203266+0000","last_clean":"2024-05-17T21:57:32.203266+0000","last_became_active":"2024-05-17T21:57:30.289189+0000","last_became_peered":"2024-05-17T21:57:30.289189+0000","last_unstale":"2024-05-17T21:57:32.203266+0000","last_undegraded":"2024-05-17T21:57:32.203266+0000","last_fullsized":"2024-05-17T21:57:32.203266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202018+0000","last_change":"2024-05-17T21:57:30.282815+0000","last_active":"2024-05-17T21:57:32.202018+0000","last_peered":"2024-05-17T21:57:32.202018+0000","last_clean":"2024-05-17T21:57:32.202018+0000","last_became_active":"2024-05-17T21:57:30.282056+0000","last_became_peered":"2024-05-17T21:57:30.282056+0000","last_unstale":"2024-05-17T21:57:32.202018+0000","last_undegraded":"2024-05-17T21:57:32.202018+0000","last_fullsized":"2024-05-17T21:57:32.202018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261817+0000","last_change":"2024-05-17T21:57:30.281437+0000","last_active":"2024-05-17T21:57:31.261817+0000","last_peered":"2024-05-17T21:57:31.261817+0000","last_clean":"2024-05-17T21:57:31.261817+0000","last_became_active":"2024-05-17T21:57:30.280235+0000","last_became_peered":"2024-05-17T21:57:30.280235+0000","last_unstale":"2024-05-17T21:57:31.261817+0000","last_undegraded":"2024-05-17T21:57:31.261817+0000","last_fullsized":"2024-05-17T21:57:31.261817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262790+0000","last_change":"2024-05-17T21:57:30.274855+0000","last_active":"2024-05-17T21:57:31.262790+0000","last_peered":"2024-05-17T21:57:31.262790+0000","last_clean":"2024-05-17T21:57:31.262790+0000","last_became_active":"2024-05-17T21:57:30.274300+0000","last_became_peered":"2024-05-17T21:57:30.274300+0000","last_unstale":"2024-05-17T21:57:31.262790+0000","last_undegraded":"2024-05-17T21:57:31.262790+0000","last_fullsized":"2024-05-17T21:57:31.262790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:33.958058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271627+0000","last_change":"2024-05-17T21:57:30.305394+0000","last_active":"2024-05-17T21:57:31.271627+0000","last_peered":"2024-05-17T21:57:31.271627+0000","last_clean":"2024-05-17T21:57:31.271627+0000","last_became_active":"2024-05-17T21:57:30.304545+0000","last_became_peered":"2024-05-17T21:57:30.304545+0000","last_unstale":"2024-05-17T21:57:31.271627+0000","last_undegraded":"2024-05-17T21:57:31.271627+0000","last_fullsized":"2024-05-17T21:57:31.271627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260730+0000","last_change":"2024-05-17T21:57:30.271395+0000","last_active":"2024-05-17T21:57:31.260730+0000","last_peered":"2024-05-17T21:57:31.260730+0000","last_clean":"2024-05-17T21:57:31.260730+0000","last_became_active":"2024-05-17T21:57:30.270232+0000","last_became_peered":"2024-05-17T21:57:30.270232+0000","last_unstale":"2024-05-17T21:57:31.260730+0000","last_undegraded":"2024-05-17T21:57:31.260730+0000","last_fullsized":"2024-05-17T21:57:31.260730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:33.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203332+0000","last_change":"2024-05-17T21:57:29.306016+0000","last_active":"2024-05-17T21:57:32.203332+0000","last_peered":"2024-05-17T21:57:32.203332+0000","last_clean":"2024-05-17T21:57:32.203332+0000","last_became_active":"2024-05-17T21:57:29.305576+0000","last_became_peered":"2024-05-17T21:57:29.305576+0000","last_unstale":"2024-05-17T21:57:32.203332+0000","last_undegraded":"2024-05-17T21:57:32.203332+0000","last_fullsized":"2024-05-17T21:57:32.203332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202752+0000","last_change":"2024-05-17T21:57:30.283060+0000","last_active":"2024-05-17T21:57:32.202752+0000","last_peered":"2024-05-17T21:57:32.202752+0000","last_clean":"2024-05-17T21:57:32.202752+0000","last_became_active":"2024-05-17T21:57:30.282526+0000","last_became_peered":"2024-05-17T21:57:30.282526+0000","last_unstale":"2024-05-17T21:57:32.202752+0000","last_undegraded":"2024-05-17T21:57:32.202752+0000","last_fullsized":"2024-05-17T21:57:32.202752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271871+0000","last_change":"2024-05-17T21:57:29.299174+0000","last_active":"2024-05-17T21:57:31.271871+0000","last_peered":"2024-05-17T21:57:31.271871+0000","last_clean":"2024-05-17T21:57:31.271871+0000","last_became_active":"2024-05-17T21:57:29.297800+0000","last_became_peered":"2024-05-17T21:57:29.297800+0000","last_unstale":"2024-05-17T21:57:31.271871+0000","last_undegraded":"2024-05-17T21:57:31.271871+0000","last_fullsized":"2024-05-17T21:57:31.271871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202014+0000","last_change":"2024-05-17T21:57:29.290665+0000","last_active":"2024-05-17T21:57:32.202014+0000","last_peered":"2024-05-17T21:57:32.202014+0000","last_clean":"2024-05-17T21:57:32.202014+0000","last_became_active":"2024-05-17T21:57:29.290298+0000","last_became_peered":"2024-05-17T21:57:29.290298+0000","last_unstale":"2024-05-17T21:57:32.202014+0000","last_undegraded":"2024-05-17T21:57:32.202014+0000","last_fullsized":"2024-05-17T21:57:32.202014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201127+0000","last_change":"2024-05-17T21:57:30.283851+0000","last_active":"2024-05-17T21:57:32.201127+0000","last_peered":"2024-05-17T21:57:32.201127+0000","last_clean":"2024-05-17T21:57:32.201127+0000","last_became_active":"2024-05-17T21:57:30.283547+0000","last_became_peered":"2024-05-17T21:57:30.283547+0000","last_unstale":"2024-05-17T21:57:32.201127+0000","last_undegraded":"2024-05-17T21:57:32.201127+0000","last_fullsized":"2024-05-17T21:57:32.201127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260166+0000","last_change":"2024-05-17T21:57:29.297693+0000","last_active":"2024-05-17T21:57:31.260166+0000","last_peered":"2024-05-17T21:57:31.260166+0000","last_clean":"2024-05-17T21:57:31.260166+0000","last_became_active":"2024-05-17T21:57:29.296843+0000","last_became_peered":"2024-05-17T21:57:29.296843+0000","last_unstale":"2024-05-17T21:57:31.260166+0000","last_undegraded":"2024-05-17T21:57:31.260166+0000","last_fullsized":"2024-05-17T21:57:31.260166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268162+0000","last_change":"2024-05-17T21:57:30.278227+0000","last_active":"2024-05-17T21:57:31.268162+0000","last_peered":"2024-05-17T21:57:31.268162+0000","last_clean":"2024-05-17T21:57:31.268162+0000","last_became_active":"2024-05-17T21:57:30.277784+0000","last_became_peered":"2024-05-17T21:57:30.277784+0000","last_unstale":"2024-05-17T21:57:31.268162+0000","last_undegraded":"2024-05-17T21:57:31.268162+0000","last_fullsized":"2024-05-17T21:57:31.268162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271960+0000","last_change":"2024-05-17T21:57:29.290141+0000","last_active":"2024-05-17T21:57:31.271960+0000","last_peered":"2024-05-17T21:57:31.271960+0000","last_clean":"2024-05-17T21:57:31.271960+0000","last_became_active":"2024-05-17T21:57:29.288975+0000","last_became_peered":"2024-05-17T21:57:29.288975+0000","last_unstale":"2024-05-17T21:57:31.271960+0000","last_undegraded":"2024-05-17T21:57:31.271960+0000","last_fullsized":"2024-05-17T21:57:31.271960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262047+0000","last_change":"2024-05-17T21:57:30.322244+0000","last_active":"2024-05-17T21:57:31.262047+0000","last_peered":"2024-05-17T21:57:31.262047+0000","last_clean":"2024-05-17T21:57:31.262047+0000","last_became_active":"2024-05-17T21:57:30.321701+0000","last_became_peered":"2024-05-17T21:57:30.321701+0000","last_unstale":"2024-05-17T21:57:31.262047+0000","last_undegraded":"2024-05-17T21:57:31.262047+0000","last_fullsized":"2024-05-17T21:57:31.262047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262204+0000","last_change":"2024-05-17T21:57:29.293457+0000","last_active":"2024-05-17T21:57:31.262204+0000","last_peered":"2024-05-17T21:57:31.262204+0000","last_clean":"2024-05-17T21:57:31.262204+0000","last_became_active":"2024-05-17T21:57:29.292418+0000","last_became_peered":"2024-05-17T21:57:29.292418+0000","last_unstale":"2024-05-17T21:57:31.262204+0000","last_undegraded":"2024-05-17T21:57:31.262204+0000","last_fullsized":"2024-05-17T21:57:31.262204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202960+0000","last_change":"2024-05-17T21:57:30.277503+0000","last_active":"2024-05-17T21:57:32.202960+0000","last_peered":"2024-05-17T21:57:32.202960+0000","last_clean":"2024-05-17T21:57:32.202960+0000","last_became_active":"2024-05-17T21:57:30.276765+0000","last_became_peered":"2024-05-17T21:57:30.276765+0000","last_unstale":"2024-05-17T21:57:32.202960+0000","last_undegraded":"2024-05-17T21:57:32.202960+0000","last_fullsized":"2024-05-17T21:57:32.202960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268442+0000","last_change":"2024-05-17T21:57:29.284139+0000","last_active":"2024-05-17T21:57:31.268442+0000","last_peered":"2024-05-17T21:57:31.268442+0000","last_clean":"2024-05-17T21:57:31.268442+0000","last_became_active":"2024-05-17T21:57:29.283786+0000","last_became_peered":"2024-05-17T21:57:29.283786+0000","last_unstale":"2024-05-17T21:57:31.268442+0000","last_undegraded":"2024-05-17T21:57:31.268442+0000","last_fullsized":"2024-05-17T21:57:31.268442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260839+0000","last_change":"2024-05-17T21:57:30.322236+0000","last_active":"2024-05-17T21:57:31.260839+0000","last_peered":"2024-05-17T21:57:31.260839+0000","last_clean":"2024-05-17T21:57:31.260839+0000","last_became_active":"2024-05-17T21:57:30.321651+0000","last_became_peered":"2024-05-17T21:57:30.321651+0000","last_unstale":"2024-05-17T21:57:31.260839+0000","last_undegraded":"2024-05-17T21:57:31.260839+0000","last_fullsized":"2024-05-17T21:57:31.260839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:11.129074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271878+0000","last_change":"2024-05-17T21:57:29.286191+0000","last_active":"2024-05-17T21:57:31.271878+0000","last_peered":"2024-05-17T21:57:31.271878+0000","last_clean":"2024-05-17T21:57:31.271878+0000","last_became_active":"2024-05-17T21:57:29.285202+0000","last_became_peered":"2024-05-17T21:57:29.285202+0000","last_unstale":"2024-05-17T21:57:31.271878+0000","last_undegraded":"2024-05-17T21:57:31.271878+0000","last_fullsized":"2024-05-17T21:57:31.271878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199606+0000","last_change":"2024-05-17T21:57:30.323488+0000","last_active":"2024-05-17T21:57:32.199606+0000","last_peered":"2024-05-17T21:57:32.199606+0000","last_clean":"2024-05-17T21:57:32.199606+0000","last_became_active":"2024-05-17T21:57:30.322790+0000","last_became_peered":"2024-05-17T21:57:30.322790+0000","last_unstale":"2024-05-17T21:57:32.199606+0000","last_undegraded":"2024-05-17T21:57:32.199606+0000","last_fullsized":"2024-05-17T21:57:32.199606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202865+0000","last_change":"2024-05-17T21:57:29.298078+0000","last_active":"2024-05-17T21:57:32.202865+0000","last_peered":"2024-05-17T21:57:32.202865+0000","last_clean":"2024-05-17T21:57:32.202865+0000","last_became_active":"2024-05-17T21:57:29.297807+0000","last_became_peered":"2024-05-17T21:57:29.297807+0000","last_unstale":"2024-05-17T21:57:32.202865+0000","last_undegraded":"2024-05-17T21:57:32.202865+0000","last_fullsized":"2024-05-17T21:57:32.202865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:24:33.244279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201096+0000","last_change":"2024-05-17T21:57:30.318056+0000","last_active":"2024-05-17T21:57:32.201096+0000","last_peered":"2024-05-17T21:57:32.201096+0000","last_clean":"2024-05-17T21:57:32.201096+0000","last_became_active":"2024-05-17T21:57:30.317771+0000","last_became_peered":"2024-05-17T21:57:30.317771+0000","last_unstale":"2024-05-17T21:57:32.201096+0000","last_undegraded":"2024-05-17T21:57:32.201096+0000","last_fullsized":"2024-05-17T21:57:32.201096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:42.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261966+0000","last_change":"2024-05-17T21:57:29.284218+0000","last_active":"2024-05-17T21:57:31.261966+0000","last_peered":"2024-05-17T21:57:31.261966+0000","last_clean":"2024-05-17T21:57:31.261966+0000","last_became_active":"2024-05-17T21:57:29.283386+0000","last_became_peered":"2024-05-17T21:57:29.283386+0000","last_unstale":"2024-05-17T21:57:31.261966+0000","last_undegraded":"2024-05-17T21:57:31.261966+0000","last_fullsized":"2024-05-17T21:57:31.261966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200000+0000","last_change":"2024-05-17T21:57:30.321543+0000","last_active":"2024-05-17T21:57:32.200000+0000","last_peered":"2024-05-17T21:57:32.200000+0000","last_clean":"2024-05-17T21:57:32.200000+0000","last_became_active":"2024-05-17T21:57:30.320566+0000","last_became_peered":"2024-05-17T21:57:30.320566+0000","last_unstale":"2024-05-17T21:57:32.200000+0000","last_undegraded":"2024-05-17T21:57:32.200000+0000","last_fullsized":"2024-05-17T21:57:32.200000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271412+0000","last_change":"2024-05-17T21:57:29.297525+0000","last_active":"2024-05-17T21:57:31.271412+0000","last_peered":"2024-05-17T21:57:31.271412+0000","last_clean":"2024-05-17T21:57:31.271412+0000","last_became_active":"2024-05-17T21:57:29.296129+0000","last_became_peered":"2024-05-17T21:57:29.296129+0000","last_unstale":"2024-05-17T21:57:31.271412+0000","last_undegraded":"2024-05-17T21:57:31.271412+0000","last_fullsized":"2024-05-17T21:57:31.271412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271732+0000","last_change":"2024-05-17T21:57:30.306121+0000","last_active":"2024-05-17T21:57:31.271732+0000","last_peered":"2024-05-17T21:57:31.271732+0000","last_clean":"2024-05-17T21:57:31.271732+0000","last_became_active":"2024-05-17T21:57:30.304496+0000","last_became_peered":"2024-05-17T21:57:30.304496+0000","last_unstale":"2024-05-17T21:57:31.271732+0000","last_undegraded":"2024-05-17T21:57:31.271732+0000","last_fullsized":"2024-05-17T21:57:31.271732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262030+0000","last_change":"2024-05-17T21:57:29.273347+0000","last_active":"2024-05-17T21:57:31.262030+0000","last_peered":"2024-05-17T21:57:31.262030+0000","last_clean":"2024-05-17T21:57:31.262030+0000","last_became_active":"2024-05-17T21:57:29.272913+0000","last_became_peered":"2024-05-17T21:57:29.272913+0000","last_unstale":"2024-05-17T21:57:31.262030+0000","last_undegraded":"2024-05-17T21:57:31.262030+0000","last_fullsized":"2024-05-17T21:57:31.262030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:34:49.617602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199328+0000","last_change":"2024-05-17T21:57:30.322675+0000","last_active":"2024-05-17T21:57:32.199328+0000","last_peered":"2024-05-17T21:57:32.199328+0000","last_clean":"2024-05-17T21:57:32.199328+0000","last_became_active":"2024-05-17T21:57:30.321924+0000","last_became_peered":"2024-05-17T21:57:30.321924+0000","last_unstale":"2024-05-17T21:57:32.199328+0000","last_undegraded":"2024-05-17T21:57:32.199328+0000","last_fullsized":"2024-05-17T21:57:32.199328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262737+0000","last_change":"2024-05-17T21:57:29.294110+0000","last_active":"2024-05-17T21:57:31.262737+0000","last_peered":"2024-05-17T21:57:31.262737+0000","last_clean":"2024-05-17T21:57:31.262737+0000","last_became_active":"2024-05-17T21:57:29.293170+0000","last_became_peered":"2024-05-17T21:57:29.293170+0000","last_unstale":"2024-05-17T21:57:31.262737+0000","last_undegraded":"2024-05-17T21:57:31.262737+0000","last_fullsized":"2024-05-17T21:57:31.262737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:28:34.887309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200251+0000","last_change":"2024-05-17T21:57:30.315778+0000","last_active":"2024-05-17T21:57:32.200251+0000","last_peered":"2024-05-17T21:57:32.200251+0000","last_clean":"2024-05-17T21:57:32.200251+0000","last_became_active":"2024-05-17T21:57:30.314758+0000","last_became_peered":"2024-05-17T21:57:30.314758+0000","last_unstale":"2024-05-17T21:57:32.200251+0000","last_undegraded":"2024-05-17T21:57:32.200251+0000","last_fullsized":"2024-05-17T21:57:32.200251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263385+0000","last_change":"2024-05-17T21:57:29.281454+0000","last_active":"2024-05-17T21:57:31.263385+0000","last_peered":"2024-05-17T21:57:31.263385+0000","last_clean":"2024-05-17T21:57:31.263385+0000","last_became_active":"2024-05-17T21:57:29.280983+0000","last_became_peered":"2024-05-17T21:57:29.280983+0000","last_unstale":"2024-05-17T21:57:31.263385+0000","last_undegraded":"2024-05-17T21:57:31.263385+0000","last_fullsized":"2024-05-17T21:57:31.263385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200703+0000","last_change":"2024-05-17T21:57:30.334810+0000","last_active":"2024-05-17T21:57:32.200703+0000","last_peered":"2024-05-17T21:57:32.200703+0000","last_clean":"2024-05-17T21:57:32.200703+0000","last_became_active":"2024-05-17T21:57:30.332951+0000","last_became_peered":"2024-05-17T21:57:30.332951+0000","last_unstale":"2024-05-17T21:57:32.200703+0000","last_undegraded":"2024-05-17T21:57:32.200703+0000","last_fullsized":"2024-05-17T21:57:32.200703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:52.659720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268772+0000","last_change":"2024-05-17T21:57:29.296523+0000","last_active":"2024-05-17T21:57:31.268772+0000","last_peered":"2024-05-17T21:57:31.268772+0000","last_clean":"2024-05-17T21:57:31.268772+0000","last_became_active":"2024-05-17T21:57:29.296297+0000","last_became_peered":"2024-05-17T21:57:29.296297+0000","last_unstale":"2024-05-17T21:57:31.268772+0000","last_undegraded":"2024-05-17T21:57:31.268772+0000","last_fullsized":"2024-05-17T21:57:31.268772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263237+0000","last_change":"2024-05-17T21:57:30.297086+0000","last_active":"2024-05-17T21:57:31.263237+0000","last_peered":"2024-05-17T21:57:31.263237+0000","last_clean":"2024-05-17T21:57:31.263237+0000","last_became_active":"2024-05-17T21:57:30.296276+0000","last_became_peered":"2024-05-17T21:57:30.296276+0000","last_unstale":"2024-05-17T21:57:31.263237+0000","last_undegraded":"2024-05-17T21:57:31.263237+0000","last_fullsized":"2024-05-17T21:57:31.263237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262701+0000","last_change":"2024-05-17T21:57:29.282987+0000","last_active":"2024-05-17T21:57:31.262701+0000","last_peered":"2024-05-17T21:57:31.262701+0000","last_clean":"2024-05-17T21:57:31.262701+0000","last_became_active":"2024-05-17T21:57:29.282594+0000","last_became_peered":"2024-05-17T21:57:29.282594+0000","last_unstale":"2024-05-17T21:57:31.262701+0000","last_undegraded":"2024-05-17T21:57:31.262701+0000","last_fullsized":"2024-05-17T21:57:31.262701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200040+0000","last_change":"2024-05-17T21:57:30.328305+0000","last_active":"2024-05-17T21:57:32.200040+0000","last_peered":"2024-05-17T21:57:32.200040+0000","last_clean":"2024-05-17T21:57:32.200040+0000","last_became_active":"2024-05-17T21:57:30.324651+0000","last_became_peered":"2024-05-17T21:57:30.324651+0000","last_unstale":"2024-05-17T21:57:32.200040+0000","last_undegraded":"2024-05-17T21:57:32.200040+0000","last_fullsized":"2024-05-17T21:57:32.200040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199220+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:32.199220+0000","last_peered":"2024-05-17T21:57:32.199220+0000","last_clean":"2024-05-17T21:57:32.199220+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:32.199220+0000","last_undegraded":"2024-05-17T21:57:32.199220+0000","last_fullsized":"2024-05-17T21:57:32.199220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261146+0000","last_change":"2024-05-17T21:57:29.277871+0000","last_active":"2024-05-17T21:57:31.261146+0000","last_peered":"2024-05-17T21:57:31.261146+0000","last_clean":"2024-05-17T21:57:31.261146+0000","last_became_active":"2024-05-17T21:57:29.276986+0000","last_became_peered":"2024-05-17T21:57:29.276986+0000","last_unstale":"2024-05-17T21:57:31.261146+0000","last_undegraded":"2024-05-17T21:57:31.261146+0000","last_fullsized":"2024-05-17T21:57:31.261146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201481+0000","last_change":"2024-05-17T21:57:30.281160+0000","last_active":"2024-05-17T21:57:32.201481+0000","last_peered":"2024-05-17T21:57:32.201481+0000","last_clean":"2024-05-17T21:57:32.201481+0000","last_became_active":"2024-05-17T21:57:30.280004+0000","last_became_peered":"2024-05-17T21:57:30.280004+0000","last_unstale":"2024-05-17T21:57:32.201481+0000","last_undegraded":"2024-05-17T21:57:32.201481+0000","last_fullsized":"2024-05-17T21:57:32.201481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198887+0000","last_change":"2024-05-17T21:57:29.308082+0000","last_active":"2024-05-17T21:57:32.198887+0000","last_peered":"2024-05-17T21:57:32.198887+0000","last_clean":"2024-05-17T21:57:32.198887+0000","last_became_active":"2024-05-17T21:57:29.306695+0000","last_became_peered":"2024-05-17T21:57:29.306695+0000","last_unstale":"2024-05-17T21:57:32.198887+0000","last_undegraded":"2024-05-17T21:57:32.198887+0000","last_fullsized":"2024-05-17T21:57:32.198887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200656+0000","last_change":"2024-05-17T21:57:30.281759+0000","last_active":"2024-05-17T21:57:32.200656+0000","last_peered":"2024-05-17T21:57:32.200656+0000","last_clean":"2024-05-17T21:57:32.200656+0000","last_became_active":"2024-05-17T21:57:30.281449+0000","last_became_peered":"2024-05-17T21:57:30.281449+0000","last_unstale":"2024-05-17T21:57:32.200656+0000","last_undegraded":"2024-05-17T21:57:32.200656+0000","last_fullsized":"2024-05-17T21:57:32.200656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:53:58.249441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262812+0000","last_change":"2024-05-17T21:57:29.280664+0000","last_active":"2024-05-17T21:57:31.262812+0000","last_peered":"2024-05-17T21:57:31.262812+0000","last_clean":"2024-05-17T21:57:31.262812+0000","last_became_active":"2024-05-17T21:57:29.280118+0000","last_became_peered":"2024-05-17T21:57:29.280118+0000","last_unstale":"2024-05-17T21:57:31.262812+0000","last_undegraded":"2024-05-17T21:57:31.262812+0000","last_fullsized":"2024-05-17T21:57:31.262812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268285+0000","last_change":"2024-05-17T21:57:30.287528+0000","last_active":"2024-05-17T21:57:31.268285+0000","last_peered":"2024-05-17T21:57:31.268285+0000","last_clean":"2024-05-17T21:57:31.268285+0000","last_became_active":"2024-05-17T21:57:30.285405+0000","last_became_peered":"2024-05-17T21:57:30.285405+0000","last_unstale":"2024-05-17T21:57:31.268285+0000","last_undegraded":"2024-05-17T21:57:31.268285+0000","last_fullsized":"2024-05-17T21:57:31.268285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262287+0000","last_change":"2024-05-17T21:57:29.281356+0000","last_active":"2024-05-17T21:57:31.262287+0000","last_peered":"2024-05-17T21:57:31.262287+0000","last_clean":"2024-05-17T21:57:31.262287+0000","last_became_active":"2024-05-17T21:57:29.279906+0000","last_became_peered":"2024-05-17T21:57:29.279906+0000","last_unstale":"2024-05-17T21:57:31.262287+0000","last_undegraded":"2024-05-17T21:57:31.262287+0000","last_fullsized":"2024-05-17T21:57:31.262287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261918+0000","last_change":"2024-05-17T21:57:30.277518+0000","last_active":"2024-05-17T21:57:31.261918+0000","last_peered":"2024-05-17T21:57:31.261918+0000","last_clean":"2024-05-17T21:57:31.261918+0000","last_became_active":"2024-05-17T21:57:30.275892+0000","last_became_peered":"2024-05-17T21:57:30.275892+0000","last_unstale":"2024-05-17T21:57:31.261918+0000","last_undegraded":"2024-05-17T21:57:31.261918+0000","last_fullsized":"2024-05-17T21:57:31.261918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268304+0000","last_change":"2024-05-17T21:57:29.270571+0000","last_active":"2024-05-17T21:57:31.268304+0000","last_peered":"2024-05-17T21:57:31.268304+0000","last_clean":"2024-05-17T21:57:31.268304+0000","last_became_active":"2024-05-17T21:57:29.270219+0000","last_became_peered":"2024-05-17T21:57:29.270219+0000","last_unstale":"2024-05-17T21:57:31.268304+0000","last_undegraded":"2024-05-17T21:57:31.268304+0000","last_fullsized":"2024-05-17T21:57:31.268304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262482+0000","last_change":"2024-05-17T21:57:30.281497+0000","last_active":"2024-05-17T21:57:31.262482+0000","last_peered":"2024-05-17T21:57:31.262482+0000","last_clean":"2024-05-17T21:57:31.262482+0000","last_became_active":"2024-05-17T21:57:30.280473+0000","last_became_peered":"2024-05-17T21:57:30.280473+0000","last_unstale":"2024-05-17T21:57:31.262482+0000","last_undegraded":"2024-05-17T21:57:31.262482+0000","last_fullsized":"2024-05-17T21:57:31.262482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200226+0000","last_change":"2024-05-17T21:57:29.310998+0000","last_active":"2024-05-17T21:57:32.200226+0000","last_peered":"2024-05-17T21:57:32.200226+0000","last_clean":"2024-05-17T21:57:32.200226+0000","last_became_active":"2024-05-17T21:57:29.310445+0000","last_became_peered":"2024-05-17T21:57:29.310445+0000","last_unstale":"2024-05-17T21:57:32.200226+0000","last_undegraded":"2024-05-17T21:57:32.200226+0000","last_fullsized":"2024-05-17T21:57:32.200226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.270156+0000","last_change":"2024-05-17T21:57:30.286938+0000","last_active":"2024-05-17T21:57:31.270156+0000","last_peered":"2024-05-17T21:57:31.270156+0000","last_clean":"2024-05-17T21:57:31.270156+0000","last_became_active":"2024-05-17T21:57:30.286141+0000","last_became_peered":"2024-05-17T21:57:30.286141+0000","last_unstale":"2024-05-17T21:57:31.270156+0000","last_undegraded":"2024-05-17T21:57:31.270156+0000","last_fullsized":"2024-05-17T21:57:31.270156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268967+0000","last_change":"2024-05-17T21:57:29.291111+0000","last_active":"2024-05-17T21:57:31.268967+0000","last_peered":"2024-05-17T21:57:31.268967+0000","last_clean":"2024-05-17T21:57:31.268967+0000","last_became_active":"2024-05-17T21:57:29.290818+0000","last_became_peered":"2024-05-17T21:57:29.290818+0000","last_unstale":"2024-05-17T21:57:31.268967+0000","last_undegraded":"2024-05-17T21:57:31.268967+0000","last_fullsized":"2024-05-17T21:57:31.268967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202605+0000","last_change":"2024-05-17T21:57:30.321709+0000","last_active":"2024-05-17T21:57:32.202605+0000","last_peered":"2024-05-17T21:57:32.202605+0000","last_clean":"2024-05-17T21:57:32.202605+0000","last_became_active":"2024-05-17T21:57:30.321312+0000","last_became_peered":"2024-05-17T21:57:30.321312+0000","last_unstale":"2024-05-17T21:57:32.202605+0000","last_undegraded":"2024-05-17T21:57:32.202605+0000","last_fullsized":"2024-05-17T21:57:32.202605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272058+0000","last_change":"2024-05-17T21:57:29.310186+0000","last_active":"2024-05-17T21:57:31.272058+0000","last_peered":"2024-05-17T21:57:31.272058+0000","last_clean":"2024-05-17T21:57:31.272058+0000","last_became_active":"2024-05-17T21:57:29.309480+0000","last_became_peered":"2024-05-17T21:57:29.309480+0000","last_unstale":"2024-05-17T21:57:31.272058+0000","last_undegraded":"2024-05-17T21:57:31.272058+0000","last_fullsized":"2024-05-17T21:57:31.272058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202172+0000","last_change":"2024-05-17T21:57:30.276347+0000","last_active":"2024-05-17T21:57:32.202172+0000","last_peered":"2024-05-17T21:57:32.202172+0000","last_clean":"2024-05-17T21:57:32.202172+0000","last_became_active":"2024-05-17T21:57:30.275932+0000","last_became_peered":"2024-05-17T21:57:30.275932+0000","last_unstale":"2024-05-17T21:57:32.202172+0000","last_undegraded":"2024-05-17T21:57:32.202172+0000","last_fullsized":"2024-05-17T21:57:32.202172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198846+0000","last_change":"2024-05-17T21:57:29.307483+0000","last_active":"2024-05-17T21:57:32.198846+0000","last_peered":"2024-05-17T21:57:32.198846+0000","last_clean":"2024-05-17T21:57:32.198846+0000","last_became_active":"2024-05-17T21:57:29.305855+0000","last_became_peered":"2024-05-17T21:57:29.305855+0000","last_unstale":"2024-05-17T21:57:32.198846+0000","last_undegraded":"2024-05-17T21:57:32.198846+0000","last_fullsized":"2024-05-17T21:57:32.198846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261305+0000","last_change":"2024-05-17T21:57:30.287920+0000","last_active":"2024-05-17T21:57:31.261305+0000","last_peered":"2024-05-17T21:57:31.261305+0000","last_clean":"2024-05-17T21:57:31.261305+0000","last_became_active":"2024-05-17T21:57:30.285382+0000","last_became_peered":"2024-05-17T21:57:30.285382+0000","last_unstale":"2024-05-17T21:57:31.261305+0000","last_undegraded":"2024-05-17T21:57:31.261305+0000","last_fullsized":"2024-05-17T21:57:31.261305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:06.097854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268342+0000","last_change":"2024-05-17T21:57:29.291983+0000","last_active":"2024-05-17T21:57:31.268342+0000","last_peered":"2024-05-17T21:57:31.268342+0000","last_clean":"2024-05-17T21:57:31.268342+0000","last_became_active":"2024-05-17T21:57:29.291695+0000","last_became_peered":"2024-05-17T21:57:29.291695+0000","last_unstale":"2024-05-17T21:57:31.268342+0000","last_undegraded":"2024-05-17T21:57:31.268342+0000","last_fullsized":"2024-05-17T21:57:31.268342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201964+0000","last_change":"2024-05-17T21:57:30.274417+0000","last_active":"2024-05-17T21:57:32.201964+0000","last_peered":"2024-05-17T21:57:32.201964+0000","last_clean":"2024-05-17T21:57:32.201964+0000","last_became_active":"2024-05-17T21:57:30.272167+0000","last_became_peered":"2024-05-17T21:57:30.272167+0000","last_unstale":"2024-05-17T21:57:32.201964+0000","last_undegraded":"2024-05-17T21:57:32.201964+0000","last_fullsized":"2024-05-17T21:57:32.201964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268209+0000","last_change":"2024-05-17T21:57:29.281250+0000","last_active":"2024-05-17T21:57:31.268209+0000","last_peered":"2024-05-17T21:57:31.268209+0000","last_clean":"2024-05-17T21:57:31.268209+0000","last_became_active":"2024-05-17T21:57:29.280913+0000","last_became_peered":"2024-05-17T21:57:29.280913+0000","last_unstale":"2024-05-17T21:57:31.268209+0000","last_undegraded":"2024-05-17T21:57:31.268209+0000","last_fullsized":"2024-05-17T21:57:31.268209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203516+0000","last_change":"2024-05-17T21:57:30.288741+0000","last_active":"2024-05-17T21:57:32.203516+0000","last_peered":"2024-05-17T21:57:32.203516+0000","last_clean":"2024-05-17T21:57:32.203516+0000","last_became_active":"2024-05-17T21:57:30.288383+0000","last_became_peered":"2024-05-17T21:57:30.288383+0000","last_unstale":"2024-05-17T21:57:32.203516+0000","last_undegraded":"2024-05-17T21:57:32.203516+0000","last_fullsized":"2024-05-17T21:57:32.203516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199760+0000","last_change":"2024-05-17T21:57:29.307532+0000","last_active":"2024-05-17T21:57:32.199760+0000","last_peered":"2024-05-17T21:57:32.199760+0000","last_clean":"2024-05-17T21:57:32.199760+0000","last_became_active":"2024-05-17T21:57:29.306783+0000","last_became_peered":"2024-05-17T21:57:29.306783+0000","last_unstale":"2024-05-17T21:57:32.199760+0000","last_undegraded":"2024-05-17T21:57:32.199760+0000","last_fullsized":"2024-05-17T21:57:32.199760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272292+0000","last_change":"2024-05-17T21:57:30.316509+0000","last_active":"2024-05-17T21:57:31.272292+0000","last_peered":"2024-05-17T21:57:31.272292+0000","last_clean":"2024-05-17T21:57:31.272292+0000","last_became_active":"2024-05-17T21:57:30.315773+0000","last_became_peered":"2024-05-17T21:57:30.315773+0000","last_unstale":"2024-05-17T21:57:31.272292+0000","last_undegraded":"2024-05-17T21:57:31.272292+0000","last_fullsized":"2024-05-17T21:57:31.272292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261905+0000","last_change":"2024-05-17T21:57:30.280585+0000","last_active":"2024-05-17T21:57:31.261905+0000","last_peered":"2024-05-17T21:57:31.261905+0000","last_clean":"2024-05-17T21:57:31.261905+0000","last_became_active":"2024-05-17T21:57:30.279214+0000","last_became_peered":"2024-05-17T21:57:30.279214+0000","last_unstale":"2024-05-17T21:57:31.261905+0000","last_undegraded":"2024-05-17T21:57:31.261905+0000","last_fullsized":"2024-05-17T21:57:31.261905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199406+0000","last_change":"2024-05-17T21:57:29.310082+0000","last_active":"2024-05-17T21:57:32.199406+0000","last_peered":"2024-05-17T21:57:32.199406+0000","last_clean":"2024-05-17T21:57:32.199406+0000","last_became_active":"2024-05-17T21:57:29.309288+0000","last_became_peered":"2024-05-17T21:57:29.309288+0000","last_unstale":"2024-05-17T21:57:32.199406+0000","last_undegraded":"2024-05-17T21:57:32.199406+0000","last_fullsized":"2024-05-17T21:57:32.199406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262800+0000","last_change":"2024-05-17T21:57:30.302983+0000","last_active":"2024-05-17T21:57:31.262800+0000","last_peered":"2024-05-17T21:57:31.262800+0000","last_clean":"2024-05-17T21:57:31.262800+0000","last_became_active":"2024-05-17T21:57:30.302146+0000","last_became_peered":"2024-05-17T21:57:30.302146+0000","last_unstale":"2024-05-17T21:57:31.262800+0000","last_undegraded":"2024-05-17T21:57:31.262800+0000","last_fullsized":"2024-05-17T21:57:31.262800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199774+0000","last_change":"2024-05-17T21:57:29.308164+0000","last_active":"2024-05-17T21:57:32.199774+0000","last_peered":"2024-05-17T21:57:32.199774+0000","last_clean":"2024-05-17T21:57:32.199774+0000","last_became_active":"2024-05-17T21:57:29.307552+0000","last_became_peered":"2024-05-17T21:57:29.307552+0000","last_unstale":"2024-05-17T21:57:32.199774+0000","last_undegraded":"2024-05-17T21:57:32.199774+0000","last_fullsized":"2024-05-17T21:57:32.199774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:46.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202699+0000","last_change":"2024-05-17T21:57:30.277781+0000","last_active":"2024-05-17T21:57:32.202699+0000","last_peered":"2024-05-17T21:57:32.202699+0000","last_clean":"2024-05-17T21:57:32.202699+0000","last_became_active":"2024-05-17T21:57:30.277206+0000","last_became_peered":"2024-05-17T21:57:30.277206+0000","last_unstale":"2024-05-17T21:57:32.202699+0000","last_undegraded":"2024-05-17T21:57:32.202699+0000","last_fullsized":"2024-05-17T21:57:32.202699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201872+0000","last_change":"2024-05-17T21:57:29.280172+0000","last_active":"2024-05-17T21:57:32.201872+0000","last_peered":"2024-05-17T21:57:32.201872+0000","last_clean":"2024-05-17T21:57:32.201872+0000","last_became_active":"2024-05-17T21:57:29.279827+0000","last_became_peered":"2024-05-17T21:57:29.279827+0000","last_unstale":"2024-05-17T21:57:32.201872+0000","last_undegraded":"2024-05-17T21:57:32.201872+0000","last_fullsized":"2024-05-17T21:57:32.201872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201468+0000","last_change":"2024-05-17T21:57:30.316828+0000","last_active":"2024-05-17T21:57:32.201468+0000","last_peered":"2024-05-17T21:57:32.201468+0000","last_clean":"2024-05-17T21:57:32.201468+0000","last_became_active":"2024-05-17T21:57:30.316471+0000","last_became_peered":"2024-05-17T21:57:30.316471+0000","last_unstale":"2024-05-17T21:57:32.201468+0000","last_undegraded":"2024-05-17T21:57:32.201468+0000","last_fullsized":"2024-05-17T21:57:32.201468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202822+0000","last_change":"2024-05-17T21:57:29.305595+0000","last_active":"2024-05-17T21:57:32.202822+0000","last_peered":"2024-05-17T21:57:32.202822+0000","last_clean":"2024-05-17T21:57:32.202822+0000","last_became_active":"2024-05-17T21:57:29.305272+0000","last_became_peered":"2024-05-17T21:57:29.305272+0000","last_unstale":"2024-05-17T21:57:32.202822+0000","last_undegraded":"2024-05-17T21:57:32.202822+0000","last_fullsized":"2024-05-17T21:57:32.202822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:33.665928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:38.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 Processor -- start 2024-05-17T21:57:39.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 -- start start 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ea80d1040 0x7f3ea8144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea8073540 0x7f3ea806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea815ea40 con 0x7f3ea815e080 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea815ed20 con 0x7f3ea80d1040 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3eacd86640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea815f000 con 0x7f3ea8073540 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3ea815e080 0x7f3ea8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33582/0 (socket says 172.21.15.83:33582) 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3439489175 learned_addr learned my addr 172.21.15.83:0/3439489175 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea659c640 1 --2- 172.21.15.83:0/3439489175 >> v2:172.21.15.155:3300/0 conn(0x7f3ea80d1040 0x7f3ea8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea6d9d640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea8073540 0x7f3ea806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295809880 0 0) 0x7f3ea815ea40 con 0x7f3ea815e080 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.125+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e90003530 con 0x7f3ea815e080 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2865172774 0 0) 0x7f3e90003530 con 0x7f3ea815e080 2024-05-17T21:57:39.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea8073540 msgr2=0x7f3ea806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea8073540 0x7f3ea806f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 >> v2:172.21.15.155:3300/0 conn(0x7f3ea80d1040 msgr2=0x7f3ea8144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 --2- 172.21.15.83:0/3439489175 >> v2:172.21.15.155:3300/0 conn(0x7f3ea80d1040 0x7f3ea8144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea806ef30 con 0x7f3ea815e080 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f3e9c001700 con 0x7f3ea815e080 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 >> v1:172.21.15.83:6789/0 conn(0x7f3ea815e080 legacy=0x7f3ea8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e90003a10 0x7f3e90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e90004810 0x7f3e90004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e90005160 0x7f3e9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea815ed20 con 0x7f3e90004810 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea815f000 con 0x7f3e90005160 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9000bf10 con 0x7f3e90003a10 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e90004810 0x7f3e90004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea6d9d640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e90005160 0x7f3e9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea659c640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e90003a10 0x7f3e90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e90003a10 msgr2=0x7f3e90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e90003a10 0x7f3e90003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e90005160 msgr2=0x7f3e9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e90005160 0x7f3e9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3439489175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e90003530 con 0x7f3e90004810 2024-05-17T21:57:39.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea6d9d640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e90005160 0x7f3e9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e90004810 0x7f3e90004c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ceb70 tx=0x7f3e9c01f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc00641a8b5a53c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c007d40 con 0x7f3e90004810 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c005b00 con 0x7f3e90004810 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea559a640 1 -- 172.21.15.83:0/3439489175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c007070 con 0x7f3e90004810 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 -- 172.21.15.83:0/3439489175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e90004810 msgr2=0x7f3e90004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3439489175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e90004810 0x7f3e90004c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ceb70 tx=0x7f3e9c01f2f0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3ea6d9d640 1 -- 172.21.15.83:0/3439489175 reap_dead start 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 -- 172.21.15.83:0/3439489175 shutdown_connections 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 -- 172.21.15.83:0/3439489175 >> 172.21.15.83:0/3439489175 conn(0x7f3ea806cd90 msgr2=0x7f3ea80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 -- 172.21.15.83:0/3439489175 shutdown_connections 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.129+0000 7f3eacd86640 1 -- 172.21.15.83:0/3439489175 wait complete. 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 Processor -- start 2024-05-17T21:57:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 -- start start 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea81f4de0 0x7f3ea80aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ea8142590 0x7f3ea8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea80736c0 con 0x7f3ea8142180 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea8073540 con 0x7f3ea8142590 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3eacd86640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea815e490 con 0x7f3ea81f4de0 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39476/0 (socket says 172.21.15.83:39476) 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3461041738 learned_addr learned my addr 172.21.15.83:0/3461041738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea659c640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea81f4de0 0x7f3ea80aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea6d9d640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ea8142590 0x7f3ea8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea81f4de0 msgr2=0x7f3ea80aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea81f4de0 0x7f3ea80aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ea8142590 msgr2=0x7f3ea8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ea8142590 0x7f3ea8161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 -- 172.21.15.83:0/3461041738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81f53a0 con 0x7f3ea8142180 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3ea5d9b640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c09bae0 tx=0x7f3e90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c001c70 con 0x7f3ea8142180 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.133+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c007d20 con 0x7f3ea8142180 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81f5680 con 0x7f3ea8142180 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c004ab0 con 0x7f3ea8142180 2024-05-17T21:57:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81f5c40 con 0x7f3ea8142180 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea8144080 con 0x7f3ea8142180 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e9c004c50 con 0x7f3ea8142180 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3e96ffd640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 0x7f3e88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3e9c12c5a0 con 0x7f3ea8142180 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3ea659c640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 0x7f3e88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.137+0000 7f3ea659c640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 0x7f3e88071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ab0 tx=0x7f3e98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.141+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3e9c0190a0 con 0x7f3ea8142180 2024-05-17T21:57:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.297+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ea8073080 con 0x7f3e8806f400 2024-05-17T21:57:39.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.305+0000 7f3e96ffd640 1 -- 172.21.15.83:0/3461041738 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f3ea8073080 con 0x7f3e8806f400 2024-05-17T21:57:39.299 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 msgr2=0x7f3e88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 0x7f3e88071910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ab0 tx=0x7f3e98008040 comp rx=0 tx=0).stop 2024-05-17T21:57:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 msgr2=0x7f3ea8161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c09bae0 tx=0x7f3e90005840 comp rx=0 tx=0).stop 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 shutdown_connections 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e8806f400 0x7f3e88071910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ea8142590 0x7f3ea8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea8142180 0x7f3ea8161a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 --2- 172.21.15.83:0/3461041738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ea81f4de0 0x7f3ea80aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 >> 172.21.15.83:0/3461041738 conn(0x7f3ea806cd90 msgr2=0x7f3ea80b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 shutdown_connections 2024-05-17T21:57:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.309+0000 7f3eacd86640 1 -- 172.21.15.83:0/3461041738 wait complete. 2024-05-17T21:57:39.319 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-17T21:57:38.967532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.706218"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200427+0000","last_change":"2024-05-17T21:57:30.334206+0000","last_active":"2024-05-17T21:57:32.200427+0000","last_peered":"2024-05-17T21:57:32.200427+0000","last_clean":"2024-05-17T21:57:32.200427+0000","last_became_active":"2024-05-17T21:57:30.333625+0000","last_became_peered":"2024-05-17T21:57:30.333625+0000","last_unstale":"2024-05-17T21:57:32.200427+0000","last_undegraded":"2024-05-17T21:57:32.200427+0000","last_fullsized":"2024-05-17T21:57:32.200427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268827+0000","last_change":"2024-05-17T21:57:30.282372+0000","last_active":"2024-05-17T21:57:31.268827+0000","last_peered":"2024-05-17T21:57:31.268827+0000","last_clean":"2024-05-17T21:57:31.268827+0000","last_became_active":"2024-05-17T21:57:30.282085+0000","last_became_peered":"2024-05-17T21:57:30.282085+0000","last_unstale":"2024-05-17T21:57:31.268827+0000","last_undegraded":"2024-05-17T21:57:31.268827+0000","last_fullsized":"2024-05-17T21:57:31.268827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201211+0000","last_change":"2024-05-17T21:57:30.274388+0000","last_active":"2024-05-17T21:57:32.201211+0000","last_peered":"2024-05-17T21:57:32.201211+0000","last_clean":"2024-05-17T21:57:32.201211+0000","last_became_active":"2024-05-17T21:57:30.274079+0000","last_became_peered":"2024-05-17T21:57:30.274079+0000","last_unstale":"2024-05-17T21:57:32.201211+0000","last_undegraded":"2024-05-17T21:57:32.201211+0000","last_fullsized":"2024-05-17T21:57:32.201211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:10.441970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262187+0000","last_change":"2024-05-17T21:57:30.299694+0000","last_active":"2024-05-17T21:57:31.262187+0000","last_peered":"2024-05-17T21:57:31.262187+0000","last_clean":"2024-05-17T21:57:31.262187+0000","last_became_active":"2024-05-17T21:57:30.298884+0000","last_became_peered":"2024-05-17T21:57:30.298884+0000","last_unstale":"2024-05-17T21:57:31.262187+0000","last_undegraded":"2024-05-17T21:57:31.262187+0000","last_fullsized":"2024-05-17T21:57:31.262187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199587+0000","last_change":"2024-05-17T21:57:30.316073+0000","last_active":"2024-05-17T21:57:32.199587+0000","last_peered":"2024-05-17T21:57:32.199587+0000","last_clean":"2024-05-17T21:57:32.199587+0000","last_became_active":"2024-05-17T21:57:30.315439+0000","last_became_peered":"2024-05-17T21:57:30.315439+0000","last_unstale":"2024-05-17T21:57:32.199587+0000","last_undegraded":"2024-05-17T21:57:32.199587+0000","last_fullsized":"2024-05-17T21:57:32.199587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261267+0000","last_change":"2024-05-17T21:57:30.295598+0000","last_active":"2024-05-17T21:57:31.261267+0000","last_peered":"2024-05-17T21:57:31.261267+0000","last_clean":"2024-05-17T21:57:31.261267+0000","last_became_active":"2024-05-17T21:57:30.294722+0000","last_became_peered":"2024-05-17T21:57:30.294722+0000","last_unstale":"2024-05-17T21:57:31.261267+0000","last_undegraded":"2024-05-17T21:57:31.261267+0000","last_fullsized":"2024-05-17T21:57:31.261267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199251+0000","last_change":"2024-05-17T21:57:30.324100+0000","last_active":"2024-05-17T21:57:32.199251+0000","last_peered":"2024-05-17T21:57:32.199251+0000","last_clean":"2024-05-17T21:57:32.199251+0000","last_became_active":"2024-05-17T21:57:30.320444+0000","last_became_peered":"2024-05-17T21:57:30.320444+0000","last_unstale":"2024-05-17T21:57:32.199251+0000","last_undegraded":"2024-05-17T21:57:32.199251+0000","last_fullsized":"2024-05-17T21:57:32.199251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200972+0000","last_change":"2024-05-17T21:57:30.302526+0000","last_active":"2024-05-17T21:57:32.200972+0000","last_peered":"2024-05-17T21:57:32.200972+0000","last_clean":"2024-05-17T21:57:32.200972+0000","last_became_active":"2024-05-17T21:57:30.302136+0000","last_became_peered":"2024-05-17T21:57:30.302136+0000","last_unstale":"2024-05-17T21:57:32.200972+0000","last_undegraded":"2024-05-17T21:57:32.200972+0000","last_fullsized":"2024-05-17T21:57:32.200972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200607+0000","last_change":"2024-05-17T21:57:30.332944+0000","last_active":"2024-05-17T21:57:32.200607+0000","last_peered":"2024-05-17T21:57:32.200607+0000","last_clean":"2024-05-17T21:57:32.200607+0000","last_became_active":"2024-05-17T21:57:30.329174+0000","last_became_peered":"2024-05-17T21:57:30.329174+0000","last_unstale":"2024-05-17T21:57:32.200607+0000","last_undegraded":"2024-05-17T21:57:32.200607+0000","last_fullsized":"2024-05-17T21:57:32.200607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:10.454507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272598+0000","last_change":"2024-05-17T21:57:30.314980+0000","last_active":"2024-05-17T21:57:31.272598+0000","last_peered":"2024-05-17T21:57:31.272598+0000","last_clean":"2024-05-17T21:57:31.272598+0000","last_became_active":"2024-05-17T21:57:30.314085+0000","last_became_peered":"2024-05-17T21:57:30.314085+0000","last_unstale":"2024-05-17T21:57:31.272598+0000","last_undegraded":"2024-05-17T21:57:31.272598+0000","last_fullsized":"2024-05-17T21:57:31.272598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272161+0000","last_change":"2024-05-17T21:57:30.316071+0000","last_active":"2024-05-17T21:57:31.272161+0000","last_peered":"2024-05-17T21:57:31.272161+0000","last_clean":"2024-05-17T21:57:31.272161+0000","last_became_active":"2024-05-17T21:57:30.314772+0000","last_became_peered":"2024-05-17T21:57:30.314772+0000","last_unstale":"2024-05-17T21:57:31.272161+0000","last_undegraded":"2024-05-17T21:57:31.272161+0000","last_fullsized":"2024-05-17T21:57:31.272161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260695+0000","last_change":"2024-05-17T21:57:30.319379+0000","last_active":"2024-05-17T21:57:31.260695+0000","last_peered":"2024-05-17T21:57:31.260695+0000","last_clean":"2024-05-17T21:57:31.260695+0000","last_became_active":"2024-05-17T21:57:30.318510+0000","last_became_peered":"2024-05-17T21:57:30.318510+0000","last_unstale":"2024-05-17T21:57:31.260695+0000","last_undegraded":"2024-05-17T21:57:31.260695+0000","last_fullsized":"2024-05-17T21:57:31.260695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:22.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203239+0000","last_change":"2024-05-17T21:57:30.315381+0000","last_active":"2024-05-17T21:57:32.203239+0000","last_peered":"2024-05-17T21:57:32.203239+0000","last_clean":"2024-05-17T21:57:32.203239+0000","last_became_active":"2024-05-17T21:57:30.315076+0000","last_became_peered":"2024-05-17T21:57:30.315076+0000","last_unstale":"2024-05-17T21:57:32.203239+0000","last_undegraded":"2024-05-17T21:57:32.203239+0000","last_fullsized":"2024-05-17T21:57:32.203239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200631+0000","last_change":"2024-05-17T21:57:30.274361+0000","last_active":"2024-05-17T21:57:32.200631+0000","last_peered":"2024-05-17T21:57:32.200631+0000","last_clean":"2024-05-17T21:57:32.200631+0000","last_became_active":"2024-05-17T21:57:30.273916+0000","last_became_peered":"2024-05-17T21:57:30.273916+0000","last_unstale":"2024-05-17T21:57:32.200631+0000","last_undegraded":"2024-05-17T21:57:32.200631+0000","last_fullsized":"2024-05-17T21:57:32.200631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:47.494853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268312+0000","last_change":"2024-05-17T21:57:30.283163+0000","last_active":"2024-05-17T21:57:31.268312+0000","last_peered":"2024-05-17T21:57:31.268312+0000","last_clean":"2024-05-17T21:57:31.268312+0000","last_became_active":"2024-05-17T21:57:30.282704+0000","last_became_peered":"2024-05-17T21:57:30.282704+0000","last_unstale":"2024-05-17T21:57:31.268312+0000","last_undegraded":"2024-05-17T21:57:31.268312+0000","last_fullsized":"2024-05-17T21:57:31.268312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260004+0000","last_change":"2024-05-17T21:57:30.296030+0000","last_active":"2024-05-17T21:57:31.260004+0000","last_peered":"2024-05-17T21:57:31.260004+0000","last_clean":"2024-05-17T21:57:31.260004+0000","last_became_active":"2024-05-17T21:57:30.294696+0000","last_became_peered":"2024-05-17T21:57:30.294696+0000","last_unstale":"2024-05-17T21:57:31.260004+0000","last_undegraded":"2024-05-17T21:57:31.260004+0000","last_fullsized":"2024-05-17T21:57:31.260004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263697+0000","last_change":"2024-05-17T21:57:30.323553+0000","last_active":"2024-05-17T21:57:31.263697+0000","last_peered":"2024-05-17T21:57:31.263697+0000","last_clean":"2024-05-17T21:57:31.263697+0000","last_became_active":"2024-05-17T21:57:30.323019+0000","last_became_peered":"2024-05-17T21:57:30.323019+0000","last_unstale":"2024-05-17T21:57:31.263697+0000","last_undegraded":"2024-05-17T21:57:31.263697+0000","last_fullsized":"2024-05-17T21:57:31.263697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262779+0000","last_change":"2024-05-17T21:57:30.319719+0000","last_active":"2024-05-17T21:57:31.262779+0000","last_peered":"2024-05-17T21:57:31.262779+0000","last_clean":"2024-05-17T21:57:31.262779+0000","last_became_active":"2024-05-17T21:57:30.319335+0000","last_became_peered":"2024-05-17T21:57:30.319335+0000","last_unstale":"2024-05-17T21:57:31.262779+0000","last_undegraded":"2024-05-17T21:57:31.262779+0000","last_fullsized":"2024-05-17T21:57:31.262779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200408+0000","last_change":"2024-05-17T21:57:30.321777+0000","last_active":"2024-05-17T21:57:32.200408+0000","last_peered":"2024-05-17T21:57:32.200408+0000","last_clean":"2024-05-17T21:57:32.200408+0000","last_became_active":"2024-05-17T21:57:30.321011+0000","last_became_peered":"2024-05-17T21:57:30.321011+0000","last_unstale":"2024-05-17T21:57:32.200408+0000","last_undegraded":"2024-05-17T21:57:32.200408+0000","last_fullsized":"2024-05-17T21:57:32.200408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272173+0000","last_change":"2024-05-17T21:57:30.314965+0000","last_active":"2024-05-17T21:57:31.272173+0000","last_peered":"2024-05-17T21:57:31.272173+0000","last_clean":"2024-05-17T21:57:31.272173+0000","last_became_active":"2024-05-17T21:57:30.314249+0000","last_became_peered":"2024-05-17T21:57:30.314249+0000","last_unstale":"2024-05-17T21:57:31.272173+0000","last_undegraded":"2024-05-17T21:57:31.272173+0000","last_fullsized":"2024-05-17T21:57:31.272173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201572+0000","last_change":"2024-05-17T21:57:30.318172+0000","last_active":"2024-05-17T21:57:32.201572+0000","last_peered":"2024-05-17T21:57:32.201572+0000","last_clean":"2024-05-17T21:57:32.201572+0000","last_became_active":"2024-05-17T21:57:30.317761+0000","last_became_peered":"2024-05-17T21:57:30.317761+0000","last_unstale":"2024-05-17T21:57:32.201572+0000","last_undegraded":"2024-05-17T21:57:32.201572+0000","last_fullsized":"2024-05-17T21:57:32.201572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262717+0000","last_change":"2024-05-17T21:57:30.320885+0000","last_active":"2024-05-17T21:57:31.262717+0000","last_peered":"2024-05-17T21:57:31.262717+0000","last_clean":"2024-05-17T21:57:31.262717+0000","last_became_active":"2024-05-17T21:57:30.320268+0000","last_became_peered":"2024-05-17T21:57:30.320268+0000","last_unstale":"2024-05-17T21:57:31.262717+0000","last_undegraded":"2024-05-17T21:57:31.262717+0000","last_fullsized":"2024-05-17T21:57:31.262717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201655+0000","last_change":"2024-05-17T21:57:30.306877+0000","last_active":"2024-05-17T21:57:32.201655+0000","last_peered":"2024-05-17T21:57:32.201655+0000","last_clean":"2024-05-17T21:57:32.201655+0000","last_became_active":"2024-05-17T21:57:30.306541+0000","last_became_peered":"2024-05-17T21:57:30.306541+0000","last_unstale":"2024-05-17T21:57:32.201655+0000","last_undegraded":"2024-05-17T21:57:32.201655+0000","last_fullsized":"2024-05-17T21:57:32.201655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260191+0000","last_change":"2024-05-17T21:57:30.285039+0000","last_active":"2024-05-17T21:57:31.260191+0000","last_peered":"2024-05-17T21:57:31.260191+0000","last_clean":"2024-05-17T21:57:31.260191+0000","last_became_active":"2024-05-17T21:57:30.284132+0000","last_became_peered":"2024-05-17T21:57:30.284132+0000","last_unstale":"2024-05-17T21:57:31.260191+0000","last_undegraded":"2024-05-17T21:57:31.260191+0000","last_fullsized":"2024-05-17T21:57:31.260191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202612+0000","last_change":"2024-05-17T21:57:30.285554+0000","last_active":"2024-05-17T21:57:32.202612+0000","last_peered":"2024-05-17T21:57:32.202612+0000","last_clean":"2024-05-17T21:57:32.202612+0000","last_became_active":"2024-05-17T21:57:30.285106+0000","last_became_peered":"2024-05-17T21:57:30.285106+0000","last_unstale":"2024-05-17T21:57:32.202612+0000","last_undegraded":"2024-05-17T21:57:32.202612+0000","last_fullsized":"2024-05-17T21:57:32.202612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272595+0000","last_change":"2024-05-17T21:57:30.318707+0000","last_active":"2024-05-17T21:57:31.272595+0000","last_peered":"2024-05-17T21:57:31.272595+0000","last_clean":"2024-05-17T21:57:31.272595+0000","last_became_active":"2024-05-17T21:57:30.317977+0000","last_became_peered":"2024-05-17T21:57:30.317977+0000","last_unstale":"2024-05-17T21:57:31.272595+0000","last_undegraded":"2024-05-17T21:57:31.272595+0000","last_fullsized":"2024-05-17T21:57:31.272595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.269288+0000","last_change":"2024-05-17T21:57:30.279948+0000","last_active":"2024-05-17T21:57:31.269288+0000","last_peered":"2024-05-17T21:57:31.269288+0000","last_clean":"2024-05-17T21:57:31.269288+0000","last_became_active":"2024-05-17T21:57:30.279346+0000","last_became_peered":"2024-05-17T21:57:30.279346+0000","last_unstale":"2024-05-17T21:57:31.269288+0000","last_undegraded":"2024-05-17T21:57:31.269288+0000","last_fullsized":"2024-05-17T21:57:31.269288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203266+0000","last_change":"2024-05-17T21:57:30.289588+0000","last_active":"2024-05-17T21:57:32.203266+0000","last_peered":"2024-05-17T21:57:32.203266+0000","last_clean":"2024-05-17T21:57:32.203266+0000","last_became_active":"2024-05-17T21:57:30.289189+0000","last_became_peered":"2024-05-17T21:57:30.289189+0000","last_unstale":"2024-05-17T21:57:32.203266+0000","last_undegraded":"2024-05-17T21:57:32.203266+0000","last_fullsized":"2024-05-17T21:57:32.203266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202018+0000","last_change":"2024-05-17T21:57:30.282815+0000","last_active":"2024-05-17T21:57:32.202018+0000","last_peered":"2024-05-17T21:57:32.202018+0000","last_clean":"2024-05-17T21:57:32.202018+0000","last_became_active":"2024-05-17T21:57:30.282056+0000","last_became_peered":"2024-05-17T21:57:30.282056+0000","last_unstale":"2024-05-17T21:57:32.202018+0000","last_undegraded":"2024-05-17T21:57:32.202018+0000","last_fullsized":"2024-05-17T21:57:32.202018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261817+0000","last_change":"2024-05-17T21:57:30.281437+0000","last_active":"2024-05-17T21:57:31.261817+0000","last_peered":"2024-05-17T21:57:31.261817+0000","last_clean":"2024-05-17T21:57:31.261817+0000","last_became_active":"2024-05-17T21:57:30.280235+0000","last_became_peered":"2024-05-17T21:57:30.280235+0000","last_unstale":"2024-05-17T21:57:31.261817+0000","last_undegraded":"2024-05-17T21:57:31.261817+0000","last_fullsized":"2024-05-17T21:57:31.261817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262790+0000","last_change":"2024-05-17T21:57:30.274855+0000","last_active":"2024-05-17T21:57:31.262790+0000","last_peered":"2024-05-17T21:57:31.262790+0000","last_clean":"2024-05-17T21:57:31.262790+0000","last_became_active":"2024-05-17T21:57:30.274300+0000","last_became_peered":"2024-05-17T21:57:30.274300+0000","last_unstale":"2024-05-17T21:57:31.262790+0000","last_undegraded":"2024-05-17T21:57:31.262790+0000","last_fullsized":"2024-05-17T21:57:31.262790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:33.958058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271627+0000","last_change":"2024-05-17T21:57:30.305394+0000","last_active":"2024-05-17T21:57:31.271627+0000","last_peered":"2024-05-17T21:57:31.271627+0000","last_clean":"2024-05-17T21:57:31.271627+0000","last_became_active":"2024-05-17T21:57:30.304545+0000","last_became_peered":"2024-05-17T21:57:30.304545+0000","last_unstale":"2024-05-17T21:57:31.271627+0000","last_undegraded":"2024-05-17T21:57:31.271627+0000","last_fullsized":"2024-05-17T21:57:31.271627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260730+0000","last_change":"2024-05-17T21:57:30.271395+0000","last_active":"2024-05-17T21:57:31.260730+0000","last_peered":"2024-05-17T21:57:31.260730+0000","last_clean":"2024-05-17T21:57:31.260730+0000","last_became_active":"2024-05-17T21:57:30.270232+0000","last_became_peered":"2024-05-17T21:57:30.270232+0000","last_unstale":"2024-05-17T21:57:31.260730+0000","last_undegraded":"2024-05-17T21:57:31.260730+0000","last_fullsized":"2024-05-17T21:57:31.260730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:33.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203332+0000","last_change":"2024-05-17T21:57:29.306016+0000","last_active":"2024-05-17T21:57:32.203332+0000","last_peered":"2024-05-17T21:57:32.203332+0000","last_clean":"2024-05-17T21:57:32.203332+0000","last_became_active":"2024-05-17T21:57:29.305576+0000","last_became_peered":"2024-05-17T21:57:29.305576+0000","last_unstale":"2024-05-17T21:57:32.203332+0000","last_undegraded":"2024-05-17T21:57:32.203332+0000","last_fullsized":"2024-05-17T21:57:32.203332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202752+0000","last_change":"2024-05-17T21:57:30.283060+0000","last_active":"2024-05-17T21:57:32.202752+0000","last_peered":"2024-05-17T21:57:32.202752+0000","last_clean":"2024-05-17T21:57:32.202752+0000","last_became_active":"2024-05-17T21:57:30.282526+0000","last_became_peered":"2024-05-17T21:57:30.282526+0000","last_unstale":"2024-05-17T21:57:32.202752+0000","last_undegraded":"2024-05-17T21:57:32.202752+0000","last_fullsized":"2024-05-17T21:57:32.202752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271871+0000","last_change":"2024-05-17T21:57:29.299174+0000","last_active":"2024-05-17T21:57:31.271871+0000","last_peered":"2024-05-17T21:57:31.271871+0000","last_clean":"2024-05-17T21:57:31.271871+0000","last_became_active":"2024-05-17T21:57:29.297800+0000","last_became_peered":"2024-05-17T21:57:29.297800+0000","last_unstale":"2024-05-17T21:57:31.271871+0000","last_undegraded":"2024-05-17T21:57:31.271871+0000","last_fullsized":"2024-05-17T21:57:31.271871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202014+0000","last_change":"2024-05-17T21:57:29.290665+0000","last_active":"2024-05-17T21:57:32.202014+0000","last_peered":"2024-05-17T21:57:32.202014+0000","last_clean":"2024-05-17T21:57:32.202014+0000","last_became_active":"2024-05-17T21:57:29.290298+0000","last_became_peered":"2024-05-17T21:57:29.290298+0000","last_unstale":"2024-05-17T21:57:32.202014+0000","last_undegraded":"2024-05-17T21:57:32.202014+0000","last_fullsized":"2024-05-17T21:57:32.202014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201127+0000","last_change":"2024-05-17T21:57:30.283851+0000","last_active":"2024-05-17T21:57:32.201127+0000","last_peered":"2024-05-17T21:57:32.201127+0000","last_clean":"2024-05-17T21:57:32.201127+0000","last_became_active":"2024-05-17T21:57:30.283547+0000","last_became_peered":"2024-05-17T21:57:30.283547+0000","last_unstale":"2024-05-17T21:57:32.201127+0000","last_undegraded":"2024-05-17T21:57:32.201127+0000","last_fullsized":"2024-05-17T21:57:32.201127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260166+0000","last_change":"2024-05-17T21:57:29.297693+0000","last_active":"2024-05-17T21:57:31.260166+0000","last_peered":"2024-05-17T21:57:31.260166+0000","last_clean":"2024-05-17T21:57:31.260166+0000","last_became_active":"2024-05-17T21:57:29.296843+0000","last_became_peered":"2024-05-17T21:57:29.296843+0000","last_unstale":"2024-05-17T21:57:31.260166+0000","last_undegraded":"2024-05-17T21:57:31.260166+0000","last_fullsized":"2024-05-17T21:57:31.260166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268162+0000","last_change":"2024-05-17T21:57:30.278227+0000","last_active":"2024-05-17T21:57:31.268162+0000","last_peered":"2024-05-17T21:57:31.268162+0000","last_clean":"2024-05-17T21:57:31.268162+0000","last_became_active":"2024-05-17T21:57:30.277784+0000","last_became_peered":"2024-05-17T21:57:30.277784+0000","last_unstale":"2024-05-17T21:57:31.268162+0000","last_undegraded":"2024-05-17T21:57:31.268162+0000","last_fullsized":"2024-05-17T21:57:31.268162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271960+0000","last_change":"2024-05-17T21:57:29.290141+0000","last_active":"2024-05-17T21:57:31.271960+0000","last_peered":"2024-05-17T21:57:31.271960+0000","last_clean":"2024-05-17T21:57:31.271960+0000","last_became_active":"2024-05-17T21:57:29.288975+0000","last_became_peered":"2024-05-17T21:57:29.288975+0000","last_unstale":"2024-05-17T21:57:31.271960+0000","last_undegraded":"2024-05-17T21:57:31.271960+0000","last_fullsized":"2024-05-17T21:57:31.271960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262047+0000","last_change":"2024-05-17T21:57:30.322244+0000","last_active":"2024-05-17T21:57:31.262047+0000","last_peered":"2024-05-17T21:57:31.262047+0000","last_clean":"2024-05-17T21:57:31.262047+0000","last_became_active":"2024-05-17T21:57:30.321701+0000","last_became_peered":"2024-05-17T21:57:30.321701+0000","last_unstale":"2024-05-17T21:57:31.262047+0000","last_undegraded":"2024-05-17T21:57:31.262047+0000","last_fullsized":"2024-05-17T21:57:31.262047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262204+0000","last_change":"2024-05-17T21:57:29.293457+0000","last_active":"2024-05-17T21:57:31.262204+0000","last_peered":"2024-05-17T21:57:31.262204+0000","last_clean":"2024-05-17T21:57:31.262204+0000","last_became_active":"2024-05-17T21:57:29.292418+0000","last_became_peered":"2024-05-17T21:57:29.292418+0000","last_unstale":"2024-05-17T21:57:31.262204+0000","last_undegraded":"2024-05-17T21:57:31.262204+0000","last_fullsized":"2024-05-17T21:57:31.262204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202960+0000","last_change":"2024-05-17T21:57:30.277503+0000","last_active":"2024-05-17T21:57:32.202960+0000","last_peered":"2024-05-17T21:57:32.202960+0000","last_clean":"2024-05-17T21:57:32.202960+0000","last_became_active":"2024-05-17T21:57:30.276765+0000","last_became_peered":"2024-05-17T21:57:30.276765+0000","last_unstale":"2024-05-17T21:57:32.202960+0000","last_undegraded":"2024-05-17T21:57:32.202960+0000","last_fullsized":"2024-05-17T21:57:32.202960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268442+0000","last_change":"2024-05-17T21:57:29.284139+0000","last_active":"2024-05-17T21:57:31.268442+0000","last_peered":"2024-05-17T21:57:31.268442+0000","last_clean":"2024-05-17T21:57:31.268442+0000","last_became_active":"2024-05-17T21:57:29.283786+0000","last_became_peered":"2024-05-17T21:57:29.283786+0000","last_unstale":"2024-05-17T21:57:31.268442+0000","last_undegraded":"2024-05-17T21:57:31.268442+0000","last_fullsized":"2024-05-17T21:57:31.268442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260839+0000","last_change":"2024-05-17T21:57:30.322236+0000","last_active":"2024-05-17T21:57:31.260839+0000","last_peered":"2024-05-17T21:57:31.260839+0000","last_clean":"2024-05-17T21:57:31.260839+0000","last_became_active":"2024-05-17T21:57:30.321651+0000","last_became_peered":"2024-05-17T21:57:30.321651+0000","last_unstale":"2024-05-17T21:57:31.260839+0000","last_undegraded":"2024-05-17T21:57:31.260839+0000","last_fullsized":"2024-05-17T21:57:31.260839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:11.129074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271878+0000","last_change":"2024-05-17T21:57:29.286191+0000","last_active":"2024-05-17T21:57:31.271878+0000","last_peered":"2024-05-17T21:57:31.271878+0000","last_clean":"2024-05-17T21:57:31.271878+0000","last_became_active":"2024-05-17T21:57:29.285202+0000","last_became_peered":"2024-05-17T21:57:29.285202+0000","last_unstale":"2024-05-17T21:57:31.271878+0000","last_undegraded":"2024-05-17T21:57:31.271878+0000","last_fullsized":"2024-05-17T21:57:31.271878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199606+0000","last_change":"2024-05-17T21:57:30.323488+0000","last_active":"2024-05-17T21:57:32.199606+0000","last_peered":"2024-05-17T21:57:32.199606+0000","last_clean":"2024-05-17T21:57:32.199606+0000","last_became_active":"2024-05-17T21:57:30.322790+0000","last_became_peered":"2024-05-17T21:57:30.322790+0000","last_unstale":"2024-05-17T21:57:32.199606+0000","last_undegraded":"2024-05-17T21:57:32.199606+0000","last_fullsized":"2024-05-17T21:57:32.199606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202865+0000","last_change":"2024-05-17T21:57:29.298078+0000","last_active":"2024-05-17T21:57:32.202865+0000","last_peered":"2024-05-17T21:57:32.202865+0000","last_clean":"2024-05-17T21:57:32.202865+0000","last_became_active":"2024-05-17T21:57:29.297807+0000","last_became_peered":"2024-05-17T21:57:29.297807+0000","last_unstale":"2024-05-17T21:57:32.202865+0000","last_undegraded":"2024-05-17T21:57:32.202865+0000","last_fullsized":"2024-05-17T21:57:32.202865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:24:33.244279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201096+0000","last_change":"2024-05-17T21:57:30.318056+0000","last_active":"2024-05-17T21:57:32.201096+0000","last_peered":"2024-05-17T21:57:32.201096+0000","last_clean":"2024-05-17T21:57:32.201096+0000","last_became_active":"2024-05-17T21:57:30.317771+0000","last_became_peered":"2024-05-17T21:57:30.317771+0000","last_unstale":"2024-05-17T21:57:32.201096+0000","last_undegraded":"2024-05-17T21:57:32.201096+0000","last_fullsized":"2024-05-17T21:57:32.201096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:42.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261966+0000","last_change":"2024-05-17T21:57:29.284218+0000","last_active":"2024-05-17T21:57:31.261966+0000","last_peered":"2024-05-17T21:57:31.261966+0000","last_clean":"2024-05-17T21:57:31.261966+0000","last_became_active":"2024-05-17T21:57:29.283386+0000","last_became_peered":"2024-05-17T21:57:29.283386+0000","last_unstale":"2024-05-17T21:57:31.261966+0000","last_undegraded":"2024-05-17T21:57:31.261966+0000","last_fullsized":"2024-05-17T21:57:31.261966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200000+0000","last_change":"2024-05-17T21:57:30.321543+0000","last_active":"2024-05-17T21:57:32.200000+0000","last_peered":"2024-05-17T21:57:32.200000+0000","last_clean":"2024-05-17T21:57:32.200000+0000","last_became_active":"2024-05-17T21:57:30.320566+0000","last_became_peered":"2024-05-17T21:57:30.320566+0000","last_unstale":"2024-05-17T21:57:32.200000+0000","last_undegraded":"2024-05-17T21:57:32.200000+0000","last_fullsized":"2024-05-17T21:57:32.200000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271412+0000","last_change":"2024-05-17T21:57:29.297525+0000","last_active":"2024-05-17T21:57:31.271412+0000","last_peered":"2024-05-17T21:57:31.271412+0000","last_clean":"2024-05-17T21:57:31.271412+0000","last_became_active":"2024-05-17T21:57:29.296129+0000","last_became_peered":"2024-05-17T21:57:29.296129+0000","last_unstale":"2024-05-17T21:57:31.271412+0000","last_undegraded":"2024-05-17T21:57:31.271412+0000","last_fullsized":"2024-05-17T21:57:31.271412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271732+0000","last_change":"2024-05-17T21:57:30.306121+0000","last_active":"2024-05-17T21:57:31.271732+0000","last_peered":"2024-05-17T21:57:31.271732+0000","last_clean":"2024-05-17T21:57:31.271732+0000","last_became_active":"2024-05-17T21:57:30.304496+0000","last_became_peered":"2024-05-17T21:57:30.304496+0000","last_unstale":"2024-05-17T21:57:31.271732+0000","last_undegraded":"2024-05-17T21:57:31.271732+0000","last_fullsized":"2024-05-17T21:57:31.271732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262030+0000","last_change":"2024-05-17T21:57:29.273347+0000","last_active":"2024-05-17T21:57:31.262030+0000","last_peered":"2024-05-17T21:57:31.262030+0000","last_clean":"2024-05-17T21:57:31.262030+0000","last_became_active":"2024-05-17T21:57:29.272913+0000","last_became_peered":"2024-05-17T21:57:29.272913+0000","last_unstale":"2024-05-17T21:57:31.262030+0000","last_undegraded":"2024-05-17T21:57:31.262030+0000","last_fullsized":"2024-05-17T21:57:31.262030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:34:49.617602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199328+0000","last_change":"2024-05-17T21:57:30.322675+0000","last_active":"2024-05-17T21:57:32.199328+0000","last_peered":"2024-05-17T21:57:32.199328+0000","last_clean":"2024-05-17T21:57:32.199328+0000","last_became_active":"2024-05-17T21:57:30.321924+0000","last_became_peered":"2024-05-17T21:57:30.321924+0000","last_unstale":"2024-05-17T21:57:32.199328+0000","last_undegraded":"2024-05-17T21:57:32.199328+0000","last_fullsized":"2024-05-17T21:57:32.199328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262737+0000","last_change":"2024-05-17T21:57:29.294110+0000","last_active":"2024-05-17T21:57:31.262737+0000","last_peered":"2024-05-17T21:57:31.262737+0000","last_clean":"2024-05-17T21:57:31.262737+0000","last_became_active":"2024-05-17T21:57:29.293170+0000","last_became_peered":"2024-05-17T21:57:29.293170+0000","last_unstale":"2024-05-17T21:57:31.262737+0000","last_undegraded":"2024-05-17T21:57:31.262737+0000","last_fullsized":"2024-05-17T21:57:31.262737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:28:34.887309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200251+0000","last_change":"2024-05-17T21:57:30.315778+0000","last_active":"2024-05-17T21:57:32.200251+0000","last_peered":"2024-05-17T21:57:32.200251+0000","last_clean":"2024-05-17T21:57:32.200251+0000","last_became_active":"2024-05-17T21:57:30.314758+0000","last_became_peered":"2024-05-17T21:57:30.314758+0000","last_unstale":"2024-05-17T21:57:32.200251+0000","last_undegraded":"2024-05-17T21:57:32.200251+0000","last_fullsized":"2024-05-17T21:57:32.200251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263385+0000","last_change":"2024-05-17T21:57:29.281454+0000","last_active":"2024-05-17T21:57:31.263385+0000","last_peered":"2024-05-17T21:57:31.263385+0000","last_clean":"2024-05-17T21:57:31.263385+0000","last_became_active":"2024-05-17T21:57:29.280983+0000","last_became_peered":"2024-05-17T21:57:29.280983+0000","last_unstale":"2024-05-17T21:57:31.263385+0000","last_undegraded":"2024-05-17T21:57:31.263385+0000","last_fullsized":"2024-05-17T21:57:31.263385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200703+0000","last_change":"2024-05-17T21:57:30.334810+0000","last_active":"2024-05-17T21:57:32.200703+0000","last_peered":"2024-05-17T21:57:32.200703+0000","last_clean":"2024-05-17T21:57:32.200703+0000","last_became_active":"2024-05-17T21:57:30.332951+0000","last_became_peered":"2024-05-17T21:57:30.332951+0000","last_unstale":"2024-05-17T21:57:32.200703+0000","last_undegraded":"2024-05-17T21:57:32.200703+0000","last_fullsized":"2024-05-17T21:57:32.200703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:52.659720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268772+0000","last_change":"2024-05-17T21:57:29.296523+0000","last_active":"2024-05-17T21:57:31.268772+0000","last_peered":"2024-05-17T21:57:31.268772+0000","last_clean":"2024-05-17T21:57:31.268772+0000","last_became_active":"2024-05-17T21:57:29.296297+0000","last_became_peered":"2024-05-17T21:57:29.296297+0000","last_unstale":"2024-05-17T21:57:31.268772+0000","last_undegraded":"2024-05-17T21:57:31.268772+0000","last_fullsized":"2024-05-17T21:57:31.268772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263237+0000","last_change":"2024-05-17T21:57:30.297086+0000","last_active":"2024-05-17T21:57:31.263237+0000","last_peered":"2024-05-17T21:57:31.263237+0000","last_clean":"2024-05-17T21:57:31.263237+0000","last_became_active":"2024-05-17T21:57:30.296276+0000","last_became_peered":"2024-05-17T21:57:30.296276+0000","last_unstale":"2024-05-17T21:57:31.263237+0000","last_undegraded":"2024-05-17T21:57:31.263237+0000","last_fullsized":"2024-05-17T21:57:31.263237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262701+0000","last_change":"2024-05-17T21:57:29.282987+0000","last_active":"2024-05-17T21:57:31.262701+0000","last_peered":"2024-05-17T21:57:31.262701+0000","last_clean":"2024-05-17T21:57:31.262701+0000","last_became_active":"2024-05-17T21:57:29.282594+0000","last_became_peered":"2024-05-17T21:57:29.282594+0000","last_unstale":"2024-05-17T21:57:31.262701+0000","last_undegraded":"2024-05-17T21:57:31.262701+0000","last_fullsized":"2024-05-17T21:57:31.262701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200040+0000","last_change":"2024-05-17T21:57:30.328305+0000","last_active":"2024-05-17T21:57:32.200040+0000","last_peered":"2024-05-17T21:57:32.200040+0000","last_clean":"2024-05-17T21:57:32.200040+0000","last_became_active":"2024-05-17T21:57:30.324651+0000","last_became_peered":"2024-05-17T21:57:30.324651+0000","last_unstale":"2024-05-17T21:57:32.200040+0000","last_undegraded":"2024-05-17T21:57:32.200040+0000","last_fullsized":"2024-05-17T21:57:32.200040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199220+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:32.199220+0000","last_peered":"2024-05-17T21:57:32.199220+0000","last_clean":"2024-05-17T21:57:32.199220+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:32.199220+0000","last_undegraded":"2024-05-17T21:57:32.199220+0000","last_fullsized":"2024-05-17T21:57:32.199220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261146+0000","last_change":"2024-05-17T21:57:29.277871+0000","last_active":"2024-05-17T21:57:31.261146+0000","last_peered":"2024-05-17T21:57:31.261146+0000","last_clean":"2024-05-17T21:57:31.261146+0000","last_became_active":"2024-05-17T21:57:29.276986+0000","last_became_peered":"2024-05-17T21:57:29.276986+0000","last_unstale":"2024-05-17T21:57:31.261146+0000","last_undegraded":"2024-05-17T21:57:31.261146+0000","last_fullsized":"2024-05-17T21:57:31.261146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201481+0000","last_change":"2024-05-17T21:57:30.281160+0000","last_active":"2024-05-17T21:57:32.201481+0000","last_peered":"2024-05-17T21:57:32.201481+0000","last_clean":"2024-05-17T21:57:32.201481+0000","last_became_active":"2024-05-17T21:57:30.280004+0000","last_became_peered":"2024-05-17T21:57:30.280004+0000","last_unstale":"2024-05-17T21:57:32.201481+0000","last_undegraded":"2024-05-17T21:57:32.201481+0000","last_fullsized":"2024-05-17T21:57:32.201481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198887+0000","last_change":"2024-05-17T21:57:29.308082+0000","last_active":"2024-05-17T21:57:32.198887+0000","last_peered":"2024-05-17T21:57:32.198887+0000","last_clean":"2024-05-17T21:57:32.198887+0000","last_became_active":"2024-05-17T21:57:29.306695+0000","last_became_peered":"2024-05-17T21:57:29.306695+0000","last_unstale":"2024-05-17T21:57:32.198887+0000","last_undegraded":"2024-05-17T21:57:32.198887+0000","last_fullsized":"2024-05-17T21:57:32.198887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200656+0000","last_change":"2024-05-17T21:57:30.281759+0000","last_active":"2024-05-17T21:57:32.200656+0000","last_peered":"2024-05-17T21:57:32.200656+0000","last_clean":"2024-05-17T21:57:32.200656+0000","last_became_active":"2024-05-17T21:57:30.281449+0000","last_became_peered":"2024-05-17T21:57:30.281449+0000","last_unstale":"2024-05-17T21:57:32.200656+0000","last_undegraded":"2024-05-17T21:57:32.200656+0000","last_fullsized":"2024-05-17T21:57:32.200656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:53:58.249441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262812+0000","last_change":"2024-05-17T21:57:29.280664+0000","last_active":"2024-05-17T21:57:31.262812+0000","last_peered":"2024-05-17T21:57:31.262812+0000","last_clean":"2024-05-17T21:57:31.262812+0000","last_became_active":"2024-05-17T21:57:29.280118+0000","last_became_peered":"2024-05-17T21:57:29.280118+0000","last_unstale":"2024-05-17T21:57:31.262812+0000","last_undegraded":"2024-05-17T21:57:31.262812+0000","last_fullsized":"2024-05-17T21:57:31.262812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268285+0000","last_change":"2024-05-17T21:57:30.287528+0000","last_active":"2024-05-17T21:57:31.268285+0000","last_peered":"2024-05-17T21:57:31.268285+0000","last_clean":"2024-05-17T21:57:31.268285+0000","last_became_active":"2024-05-17T21:57:30.285405+0000","last_became_peered":"2024-05-17T21:57:30.285405+0000","last_unstale":"2024-05-17T21:57:31.268285+0000","last_undegraded":"2024-05-17T21:57:31.268285+0000","last_fullsized":"2024-05-17T21:57:31.268285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262287+0000","last_change":"2024-05-17T21:57:29.281356+0000","last_active":"2024-05-17T21:57:31.262287+0000","last_peered":"2024-05-17T21:57:31.262287+0000","last_clean":"2024-05-17T21:57:31.262287+0000","last_became_active":"2024-05-17T21:57:29.279906+0000","last_became_peered":"2024-05-17T21:57:29.279906+0000","last_unstale":"2024-05-17T21:57:31.262287+0000","last_undegraded":"2024-05-17T21:57:31.262287+0000","last_fullsized":"2024-05-17T21:57:31.262287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261918+0000","last_change":"2024-05-17T21:57:30.277518+0000","last_active":"2024-05-17T21:57:31.261918+0000","last_peered":"2024-05-17T21:57:31.261918+0000","last_clean":"2024-05-17T21:57:31.261918+0000","last_became_active":"2024-05-17T21:57:30.275892+0000","last_became_peered":"2024-05-17T21:57:30.275892+0000","last_unstale":"2024-05-17T21:57:31.261918+0000","last_undegraded":"2024-05-17T21:57:31.261918+0000","last_fullsized":"2024-05-17T21:57:31.261918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268304+0000","last_change":"2024-05-17T21:57:29.270571+0000","last_active":"2024-05-17T21:57:31.268304+0000","last_peered":"2024-05-17T21:57:31.268304+0000","last_clean":"2024-05-17T21:57:31.268304+0000","last_became_active":"2024-05-17T21:57:29.270219+0000","last_became_peered":"2024-05-17T21:57:29.270219+0000","last_unstale":"2024-05-17T21:57:31.268304+0000","last_undegraded":"2024-05-17T21:57:31.268304+0000","last_fullsized":"2024-05-17T21:57:31.268304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262482+0000","last_change":"2024-05-17T21:57:30.281497+0000","last_active":"2024-05-17T21:57:31.262482+0000","last_peered":"2024-05-17T21:57:31.262482+0000","last_clean":"2024-05-17T21:57:31.262482+0000","last_became_active":"2024-05-17T21:57:30.280473+0000","last_became_peered":"2024-05-17T21:57:30.280473+0000","last_unstale":"2024-05-17T21:57:31.262482+0000","last_undegraded":"2024-05-17T21:57:31.262482+0000","last_fullsized":"2024-05-17T21:57:31.262482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200226+0000","last_change":"2024-05-17T21:57:29.310998+0000","last_active":"2024-05-17T21:57:32.200226+0000","last_peered":"2024-05-17T21:57:32.200226+0000","last_clean":"2024-05-17T21:57:32.200226+0000","last_became_active":"2024-05-17T21:57:29.310445+0000","last_became_peered":"2024-05-17T21:57:29.310445+0000","last_unstale":"2024-05-17T21:57:32.200226+0000","last_undegraded":"2024-05-17T21:57:32.200226+0000","last_fullsized":"2024-05-17T21:57:32.200226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.270156+0000","last_change":"2024-05-17T21:57:30.286938+0000","last_active":"2024-05-17T21:57:31.270156+0000","last_peered":"2024-05-17T21:57:31.270156+0000","last_clean":"2024-05-17T21:57:31.270156+0000","last_became_active":"2024-05-17T21:57:30.286141+0000","last_became_peered":"2024-05-17T21:57:30.286141+0000","last_unstale":"2024-05-17T21:57:31.270156+0000","last_undegraded":"2024-05-17T21:57:31.270156+0000","last_fullsized":"2024-05-17T21:57:31.270156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268967+0000","last_change":"2024-05-17T21:57:29.291111+0000","last_active":"2024-05-17T21:57:31.268967+0000","last_peered":"2024-05-17T21:57:31.268967+0000","last_clean":"2024-05-17T21:57:31.268967+0000","last_became_active":"2024-05-17T21:57:29.290818+0000","last_became_peered":"2024-05-17T21:57:29.290818+0000","last_unstale":"2024-05-17T21:57:31.268967+0000","last_undegraded":"2024-05-17T21:57:31.268967+0000","last_fullsized":"2024-05-17T21:57:31.268967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202605+0000","last_change":"2024-05-17T21:57:30.321709+0000","last_active":"2024-05-17T21:57:32.202605+0000","last_peered":"2024-05-17T21:57:32.202605+0000","last_clean":"2024-05-17T21:57:32.202605+0000","last_became_active":"2024-05-17T21:57:30.321312+0000","last_became_peered":"2024-05-17T21:57:30.321312+0000","last_unstale":"2024-05-17T21:57:32.202605+0000","last_undegraded":"2024-05-17T21:57:32.202605+0000","last_fullsized":"2024-05-17T21:57:32.202605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272058+0000","last_change":"2024-05-17T21:57:29.310186+0000","last_active":"2024-05-17T21:57:31.272058+0000","last_peered":"2024-05-17T21:57:31.272058+0000","last_clean":"2024-05-17T21:57:31.272058+0000","last_became_active":"2024-05-17T21:57:29.309480+0000","last_became_peered":"2024-05-17T21:57:29.309480+0000","last_unstale":"2024-05-17T21:57:31.272058+0000","last_undegraded":"2024-05-17T21:57:31.272058+0000","last_fullsized":"2024-05-17T21:57:31.272058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202172+0000","last_change":"2024-05-17T21:57:30.276347+0000","last_active":"2024-05-17T21:57:32.202172+0000","last_peered":"2024-05-17T21:57:32.202172+0000","last_clean":"2024-05-17T21:57:32.202172+0000","last_became_active":"2024-05-17T21:57:30.275932+0000","last_became_peered":"2024-05-17T21:57:30.275932+0000","last_unstale":"2024-05-17T21:57:32.202172+0000","last_undegraded":"2024-05-17T21:57:32.202172+0000","last_fullsized":"2024-05-17T21:57:32.202172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198846+0000","last_change":"2024-05-17T21:57:29.307483+0000","last_active":"2024-05-17T21:57:32.198846+0000","last_peered":"2024-05-17T21:57:32.198846+0000","last_clean":"2024-05-17T21:57:32.198846+0000","last_became_active":"2024-05-17T21:57:29.305855+0000","last_became_peered":"2024-05-17T21:57:29.305855+0000","last_unstale":"2024-05-17T21:57:32.198846+0000","last_undegraded":"2024-05-17T21:57:32.198846+0000","last_fullsized":"2024-05-17T21:57:32.198846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261305+0000","last_change":"2024-05-17T21:57:30.287920+0000","last_active":"2024-05-17T21:57:31.261305+0000","last_peered":"2024-05-17T21:57:31.261305+0000","last_clean":"2024-05-17T21:57:31.261305+0000","last_became_active":"2024-05-17T21:57:30.285382+0000","last_became_peered":"2024-05-17T21:57:30.285382+0000","last_unstale":"2024-05-17T21:57:31.261305+0000","last_undegraded":"2024-05-17T21:57:31.261305+0000","last_fullsized":"2024-05-17T21:57:31.261305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:06.097854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268342+0000","last_change":"2024-05-17T21:57:29.291983+0000","last_active":"2024-05-17T21:57:31.268342+0000","last_peered":"2024-05-17T21:57:31.268342+0000","last_clean":"2024-05-17T21:57:31.268342+0000","last_became_active":"2024-05-17T21:57:29.291695+0000","last_became_peered":"2024-05-17T21:57:29.291695+0000","last_unstale":"2024-05-17T21:57:31.268342+0000","last_undegraded":"2024-05-17T21:57:31.268342+0000","last_fullsized":"2024-05-17T21:57:31.268342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201964+0000","last_change":"2024-05-17T21:57:30.274417+0000","last_active":"2024-05-17T21:57:32.201964+0000","last_peered":"2024-05-17T21:57:32.201964+0000","last_clean":"2024-05-17T21:57:32.201964+0000","last_became_active":"2024-05-17T21:57:30.272167+0000","last_became_peered":"2024-05-17T21:57:30.272167+0000","last_unstale":"2024-05-17T21:57:32.201964+0000","last_undegraded":"2024-05-17T21:57:32.201964+0000","last_fullsized":"2024-05-17T21:57:32.201964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268209+0000","last_change":"2024-05-17T21:57:29.281250+0000","last_active":"2024-05-17T21:57:31.268209+0000","last_peered":"2024-05-17T21:57:31.268209+0000","last_clean":"2024-05-17T21:57:31.268209+0000","last_became_active":"2024-05-17T21:57:29.280913+0000","last_became_peered":"2024-05-17T21:57:29.280913+0000","last_unstale":"2024-05-17T21:57:31.268209+0000","last_undegraded":"2024-05-17T21:57:31.268209+0000","last_fullsized":"2024-05-17T21:57:31.268209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203516+0000","last_change":"2024-05-17T21:57:30.288741+0000","last_active":"2024-05-17T21:57:32.203516+0000","last_peered":"2024-05-17T21:57:32.203516+0000","last_clean":"2024-05-17T21:57:32.203516+0000","last_became_active":"2024-05-17T21:57:30.288383+0000","last_became_peered":"2024-05-17T21:57:30.288383+0000","last_unstale":"2024-05-17T21:57:32.203516+0000","last_undegraded":"2024-05-17T21:57:32.203516+0000","last_fullsized":"2024-05-17T21:57:32.203516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199760+0000","last_change":"2024-05-17T21:57:29.307532+0000","last_active":"2024-05-17T21:57:32.199760+0000","last_peered":"2024-05-17T21:57:32.199760+0000","last_clean":"2024-05-17T21:57:32.199760+0000","last_became_active":"2024-05-17T21:57:29.306783+0000","last_became_peered":"2024-05-17T21:57:29.306783+0000","last_unstale":"2024-05-17T21:57:32.199760+0000","last_undegraded":"2024-05-17T21:57:32.199760+0000","last_fullsized":"2024-05-17T21:57:32.199760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272292+0000","last_change":"2024-05-17T21:57:30.316509+0000","last_active":"2024-05-17T21:57:31.272292+0000","last_peered":"2024-05-17T21:57:31.272292+0000","last_clean":"2024-05-17T21:57:31.272292+0000","last_became_active":"2024-05-17T21:57:30.315773+0000","last_became_peered":"2024-05-17T21:57:30.315773+0000","last_unstale":"2024-05-17T21:57:31.272292+0000","last_undegraded":"2024-05-17T21:57:31.272292+0000","last_fullsized":"2024-05-17T21:57:31.272292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261905+0000","last_change":"2024-05-17T21:57:30.280585+0000","last_active":"2024-05-17T21:57:31.261905+0000","last_peered":"2024-05-17T21:57:31.261905+0000","last_clean":"2024-05-17T21:57:31.261905+0000","last_became_active":"2024-05-17T21:57:30.279214+0000","last_became_peered":"2024-05-17T21:57:30.279214+0000","last_unstale":"2024-05-17T21:57:31.261905+0000","last_undegraded":"2024-05-17T21:57:31.261905+0000","last_fullsized":"2024-05-17T21:57:31.261905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199406+0000","last_change":"2024-05-17T21:57:29.310082+0000","last_active":"2024-05-17T21:57:32.199406+0000","last_peered":"2024-05-17T21:57:32.199406+0000","last_clean":"2024-05-17T21:57:32.199406+0000","last_became_active":"2024-05-17T21:57:29.309288+0000","last_became_peered":"2024-05-17T21:57:29.309288+0000","last_unstale":"2024-05-17T21:57:32.199406+0000","last_undegraded":"2024-05-17T21:57:32.199406+0000","last_fullsized":"2024-05-17T21:57:32.199406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262800+0000","last_change":"2024-05-17T21:57:30.302983+0000","last_active":"2024-05-17T21:57:31.262800+0000","last_peered":"2024-05-17T21:57:31.262800+0000","last_clean":"2024-05-17T21:57:31.262800+0000","last_became_active":"2024-05-17T21:57:30.302146+0000","last_became_peered":"2024-05-17T21:57:30.302146+0000","last_unstale":"2024-05-17T21:57:31.262800+0000","last_undegraded":"2024-05-17T21:57:31.262800+0000","last_fullsized":"2024-05-17T21:57:31.262800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199774+0000","last_change":"2024-05-17T21:57:29.308164+0000","last_active":"2024-05-17T21:57:32.199774+0000","last_peered":"2024-05-17T21:57:32.199774+0000","last_clean":"2024-05-17T21:57:32.199774+0000","last_became_active":"2024-05-17T21:57:29.307552+0000","last_became_peered":"2024-05-17T21:57:29.307552+0000","last_unstale":"2024-05-17T21:57:32.199774+0000","last_undegraded":"2024-05-17T21:57:32.199774+0000","last_fullsized":"2024-05-17T21:57:32.199774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:46.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202699+0000","last_change":"2024-05-17T21:57:30.277781+0000","last_active":"2024-05-17T21:57:32.202699+0000","last_peered":"2024-05-17T21:57:32.202699+0000","last_clean":"2024-05-17T21:57:32.202699+0000","last_became_active":"2024-05-17T21:57:30.277206+0000","last_became_peered":"2024-05-17T21:57:30.277206+0000","last_unstale":"2024-05-17T21:57:32.202699+0000","last_undegraded":"2024-05-17T21:57:32.202699+0000","last_fullsized":"2024-05-17T21:57:32.202699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201872+0000","last_change":"2024-05-17T21:57:29.280172+0000","last_active":"2024-05-17T21:57:32.201872+0000","last_peered":"2024-05-17T21:57:32.201872+0000","last_clean":"2024-05-17T21:57:32.201872+0000","last_became_active":"2024-05-17T21:57:29.279827+0000","last_became_peered":"2024-05-17T21:57:29.279827+0000","last_unstale":"2024-05-17T21:57:32.201872+0000","last_undegraded":"2024-05-17T21:57:32.201872+0000","last_fullsized":"2024-05-17T21:57:32.201872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201468+0000","last_change":"2024-05-17T21:57:30.316828+0000","last_active":"2024-05-17T21:57:32.201468+0000","last_peered":"2024-05-17T21:57:32.201468+0000","last_clean":"2024-05-17T21:57:32.201468+0000","last_became_active":"2024-05-17T21:57:30.316471+0000","last_became_peered":"2024-05-17T21:57:30.316471+0000","last_unstale":"2024-05-17T21:57:32.201468+0000","last_undegraded":"2024-05-17T21:57:32.201468+0000","last_fullsized":"2024-05-17T21:57:32.201468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202822+0000","last_change":"2024-05-17T21:57:29.305595+0000","last_active":"2024-05-17T21:57:32.202822+0000","last_peered":"2024-05-17T21:57:32.202822+0000","last_clean":"2024-05-17T21:57:32.202822+0000","last_became_active":"2024-05-17T21:57:29.305272+0000","last_became_peered":"2024-05-17T21:57:29.305272+0000","last_unstale":"2024-05-17T21:57:32.202822+0000","last_undegraded":"2024-05-17T21:57:32.202822+0000","last_fullsized":"2024-05-17T21:57:32.202822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:33.665928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:39.323 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T21:57:39.323 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-17T21:57:39.323 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-17T21:57:39.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-17T21:57:39.324 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-17T21:57:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 Processor -- start 2024-05-17T21:57:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 -- start start 2024-05-17T21:57:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1120072d70 0x7f112009bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11200d1950 con 0x7f112009c570 2024-05-17T21:57:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f11200d1670 con 0x7f1120072d70 2024-05-17T21:57:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f1126adf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1120071280 con 0x7f11200cf2e0 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f112487b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f112507c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1120072d70 0x7f112009bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f112487b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49250/0 (socket says 172.21.15.83:49250) 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.521+0000 7f112487b640 1 -- 172.21.15.83:0/2798122937 learned_addr learned my addr 172.21.15.83:0/2798122937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f112487b640 1 -- 172.21.15.83:0/2798122937 >> v2:172.21.15.155:3300/0 conn(0x7f1120072d70 msgr2=0x7f112009bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f112487b640 1 --2- 172.21.15.83:0/2798122937 >> v2:172.21.15.155:3300/0 conn(0x7f1120072d70 0x7f112009bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f112487b640 1 -- 172.21.15.83:0/2798122937 >> v1:172.21.15.155:6789/0 conn(0x7f112009c570 legacy=0x7f11200ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f112487b640 1 -- 172.21.15.83:0/2798122937 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1120071560 con 0x7f11200cf2e0 2024-05-17T21:57:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f112487b640 1 --2- 172.21.15.83:0/2798122937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1118009870 tx=0x7f111801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c94470f6366145 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f111f7fe640 1 -- 172.21.15.83:0/2798122937 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180b5070 con 0x7f11200cf2e0 2024-05-17T21:57:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f111f7fe640 1 -- 172.21.15.83:0/2798122937 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11180a8a50 con 0x7f11200cf2e0 2024-05-17T21:57:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f111f7fe640 1 -- 172.21.15.83:0/2798122937 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a8e20 con 0x7f11200cf2e0 2024-05-17T21:57:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 -- 172.21.15.83:0/2798122937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 msgr2=0x7f112009b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 --2- 172.21.15.83:0/2798122937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1118009870 tx=0x7f111801ed50 comp rx=0 tx=0).stop 2024-05-17T21:57:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 -- 172.21.15.83:0/2798122937 shutdown_connections 2024-05-17T21:57:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 --2- 172.21.15.83:0/2798122937 >> v2:172.21.15.155:3300/0 conn(0x7f1120072d70 0x7f112009bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 --2- 172.21.15.83:0/2798122937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11200cf2e0 0x7f112009b990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 -- 172.21.15.83:0/2798122937 >> 172.21.15.83:0/2798122937 conn(0x7f11200b5120 msgr2=0x7f11200b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 -- 172.21.15.83:0/2798122937 shutdown_connections 2024-05-17T21:57:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.525+0000 7f1126adf640 1 -- 172.21.15.83:0/2798122937 wait complete. 2024-05-17T21:57:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 Processor -- start 2024-05-17T21:57:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- start start 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 0x7f11200c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f112009c570 0x7f11201f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11200d1380 con 0x7f11200cf2e0 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11200d1200 con 0x7f1120072d70 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11200b8de0 con 0x7f112009c570 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39492/0 (socket says 172.21.15.83:39492) 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 -- 172.21.15.83:0/2188538884 learned_addr learned my addr 172.21.15.83:0/2188538884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f111ffff640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f112009c570 0x7f11201f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112487b640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 0x7f11200c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 -- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f112009c570 msgr2=0x7f11201f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f112009c570 0x7f11201f9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 -- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 msgr2=0x7f11200c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 0x7f11200c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 -- 172.21.15.83:0/2188538884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11200c78e0 con 0x7f11200cf2e0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112487b640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 0x7f11200c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f112507c640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f111000e7f0 tx=0x7f111000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110017070 con 0x7f11200cf2e0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11200c7bc0 con 0x7f11200cf2e0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111000c940 con 0x7f11200cf2e0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111000eea0 con 0x7f11200cf2e0 2024-05-17T21:57:39.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.529+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11200c8180 con 0x7f11200cf2e0 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11200c8380 con 0x7f11200cf2e0 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f111001a070 con 0x7f11200cf2e0 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f111d7fa640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 0x7f1100071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f111008f9d0 con 0x7f11200cf2e0 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f112487b640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 0x7f1100071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.533+0000 7f112487b640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 0x7f1100071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11180b2000 tx=0x7f11180aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.537+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f111005c820 con 0x7f11200cf2e0 2024-05-17T21:57:39.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.737+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f112009b990 con 0x7f11200cf2e0 2024-05-17T21:57:39.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.745+0000 7f111d7fa640 1 -- 172.21.15.83:0/2188538884 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179542 (secure 0 0 0) 0x7f111005c120 con 0x7f11200cf2e0 2024-05-17T21:57:39.739 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 msgr2=0x7f1100071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 0x7f1100071940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11180b2000 tx=0x7f11180aa070 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 msgr2=0x7f11201f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f111000e7f0 tx=0x7f111000ecb0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 shutdown_connections 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11200cf2e0 0x7f11201f9fa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f112009c570 0x7f11201f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f110006f430 0x7f1100071940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 --2- 172.21.15.83:0/2188538884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1120072d70 0x7f11200c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 >> 172.21.15.83:0/2188538884 conn(0x7f11200b5120 msgr2=0x7f1120160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 shutdown_connections 2024-05-17T21:57:39.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.749+0000 7f1126adf640 1 -- 172.21.15.83:0/2188538884 wait complete. 2024-05-17T21:57:39.759 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":380985748},{"type":"v1","addr":"172.21.15.83:6833","nonce":380985748}]},"active_addr":"172.21.15.83:6833/380985748","active_change":"2024-05-17T21:57:10.949875+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1951813468}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2833068380}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2056728465}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1953854626}]}]} 2024-05-17T21:57:39.763 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-17T21:57:39.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-17T21:57:39.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 Processor -- start 2024-05-17T21:57:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 -- start start 2024-05-17T21:57:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75dc144080 0x7f75dc072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc073780 con 0x7f75dc073370 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc15e790 con 0x7f75dc0d1040 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e3283640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75dc15ea40 con 0x7f75dc144080 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e081e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75dc144080 0x7f75dc072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e1820640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f75dc073370 0x7f75dc06f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33586/0 (socket says 172.21.15.83:33586) 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e1820640 1 -- 172.21.15.83:0/184732643 learned_addr learned my addr 172.21.15.83:0/184732643 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75d3fff640 1 -- 172.21.15.83:0/184732643 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2501748635 0 0) 0x7f75d40014a0 con 0x7f75dc073370 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75d3fff640 1 -- 172.21.15.83:0/184732643 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75c8003530 con 0x7f75dc073370 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e081e640 1 -- 172.21.15.83:0/184732643 >> v1:172.21.15.155:6789/0 conn(0x7f75dc0d1040 legacy=0x7f75dc15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e081e640 1 -- 172.21.15.83:0/184732643 >> v1:172.21.15.83:6789/0 conn(0x7f75dc073370 legacy=0x7f75dc06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.909+0000 7f75e081e640 1 -- 172.21.15.83:0/184732643 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc15ed20 con 0x7f75dc144080 2024-05-17T21:57:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e081e640 1 --2- 172.21.15.83:0/184732643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc144080 0x7f75dc072e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84b6be1fe053345 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75d3fff640 1 -- 172.21.15.83:0/184732643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0a7070 con 0x7f75dc144080 2024-05-17T21:57:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75d3fff640 1 -- 172.21.15.83:0/184732643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc0b0070 con 0x7f75dc144080 2024-05-17T21:57:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75d3fff640 1 -- 172.21.15.83:0/184732643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0ab870 con 0x7f75dc144080 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 -- 172.21.15.83:0/184732643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc144080 msgr2=0x7f75dc072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 --2- 172.21.15.83:0/184732643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc144080 0x7f75dc072e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01c470 comp rx=0 tx=0).stop 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 -- 172.21.15.83:0/184732643 shutdown_connections 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 --2- 172.21.15.83:0/184732643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc144080 0x7f75dc072e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 -- 172.21.15.83:0/184732643 >> 172.21.15.83:0/184732643 conn(0x7f75dc06cd90 msgr2=0x7f75dc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 -- 172.21.15.83:0/184732643 shutdown_connections 2024-05-17T21:57:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.913+0000 7f75e3283640 1 -- 172.21.15.83:0/184732643 wait complete. 2024-05-17T21:57:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 Processor -- start 2024-05-17T21:57:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- start start 2024-05-17T21:57:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75dc073370 0x7f75dc13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 0x7f75dc1fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc070650 con 0x7f75dc0d1040 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc144490 con 0x7f75dc073370 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0707d0 con 0x7f75dc144080 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39496/0 (socket says 172.21.15.83:39496) 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 -- 172.21.15.83:0/979098871 learned_addr learned my addr 172.21.15.83:0/979098871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e101f640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75dc073370 0x7f75dc13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e1820640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 0x7f75dc1fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 -- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 msgr2=0x7f75dc1fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 0x7f75dc1fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 -- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75dc073370 msgr2=0x7f75dc13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75dc073370 0x7f75dc13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 -- 172.21.15.83:0/979098871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc13e240 con 0x7f75dc0d1040 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e1820640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 0x7f75dc1fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e081e640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0abe60 con 0x7f75dc0d1040 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc004230 con 0x7f75dc0d1040 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc13e520 con 0x7f75dc0d1040 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003620 con 0x7f75dc0d1040 2024-05-17T21:57:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc13eae0 con 0x7f75dc0d1040 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.917+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc13dfb0 con 0x7f75dc0d1040 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.921+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75cc0037c0 con 0x7f75dc0d1040 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.921+0000 7f75d1ffb640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 0x7f75c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.921+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75cc129a60 con 0x7f75dc0d1040 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.921+0000 7f75e101f640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 0x7f75c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.921+0000 7f75e101f640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 0x7f75c0071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75d8001650 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:39.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:39.925+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f75cc0c6030 con 0x7f75dc0d1040 2024-05-17T21:57:40.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.073+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75dc072e30 con 0x7f75dc0d1040 2024-05-17T21:57:40.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75d1ffb640 1 -- 172.21.15.83:0/979098871 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f75cc016090 con 0x7f75dc0d1040 2024-05-17T21:57:40.068 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:40.068 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:40.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 msgr2=0x7f75c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 0x7f75c0071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75d8001650 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 msgr2=0x7f75dc1facb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0097c0 tx=0x7f75cc01acb0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 shutdown_connections 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75dc144080 0x7f75dc1fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc0d1040 0x7f75dc1facb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75c006f400 0x7f75c0071910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 --2- 172.21.15.83:0/979098871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75dc073370 0x7f75dc13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 >> 172.21.15.83:0/979098871 conn(0x7f75dc06cd90 msgr2=0x7f75dc13c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 shutdown_connections 2024-05-17T21:57:40.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.077+0000 7f75e3283640 1 -- 172.21.15.83:0/979098871 wait complete. 2024-05-17T21:57:40.089 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-17T21:57:40.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 Processor -- start 2024-05-17T21:57:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 -- start start 2024-05-17T21:57:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 0x7fe7f00703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7f01605a0 0x7fe7f01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f01465d0 con 0x7fe7f01461c0 2024-05-17T21:57:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f0160ef0 con 0x7fe7f01605a0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f79ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f006ef30 con 0x7fe7f006ffd0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f5756640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe7f01461c0 0x7fe7f015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33600/0 (socket says 172.21.15.83:33600) 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f5756640 1 -- 172.21.15.83:0/701010277 learned_addr learned my addr 172.21.15.83:0/701010277 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f4f55640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 0x7fe7f00703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110724052 0 0) 0x7fe7f01465d0 con 0x7fe7f01461c0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7f5f57640 1 --2- 172.21.15.83:0/701010277 >> v2:172.21.15.155:3300/0 conn(0x7fe7f01605a0 0x7fe7f01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7dc003530 con 0x7fe7f01461c0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 773010987 0 0) 0x7fe7dc003530 con 0x7fe7f01461c0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 msgr2=0x7fe7f00703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 0x7fe7f00703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 >> v2:172.21.15.155:3300/0 conn(0x7fe7f01605a0 msgr2=0x7fe7f01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 --2- 172.21.15.83:0/701010277 >> v2:172.21.15.155:3300/0 conn(0x7fe7f01605a0 0x7fe7f01609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0070920 con 0x7fe7f01461c0 2024-05-17T21:57:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe7ec001700 con 0x7fe7f01461c0 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 >> v1:172.21.15.83:6789/0 conn(0x7fe7f01461c0 legacy=0x7fe7f015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7dc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 0x7fe7dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7dc004d50 0x7fe7dc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0160ef0 con 0x7fe7dc003a10 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f006ef30 con 0x7fe7dc004d50 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.233+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7dc00bab0 con 0x7fe7f006ffd0 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7dc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 -- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 msgr2=0x7fe7dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f006ffd0 0x7fe7dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 -- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7dc004d50 msgr2=0x7fe7dc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7dc004d50 0x7fe7dc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 -- 172.21.15.83:0/701010277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc003530 con 0x7fe7dc003a10 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5756640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7dc003e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec007d20 tx=0x7fe7ec01ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe03247888964cab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007d60 con 0x7fe7dc003a10 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0ab070 con 0x7fe7dc003a10 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7e7fff640 1 -- 172.21.15.83:0/701010277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0058e0 con 0x7fe7dc003a10 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 -- 172.21.15.83:0/701010277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 msgr2=0x7fe7dc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/701010277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7dc003e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec007d20 tx=0x7fe7ec01ee60 comp rx=0 tx=0).stop 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f5f57640 1 -- 172.21.15.83:0/701010277 reap_dead start 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 -- 172.21.15.83:0/701010277 shutdown_connections 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 -- 172.21.15.83:0/701010277 >> 172.21.15.83:0/701010277 conn(0x7fe7f006cd90 msgr2=0x7fe7f00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 -- 172.21.15.83:0/701010277 shutdown_connections 2024-05-17T21:57:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.237+0000 7fe7f79ba640 1 -- 172.21.15.83:0/701010277 wait complete. 2024-05-17T21:57:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 Processor -- start 2024-05-17T21:57:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- start start 2024-05-17T21:57:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f0141580 0x7fe7f013d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f00733d0 0x7fe7f00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015e880 con 0x7fe7dc003a10 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f006fab0 con 0x7fe7f00733d0 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00d10a0 con 0x7fe7f0141580 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39520/0 (socket says 172.21.15.83:39520) 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 -- 172.21.15.83:0/3036899639 learned_addr learned my addr 172.21.15.83:0/3036899639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5f57640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f00733d0 0x7fe7f00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f4f55640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f0141580 0x7fe7f013d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 -- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f0141580 msgr2=0x7fe7f013d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f0141580 0x7fe7f013d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 -- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f00733d0 msgr2=0x7fe7f00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f00733d0 0x7fe7f00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 -- 172.21.15.83:0/3036899639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0073d20 con 0x7fe7dc003a10 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f5756640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec025430 tx=0x7fe7dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007d60 con 0x7fe7dc003a10 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0a7a30 con 0x7fe7dc003a10 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0047f0 con 0x7fe7dc003a10 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f0141d60 con 0x7fe7dc003a10 2024-05-17T21:57:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.241+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f0142320 con 0x7fe7dc003a10 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f01609b0 con 0x7fe7dc003a10 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7ec004990 con 0x7fe7dc003a10 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7e5ffb640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 0x7fe7d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec12abe0 con 0x7fe7dc003a10 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7f4f55640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 0x7fe7d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.245+0000 7fe7f4f55640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 0x7fe7d4071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0145a70 tx=0x7fe7e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.249+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe7ec0c1060 con 0x7fe7dc003a10 2024-05-17T21:57:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.401+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7f00b1880 con 0x7fe7dc003a10 2024-05-17T21:57:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.401+0000 7fe7e5ffb640 1 -- 172.21.15.83:0/3036899639 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fe7f0073d20 con 0x7fe7dc003a10 2024-05-17T21:57:40.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:40.395 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 msgr2=0x7fe7d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 0x7fe7d4071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0145a70 tx=0x7fe7e0008040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 msgr2=0x7fe7f0158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec025430 tx=0x7fe7dc005840 comp rx=0 tx=0).stop 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 shutdown_connections 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe7d406f430 0x7fe7d4071940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f00733d0 0x7fe7f00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7f0141580 0x7fe7f013d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 --2- 172.21.15.83:0/3036899639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7dc003a10 0x7fe7f0158f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 >> 172.21.15.83:0/3036899639 conn(0x7fe7f006cd90 msgr2=0x7fe7f015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 shutdown_connections 2024-05-17T21:57:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.405+0000 7fe7f79ba640 1 -- 172.21.15.83:0/3036899639 wait complete. 2024-05-17T21:57:40.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-17T21:57:40.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-17T21:57:40.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-17T21:57:40.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-17T21:57:40.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-17T21:57:40.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-17T21:57:40.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-17T21:57:40.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-17T21:57:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 Processor -- start 2024-05-17T21:57:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 -- start start 2024-05-17T21:57:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f35db1e9640 1 Processor -- start 2024-05-17T21:57:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f35db1e9640 1 -- start start 2024-05-17T21:57:40.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7fcf575fb640 1 Processor -- start 2024-05-17T21:57:40.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7fcf575fb640 1 -- start start 2024-05-17T21:57:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 0x7f9b380739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b381681e0 0x7f9b38164f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b380b4f70 con 0x7f9b38167dd0 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9b380b5250 con 0x7f9b381681e0 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3e842640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b380b5530 con 0x7f9b380b4990 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b377fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9b38167dd0 0x7f9b3806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33606/0 (socket says 172.21.15.83:33606) 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b377fe640 1 -- 172.21.15.83:0/1215797270 learned_addr learned my addr 172.21.15.83:0/1215797270 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f7858c42640 1 Processor -- start 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7fdbeeb95640 1 Processor -- start 2024-05-17T21:57:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7fdbeeb95640 1 -- start start 2024-05-17T21:57:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b37fff640 1 --2- 172.21.15.83:0/1215797270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 0x7f9b380739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f9b3cddf640 1 --2- 172.21.15.83:0/1215797270 >> v2:172.21.15.155:3300/0 conn(0x7f9b381681e0 0x7f9b38164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.645+0000 7f7858c42640 1 -- start start 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35db1e9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35d40b4990 0x7f35d40739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35db1e9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 0x7f35d41681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35db1e9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d40b4f70 con 0x7f35d406eea0 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35db1e9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40b5250 con 0x7f35d40b4990 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35db1e9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f35d40b5530 con 0x7f35d4167dd0 2024-05-17T21:57:40.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b36ffd640 1 -- 172.21.15.83:0/1215797270 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929921519 0 0) 0x7f9b380b4f70 con 0x7f9b38167dd0 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b36ffd640 1 -- 172.21.15.83:0/1215797270 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b20003530 con 0x7f9b38167dd0 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b37fff640 1 -- 172.21.15.83:0/1215797270 >> v2:172.21.15.155:3300/0 conn(0x7f9b381681e0 msgr2=0x7f9b38164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b37fff640 1 --2- 172.21.15.83:0/1215797270 >> v2:172.21.15.155:3300/0 conn(0x7f9b381681e0 0x7f9b38164f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b37fff640 1 -- 172.21.15.83:0/1215797270 >> v1:172.21.15.83:6789/0 conn(0x7f9b38167dd0 legacy=0x7f9b3806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f9b37fff640 1 -- 172.21.15.83:0/1215797270 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b380b5810 con 0x7f9b380b4990 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d9786640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f35d406eea0 0x7f35d406f2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33614/0 (socket says 172.21.15.83:33614) 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d9786640 1 -- 172.21.15.83:0/2620817313 learned_addr learned my addr 172.21.15.83:0/2620817313 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d8f85640 1 --2- 172.21.15.83:0/2620817313 >> v2:172.21.15.83:3300/0 conn(0x7f35d40b4990 0x7f35d40739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d37fe640 1 -- 172.21.15.83:0/2620817313 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804930653 0 0) 0x7f35d40b4f70 con 0x7f35d406eea0 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d37fe640 1 -- 172.21.15.83:0/2620817313 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35bc003530 con 0x7f35d406eea0 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf575fb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf575fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf500af070 con 0x7fcf5015fc40 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf575fb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf500b58a0 con 0x7fcf50072db0 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf575fb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcf500af660 con 0x7fcf50168990 2024-05-17T21:57:40.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbeeb95640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe8164490 0x7fdbe81648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbeeb95640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe8070bf0 con 0x7fdbe8161810 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbeeb95640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbe8164ec0 con 0x7fdbe80b33c0 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbeeb95640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe81651a0 con 0x7fdbe8164490 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d3fff640 1 --2- 172.21.15.83:0/2620817313 >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 0x7f35d41681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49318/0 (socket says 172.21.15.83:49318) 2024-05-17T21:57:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 -- 172.21.15.83:0/445119667 learned_addr learned my addr 172.21.15.83:0/445119667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbed132640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fdbe80b33c0 0x7fdbe80b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48724/0 (socket says 172.21.15.83:48724) 2024-05-17T21:57:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbed132640 1 -- 172.21.15.83:0/2273597904 learned_addr learned my addr 172.21.15.83:0/2273597904 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe7fff640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe8164490 0x7fdbe81648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fed8f3bf640 1 Processor -- start 2024-05-17T21:57:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fd67c1bc640 1 Processor -- start 2024-05-17T21:57:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025103185 0 0) 0x7fdbe8164ec0 con 0x7fdbe80b33c0 2024-05-17T21:57:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbd8003530 con 0x7fdbe80b33c0 2024-05-17T21:57:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f7858c42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 -- 172.21.15.83:0/445119667 >> v1:172.21.15.155:6789/0 conn(0x7fcf50072db0 legacy=0x7fcf500731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 -- 172.21.15.83:0/445119667 >> v1:172.21.15.83:6789/0 conn(0x7fcf5015fc40 legacy=0x7fcf500b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fcf55b98640 1 -- 172.21.15.83:0/445119667 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf500b5b80 con 0x7fcf50168990 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f7858c42640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78540b5a10 con 0x7f78540b4cb0 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f7858c42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78540b5cf0 con 0x7f78540b33f0 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f7858c42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78540b57d0 con 0x7f785415d820 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d8f85640 1 -- 172.21.15.83:0/2620817313 >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 msgr2=0x7f35d41681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d8f85640 1 --2- 172.21.15.83:0/2620817313 >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 0x7f35d41681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d8f85640 1 -- 172.21.15.83:0/2620817313 >> v1:172.21.15.83:6789/0 conn(0x7f35d406eea0 legacy=0x7f35d406f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d8f85640 1 -- 172.21.15.83:0/2620817313 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40b5810 con 0x7f35d40b4990 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f35d3fff640 1 --2- 172.21.15.83:0/2620817313 >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 0x7f35d41681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fd67c1bc640 1 -- start start 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1508137428 0 0) 0x7fdbd8003530 con 0x7fdbe80b33c0 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe8164490 msgr2=0x7fdbe81648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe8164490 0x7fdbe81648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 >> v1:172.21.15.83:6789/0 conn(0x7fdbe8161810 legacy=0x7fdbe80704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe806fce0 con 0x7fdbe80b33c0 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fdbe0001a80 con 0x7fdbe80b33c0 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 >> v1:172.21.15.155:6789/0 conn(0x7fdbe80b33c0 legacy=0x7fdbe80b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8003a10 0x7fdbd8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd8004810 0x7fdbd8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbd80052b0 0x7fdbd800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81651a0 con 0x7fdbd8003a10 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd800c150 con 0x7fdbd8004810 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd800c350 con 0x7fdbd80052b0 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f785259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f785259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49344/0 (socket says 172.21.15.83:49344) 2024-05-17T21:57:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f785259c640 1 -- 172.21.15.83:0/2727770059 learned_addr learned my addr 172.21.15.83:0/2727770059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7fed8f3bf640 1 -- start start 2024-05-17T21:57:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f84c39d1640 1 Processor -- start 2024-05-17T21:57:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.649+0000 7f84c39d1640 1 -- start start 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8003a10 0x7fdbd8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 -- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbd80052b0 msgr2=0x7fdbd800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbd80052b0 0x7fdbd800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 -- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd8004810 msgr2=0x7fdbd8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd8004810 0x7fdbd8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 -- 172.21.15.83:0/2273597904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe806fce0 con 0x7fdbd8003a10 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8f3bf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fed880b5bd0 0x7fed8815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8f3bf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fed8815d810 0x7fed8806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8f3bf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed880b53e0 con 0x7fed880b44f0 2024-05-17T21:57:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8f3bf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fed880b4900 con 0x7fed880b5bd0 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8f3bf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fed880b4b40 con 0x7fed8815d810 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd67c1bc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd67c1bc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd66c00b8a0 0x7fd66c00bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd67c1bc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66c037540 con 0x7fd66c03a170 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd67c1bc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c048540 con 0x7fd66c00b8a0 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd67c1bc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c048260 con 0x7fd66c0073e0 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8d15b640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fed880b44f0 0x7fed880b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48744/0 (socket says 172.21.15.83:48744) 2024-05-17T21:57:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8d15b640 1 -- 172.21.15.83:0/144191810 learned_addr learned my addr 172.21.15.83:0/144191810 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b37fff640 1 --2- 172.21.15.83:0/1215797270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 0x7f9b380739f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b280097c0 tx=0x7f9b2801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6611fd3eb607e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49358/0 (socket says 172.21.15.83:49358) 2024-05-17T21:57:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 -- 172.21.15.83:0/1892303598 learned_addr learned my addr 172.21.15.83:0/1892303598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c39d1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84bc15fdf0 0x7f84bc0d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c39d1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84bc0d0e90 con 0x7f84bc069a50 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c39d1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84bc0d1170 con 0x7f84bc0d0080 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c39d1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f84bc13ee80 con 0x7f84bc15fdf0 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8c95a640 1 --2- 172.21.15.83:0/144191810 >> v2:172.21.15.83:3300/0 conn(0x7fed880b5bd0 0x7fed8815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8d95c640 1 --2- 172.21.15.83:0/144191810 >> v2:172.21.15.155:3300/0 conn(0x7fed8815d810 0x7fed8806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c0f6c640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f84bc0d0080 0x7f84bc15f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48754/0 (socket says 172.21.15.83:48754) 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c0f6c640 1 -- 172.21.15.83:0/3447700483 learned_addr learned my addr 172.21.15.83:0/3447700483 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf55b98640 1 --2- 172.21.15.83:0/445119667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf400098a0 tx=0x7fcf4001c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255f520094849f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f785259c640 1 -- 172.21.15.83:0/2727770059 >> v1:172.21.15.155:6789/0 conn(0x7f78540b33f0 legacy=0x7f78540b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f785259c640 1 -- 172.21.15.83:0/2727770059 >> v1:172.21.15.83:6789/0 conn(0x7f78540b4cb0 legacy=0x7f78540b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f785259c640 1 -- 172.21.15.83:0/2727770059 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78540af160 con 0x7f785415d820 2024-05-17T21:57:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/3447700483 >> v2:172.21.15.155:3300/0 conn(0x7f84bc15fdf0 0x7f84bc0d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35d8f85640 1 --2- 172.21.15.83:0/2620817313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 0x7f35d40739f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35d40b09c0 tx=0x7f35c401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fbe9052111115 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b36ffd640 1 -- 172.21.15.83:0/1215797270 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280b0070 con 0x7f9b380b4990 2024-05-17T21:57:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b36ffd640 1 -- 172.21.15.83:0/1215797270 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b280ab600 con 0x7f9b380b4990 2024-05-17T21:57:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b36ffd640 1 -- 172.21.15.83:0/1215797270 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280ab9d0 con 0x7f9b380b4990 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 -- 172.21.15.83:0/1892303598 >> v2:172.21.15.83:3300/0 conn(0x7fd66c00b8a0 msgr2=0x7fd66c00bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 --2- 172.21.15.83:0/1892303598 >> v2:172.21.15.83:3300/0 conn(0x7fd66c00b8a0 0x7fd66c00bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 -- 172.21.15.83:0/1892303598 >> v1:172.21.15.83:6789/0 conn(0x7fd66c03a170 legacy=0x7fd66c006cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fd679757640 1 -- 172.21.15.83:0/1892303598 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c037260 con 0x7fd66c0073e0 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35d37fe640 1 -- 172.21.15.83:0/2620817313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40b0070 con 0x7f35d40b4990 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35d37fe640 1 -- 172.21.15.83:0/2620817313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c40ab600 con 0x7f35d40b4990 2024-05-17T21:57:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35d37fe640 1 -- 172.21.15.83:0/2620817313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40701d0 con 0x7f35d40b4990 2024-05-17T21:57:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107627032 0 0) 0x7fed880b53e0 con 0x7fed880b44f0 2024-05-17T21:57:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed74003530 con 0x7fed880b44f0 2024-05-17T21:57:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf47fff640 1 -- 172.21.15.83:0/445119667 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf400b2070 con 0x7fcf50168990 2024-05-17T21:57:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84affff640 1 -- 172.21.15.83:0/3447700483 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116245823 0 0) 0x7f84bc0d1170 con 0x7f84bc0d0080 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1528191552 0 0) 0x7fed74003530 con 0x7fed880b44f0 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 >> v2:172.21.15.155:3300/0 conn(0x7fed8815d810 msgr2=0x7fed8806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 --2- 172.21.15.83:0/144191810 >> v2:172.21.15.155:3300/0 conn(0x7fed8815d810 0x7fed8806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 >> v2:172.21.15.83:3300/0 conn(0x7fed880b5bd0 msgr2=0x7fed8815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 --2- 172.21.15.83:0/144191810 >> v2:172.21.15.83:3300/0 conn(0x7fed880b5bd0 0x7fed8815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880b4e20 con 0x7fed880b44f0 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fed78001700 con 0x7fed880b44f0 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 >> v1:172.21.15.155:6789/0 conn(0x7fed880b44f0 legacy=0x7fed880b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed74003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed74004810 0x7fed74004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed74005280 0x7fed7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b4900 con 0x7fed74004810 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b4b40 con 0x7fed74005280 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed7400c290 con 0x7fed74003a60 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c1f6e640 1 -- 172.21.15.83:0/3447700483 >> v1:172.21.15.155:6789/0 conn(0x7f84bc0d0080 legacy=0x7f84bc15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c1f6e640 1 -- 172.21.15.83:0/3447700483 >> v1:172.21.15.83:6789/0 conn(0x7f84bc069a50 legacy=0x7f84bc13e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf47fff640 1 -- 172.21.15.83:0/445119667 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf400ad680 con 0x7fcf50168990 2024-05-17T21:57:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf47fff640 1 -- 172.21.15.83:0/445119667 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf400ada50 con 0x7fcf50168990 2024-05-17T21:57:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f84c1f6e640 1 -- 172.21.15.83:0/3447700483 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc162830 con 0x7f84bc15fdf0 2024-05-17T21:57:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b3e842640 1 -- 172.21.15.83:0/1215797270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 msgr2=0x7f9b380739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f9b3e842640 1 --2- 172.21.15.83:0/1215797270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 0x7f9b380739f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b280097c0 tx=0x7f9b2801c4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f785259c640 1 --2- 172.21.15.83:0/2727770059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f784c009e80 tx=0x7f784c01ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380b138fde371469 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbec931640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8003a10 0x7fdbd8003e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4001770 tx=0x7fdbd401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf6fc242c6e0e0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35db1e9640 1 -- 172.21.15.83:0/2620817313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 msgr2=0x7f35d40739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7f35db1e9640 1 --2- 172.21.15.83:0/2620817313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 0x7f35d40739f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35d40b09c0 tx=0x7f35c401c3f0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fed8d15b640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed74003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf575fb640 1 -- 172.21.15.83:0/445119667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 msgr2=0x7fcf5016adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf575fb640 1 --2- 172.21.15.83:0/445119667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf400098a0 tx=0x7fcf4001c580 comp rx=0 tx=0).stop 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf575fb640 1 -- 172.21.15.83:0/445119667 shutdown_connections 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf575fb640 1 --2- 172.21.15.83:0/445119667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5016adb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fcf575fb640 1 -- 172.21.15.83:0/445119667 >> 172.21.15.83:0/445119667 conn(0x7fcf500d1440 msgr2=0x7fcf500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40013c0 con 0x7fdbd8003a10 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd40052f0 con 0x7fdbd8003a10 2024-05-17T21:57:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.653+0000 7fdbe77fe640 1 -- 172.21.15.83:0/2273597904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4003840 con 0x7fdbd8003a10 2024-05-17T21:57:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fcf575fb640 1 -- 172.21.15.83:0/445119667 shutdown_connections 2024-05-17T21:57:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f785159a640 1 -- 172.21.15.83:0/2727770059 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0a58b0 con 0x7f785415d820 2024-05-17T21:57:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f785159a640 1 -- 172.21.15.83:0/2727770059 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c0a5a50 con 0x7f785415d820 2024-05-17T21:57:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f785159a640 1 -- 172.21.15.83:0/2727770059 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c0a5e20 con 0x7f785415d820 2024-05-17T21:57:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 -- 172.21.15.83:0/1215797270 shutdown_connections 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 --2- 172.21.15.83:0/1215797270 >> v2:172.21.15.155:3300/0 conn(0x7f9b381681e0 0x7f9b38164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 --2- 172.21.15.83:0/1215797270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b380b4990 0x7f9b380739f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 -- 172.21.15.83:0/1215797270 >> 172.21.15.83:0/1215797270 conn(0x7f9b380d1440 msgr2=0x7f9b380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 -- 172.21.15.83:0/1215797270 shutdown_connections 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 -- 172.21.15.83:0/1215797270 wait complete. 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 -- 172.21.15.83:0/2727770059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 msgr2=0x7f785406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 --2- 172.21.15.83:0/2727770059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f784c009e80 tx=0x7f784c01ca40 comp rx=0 tx=0).stop 2024-05-17T21:57:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- 172.21.15.83:0/2273597904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8003a10 msgr2=0x7fdbd8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 --2- 172.21.15.83:0/2273597904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbd8003a10 0x7fdbd8003e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4001770 tx=0x7fdbd401c3d0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbed132640 1 -- 172.21.15.83:0/2273597904 reap_dead start 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- 172.21.15.83:0/2273597904 shutdown_connections 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- 172.21.15.83:0/2273597904 >> 172.21.15.83:0/2273597904 conn(0x7fdbe806c790 msgr2=0x7fdbe806cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- 172.21.15.83:0/2273597904 shutdown_connections 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 -- 172.21.15.83:0/2620817313 shutdown_connections 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 --2- 172.21.15.83:0/2620817313 >> v2:172.21.15.155:3300/0 conn(0x7f35d4167dd0 0x7f35d41681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 --2- 172.21.15.83:0/2620817313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 0x7f35d40739f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 -- 172.21.15.83:0/2620817313 >> 172.21.15.83:0/2620817313 conn(0x7f35d40d1440 msgr2=0x7f35d40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 -- 172.21.15.83:0/2620817313 shutdown_connections 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8c95a640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed74004810 0x7fed74004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d95c640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed74005280 0x7fed7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d15b640 1 -- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed74005280 msgr2=0x7fed7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d15b640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed74005280 0x7fed7400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d15b640 1 -- 172.21.15.83:0/144191810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed74004810 msgr2=0x7fed74004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d15b640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed74004810 0x7fed74004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d15b640 1 -- 172.21.15.83:0/144191810 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880b4e20 con 0x7fed74003a60 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d95c640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed74005280 0x7fed7400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8d95c640 1 -- 172.21.15.83:0/144191810 reap_dead start 2024-05-17T21:57:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fed8c95a640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed74004810 0x7fed74004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:40.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fcf575fb640 1 -- 172.21.15.83:0/445119667 wait complete. 2024-05-17T21:57:40.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/3447700483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc15fdf0 0x7f84bc0d0490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009d90 tx=0x7f84b801c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9202a3228e4030c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- 172.21.15.83:0/2273597904 wait complete. 2024-05-17T21:57:40.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 -- 172.21.15.83:0/2620817313 wait complete. 2024-05-17T21:57:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84affff640 1 -- 172.21.15.83:0/3447700483 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80a66d0 con 0x7f84bc15fdf0 2024-05-17T21:57:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84affff640 1 -- 172.21.15.83:0/3447700483 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b80a6870 con 0x7f84bc15fdf0 2024-05-17T21:57:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84affff640 1 -- 172.21.15.83:0/3447700483 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80a6c40 con 0x7f84bc15fdf0 2024-05-17T21:57:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 Processor -- start 2024-05-17T21:57:40.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f9b3e842640 1 -- start start 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 -- 172.21.15.83:0/2727770059 shutdown_connections 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 --2- 172.21.15.83:0/2727770059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785406f3b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 -- 172.21.15.83:0/2727770059 >> 172.21.15.83:0/2727770059 conn(0x7f785406c790 msgr2=0x7f785406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 -- 172.21.15.83:0/2727770059 shutdown_connections 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f7858c42640 1 -- 172.21.15.83:0/2727770059 wait complete. 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 Processor -- start 2024-05-17T21:57:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fdbeeb95640 1 -- start start 2024-05-17T21:57:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f35db1e9640 1 Processor -- start 2024-05-17T21:57:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 -- start start 2024-05-17T21:57:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fd679757640 1 --2- 172.21.15.83:0/1892303598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd67001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f7747d0023b8a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fcf575fb640 1 Processor -- start 2024-05-17T21:57:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7fcf575fb640 1 -- start start 2024-05-17T21:57:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fd678f56640 1 -- 172.21.15.83:0/1892303598 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700b0070 con 0x7fd66c0073e0 2024-05-17T21:57:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fd678f56640 1 -- 172.21.15.83:0/1892303598 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700ab600 con 0x7fd66c0073e0 2024-05-17T21:57:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fd678f56640 1 -- 172.21.15.83:0/1892303598 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700ab9d0 con 0x7fd66c0073e0 2024-05-17T21:57:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8d15b640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed74003ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed7800d320 tx=0x7fed7801d200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871eda8ed8c46ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84c39d1640 1 -- 172.21.15.83:0/3447700483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc15fdf0 msgr2=0x7f84bc0d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.657+0000 7f84c39d1640 1 --2- 172.21.15.83:0/3447700483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc15fdf0 0x7f84bc0d0490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009d90 tx=0x7f84b801c770 comp rx=0 tx=0).stop 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 -- 172.21.15.83:0/3447700483 shutdown_connections 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 --2- 172.21.15.83:0/3447700483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc15fdf0 0x7f84bc0d0490 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 -- 172.21.15.83:0/3447700483 >> 172.21.15.83:0/3447700483 conn(0x7f84bc0b3600 msgr2=0x7f84bc0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 -- 172.21.15.83:0/3447700483 shutdown_connections 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 -- 172.21.15.83:0/3447700483 wait complete. 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780b1070 con 0x7fed74003a60 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed780ac640 con 0x7fed74003a60 2024-05-17T21:57:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed7ffff640 1 -- 172.21.15.83:0/144191810 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78007050 con 0x7fed74003a60 2024-05-17T21:57:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f7858c42640 1 Processor -- start 2024-05-17T21:57:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50072db0 0x7fcf50201200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b3e842640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 0x7f9b38159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 0x7fcf500b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 -- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 msgr2=0x7fed74003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed74003ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed7800d320 tx=0x7fed7801d200 comp rx=0 tx=0).stop 2024-05-17T21:57:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b37fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 0x7f9b38159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b37fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 0x7f9b38159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39564/0 (socket says 172.21.15.83:39564) 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 0x7fcf500b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 0x7fcf500b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39566/0 (socket says 172.21.15.83:39566) 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf55397640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50072db0 0x7fcf50201200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf55397640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50072db0 0x7fcf50201200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37790/0 (socket says 172.21.15.83:37790) 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b37fff640 1 -- 172.21.15.83:0/738127275 learned_addr learned my addr 172.21.15.83:0/738127275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 -- 172.21.15.83:0/1809958656 learned_addr learned my addr 172.21.15.83:0/1809958656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b3e842640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b38167dd0 0x7f9b380b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 0x7fdbe80aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500b54f0 con 0x7fcf5015fc40 2024-05-17T21:57:40.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b3e842640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b381681e0 0x7f9b38162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500b0b00 con 0x7fcf50072db0 2024-05-17T21:57:40.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcf50159fc0 con 0x7fcf50168990 2024-05-17T21:57:40.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe8152ba0 0x7fdbe8200cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3806ff40 con 0x7f9b380b4990 2024-05-17T21:57:40.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 0x7f35d415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf55b98640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 0x7fdbe80aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 0x7fdbe80aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39572/0 (socket says 172.21.15.83:39572) 2024-05-17T21:57:40.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 0x7f35d4164160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fd67c1bc640 1 -- 172.21.15.83:0/1892303598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 msgr2=0x7fd66c00b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/1892303598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd67001c3f0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 -- 172.21.15.83:0/1035545994 learned_addr learned my addr 172.21.15.83:0/1035545994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe82011f0 0x7fdbe81655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d4167dd0 0x7f35d4160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 Processor -- start 2024-05-17T21:57:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8161e30 con 0x7fdbe8201870 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 0x7f35d415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4170f10 con 0x7f35d40b4990 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbe7fff640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe8152ba0 0x7fdbe8200cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 0x7f35d415d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49390/0 (socket says 172.21.15.83:49390) 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 -- 172.21.15.83:0/4014949206 learned_addr learned my addr 172.21.15.83:0/4014949206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4170d90 con 0x7f35d4167dd0 2024-05-17T21:57:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe806f6f0 con 0x7fdbe8152ba0 2024-05-17T21:57:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35d4171090 con 0x7f35d406eea0 2024-05-17T21:57:40.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbe816fbf0 con 0x7fdbe82011f0 2024-05-17T21:57:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b377fe640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b38167dd0 0x7f9b380b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f9b3cddf640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b381681e0 0x7f9b38162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbed132640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe82011f0 0x7fdbe81655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 -- 172.21.15.83:0/144191810 shutdown_connections 2024-05-17T21:57:40.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 --2- 172.21.15.83:0/144191810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed74003ed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 -- 172.21.15.83:0/144191810 >> 172.21.15.83:0/144191810 conn(0x7fed8806c790 msgr2=0x7fed8806cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 -- 172.21.15.83:0/144191810 shutdown_connections 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fed8f3bf640 1 -- 172.21.15.83:0/144191810 wait complete. 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 -- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 msgr2=0x7fcf5015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 -- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50072db0 msgr2=0x7fcf50201200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf50072db0 0x7fcf50201200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5015a750 con 0x7fcf5015fc40 2024-05-17T21:57:40.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf55b98640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf50168990 0x7fcf5015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:57:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d9786640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d4167dd0 0x7f35d4160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fcf54b96640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 0x7fcf500b10a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c420 tx=0x7fcf4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 -- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d4167dd0 msgr2=0x7f35d4160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d4167dd0 0x7f35d4160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 -- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 msgr2=0x7f35d4164160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40b4990 0x7f35d4164160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f35d8f85640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41fd090 con 0x7f35d406eea0 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f7858c42640 1 -- start start 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 -- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe82011f0 msgr2=0x7fdbe81655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbe82011f0 0x7fdbe81655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 -- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe8152ba0 msgr2=0x7fdbe8200cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe8152ba0 0x7fdbe8200cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7fdbec931640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe8201600 con 0x7fdbe8201870 2024-05-17T21:57:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 -- 172.21.15.83:0/1892303598 shutdown_connections 2024-05-17T21:57:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/1892303598 >> v2:172.21.15.83:3300/0 conn(0x7fd66c00b8a0 0x7fd66c00bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/1892303598 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c0073e0 0x7fd66c00b360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 -- 172.21.15.83:0/1892303598 >> 172.21.15.83:0/1892303598 conn(0x7fd66c01dfc0 msgr2=0x7fd66c039a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c017070 con 0x7fcf5015fc40 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4c0052d0 con 0x7fcf5015fc40 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0054c0 con 0x7fcf5015fc40 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5015aa30 con 0x7fcf5015fc40 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 -- 172.21.15.83:0/1892303598 shutdown_connections 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 -- 172.21.15.83:0/1892303598 wait complete. 2024-05-17T21:57:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35d8f85640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 0x7f35d415d3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f35c401c8d0 tx=0x7f35c40191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbec931640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 0x7fdbe80aa310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4000c10 tx=0x7fdbd8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5015aff0 con 0x7fcf5015fc40 2024-05-17T21:57:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40b8020 con 0x7f35d406eea0 2024-05-17T21:57:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c40b4600 con 0x7f35d406eea0 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40b0040 con 0x7f35d406eea0 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40013c0 con 0x7fdbe8201870 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd40054a0 con 0x7fdbe8201870 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4003e00 con 0x7fdbe8201870 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d41fd630 con 0x7f35d406eea0 2024-05-17T21:57:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41fdbf0 con 0x7f35d406eea0 2024-05-17T21:57:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.661+0000 7f84c39d1640 1 -- start start 2024-05-17T21:57:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35d41fdfd0 con 0x7f35d406eea0 2024-05-17T21:57:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe80aa850 con 0x7fdbe8201870 2024-05-17T21:57:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81682e0 con 0x7fdbe8201870 2024-05-17T21:57:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdbc8001040 con 0x7fdbe8201870 2024-05-17T21:57:40.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c39d1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc069a50 0x7f84bc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf30001040 con 0x7fcf5015fc40 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c39d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84bc0d0080 0x7f84bc1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c176d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc069a50 0x7f84bc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c176d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc069a50 0x7f84bc155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37812/0 (socket says 172.21.15.83:37812) 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c0f6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84bc0d0080 0x7f84bc1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c0f6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84bc0d0080 0x7f84bc1379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39592/0 (socket says 172.21.15.83:39592) 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c176d640 1 -- 172.21.15.83:0/1032291215 learned_addr learned my addr 172.21.15.83:0/1032291215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f84c39d1640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84bc155600 0x7f84bc13d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78540b33f0 0x7f7854160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 0x7f7854150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78540b07e0 con 0x7f78540b4cb0 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f78540b0660 con 0x7f78540b33f0 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7858c42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f78540b0960 con 0x7f785415d820 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 0x7f7854150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 0x7f7854150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39594/0 (socket says 172.21.15.83:39594) 2024-05-17T21:57:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 -- 172.21.15.83:0/1559471027 learned_addr learned my addr 172.21.15.83:0/1559471027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3806fdc0 con 0x7f9b381681e0 2024-05-17T21:57:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b380700c0 con 0x7f9b38167dd0 2024-05-17T21:57:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 Processor -- start 2024-05-17T21:57:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fd67c1bc640 1 -- start start 2024-05-17T21:57:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f785259c640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78540b33f0 0x7f7854160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 -- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 msgr2=0x7f785415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7852d9d640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fed8f3bf640 1 Processor -- start 2024-05-17T21:57:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7fed8f3bf640 1 -- start start 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b37fff640 1 -- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b38167dd0 msgr2=0x7f9b380b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b37fff640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b38167dd0 0x7f9b380b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b37fff640 1 -- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b381681e0 msgr2=0x7f9b38162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b37fff640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b381681e0 0x7f9b38162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f9b37fff640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b380b2060 con 0x7f9b380b4990 2024-05-17T21:57:40.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 -- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78540b33f0 msgr2=0x7f7854160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78540b33f0 0x7f7854160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.665+0000 7f7851d9b640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785414e520 con 0x7f78540b4cb0 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f785259c640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78540b33f0 0x7f7854160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b377fe640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b38167dd0 0x7f9b380b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b37fff640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 0x7f9b38159830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b280190b0 tx=0x7f9b28019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f7852d9d640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f785415d820 0x7f785415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc16da00 con 0x7f84bc0d0080 2024-05-17T21:57:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc16d880 con 0x7f84bc069a50 2024-05-17T21:57:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f84bc16db80 con 0x7f84bc155600 2024-05-17T21:57:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84bc155600 0x7f84bc13d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35c40bb020 con 0x7f35d406eea0 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdbd40bf430 con 0x7fdbe8201870 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdbcc06f4d0 0x7fdbcc0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbd41288d0 con 0x7fdbe8201870 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c0073e0 0x7fd66c0e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7fdbcc077740 0x7fdbcc079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdbd40b9ec0 con 0x7fdbcc077740 2024-05-17T21:57:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 0x7fd66c04a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdbd4128cc0 con 0x7fdbe8201870 2024-05-17T21:57:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c03a170 0x7fd66c0329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b3cddf640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b381681e0 0x7f9b38162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed88146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f35c006f400 0x7f35c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c053430 con 0x7fd66c03a170 2024-05-17T21:57:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0532b0 con 0x7fd66c0073e0 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c0535b0 con 0x7fd66c00b8a0 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 0x7fd66c04a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35c41296a0 con 0x7f35d406eea0 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 0x7fd66c04a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49434/0 (socket says 172.21.15.83:49434) 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 -- 172.21.15.83:0/2891371965 learned_addr learned my addr 172.21.15.83:0/2891371965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679f58640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c0073e0 0x7fd66c0e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed880b1650 0x7fed881495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f35c0077650 0x7f35c0079aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d3fff640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f35c006f400 0x7f35c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35c40c5e70 con 0x7f35c0077650 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f35c4129b30 con 0x7f35d406eea0 2024-05-17T21:57:40.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 -- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc069a50 msgr2=0x7f84bc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc069a50 0x7f84bc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 -- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84bc0d0080 msgr2=0x7f84bc1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84bc0d0080 0x7f84bc1379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc154560 con 0x7f84bc155600 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed881478b0 0x7fed88149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b6160 con 0x7fed880b1650 2024-05-17T21:57:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b5fe0 con 0x7fed881478b0 2024-05-17T21:57:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8f3bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fed880b3b80 con 0x7fed74003a60 2024-05-17T21:57:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f7851d9b640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 0x7f7854150d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f784800c430 tx=0x7f784800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed88146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed88146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49436/0 (socket says 172.21.15.83:49436) 2024-05-17T21:57:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 -- 172.21.15.83:0/1018896181 learned_addr learned my addr 172.21.15.83:0/1018896181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:40.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d95c640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed881478b0 0x7fed88149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 -- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c0073e0 msgr2=0x7fd66c0e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c0073e0 0x7fd66c0e8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 -- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c03a170 msgr2=0x7fd66c0329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67a759640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c03a170 0x7fd66c0329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c03a170 0x7fd66c0329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c04b3e0 con 0x7fd66c00b8a0 2024-05-17T21:57:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f84c1f6e640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84bc155600 0x7f84bc13d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ac000 tx=0x7f84b8019510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe7fff640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdbcc06f4d0 0x7fdbcc0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbed132640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7fdbcc077740 0x7fdbcc079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbed132640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7fdbcc077740 0x7fdbcc079b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d9786640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f35c0077650 0x7f35c0079aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280b0040 con 0x7f9b380b4990 2024-05-17T21:57:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe7fff640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdbcc06f4d0 0x7fdbcc0719e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b69c0 tx=0x7fdbdc009410 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b280ba070 con 0x7f9b380b4990 2024-05-17T21:57:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280b4840 con 0x7f9b380b4990 2024-05-17T21:57:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d3fff640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f35c006f400 0x7f35c0071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35d4073770 tx=0x7f35c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b380b2340 con 0x7f9b380b4990 2024-05-17T21:57:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 -- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed881478b0 msgr2=0x7fed88149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed881478b0 0x7fed88149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 -- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed880b1650 msgr2=0x7fed881495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed880b1650 0x7fed881495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fed8d15b640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880b1e30 con 0x7fed74003a60 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b380b2900 con 0x7f9b380b4990 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd679757640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 0x7fd66c04a800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd6700191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700b0050 con 0x7fd66c00b8a0 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c0e96d0 con 0x7fd66c00b8a0 2024-05-17T21:57:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd670000e50 con 0x7fd66c00b8a0 2024-05-17T21:57:40.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c048ac0 con 0x7fd66c00b8a0 2024-05-17T21:57:40.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdbd40b9ec0 con 0x7fdbcc077740 2024-05-17T21:57:40.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7f35d9786640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f35c0077650 0x7f35c0079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd66c053a50 con 0x7fd66c00b8a0 2024-05-17T21:57:40.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed8d15b640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed88146990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c10 tx=0x7fed74005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700b47b0 con 0x7fd66c00b8a0 2024-05-17T21:57:40.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b380b2ce0 con 0x7f9b380b4990 2024-05-17T21:57:40.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780b1030 con 0x7fed74003a60 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.669+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf4c01a070 con 0x7fcf5015fc40 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf45ffb640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcf3406f430 0x7fcf34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf4c08e720 con 0x7fcf5015fc40 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf45ffb640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fcf34077700 0x7fcf34079b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf3407a220 con 0x7fcf34077700 2024-05-17T21:57:40.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcf4c08eb10 con 0x7fcf5015fc40 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed8f3bf640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed880b2110 con 0x7fed74003a60 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed8f3bf640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed880b26d0 con 0x7fed74003a60 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed780015c0 con 0x7fed74003a60 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004db0 con 0x7fed74003a60 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf55397640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcf3406f430 0x7fcf34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf55397640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcf3406f430 0x7fcf34071940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf480012f0 tx=0x7fcf48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fed8f3bf640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed88069ab0 con 0x7fed74003a60 2024-05-17T21:57:40.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf55b98640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fcf34077700 0x7fcf34079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80c0020 con 0x7f84bc155600 2024-05-17T21:57:40.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b80b8840 con 0x7f84bc155600 2024-05-17T21:57:40.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b8bf0 con 0x7f84bc155600 2024-05-17T21:57:40.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc154840 con 0x7f84bc155600 2024-05-17T21:57:40.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc154e00 con 0x7f84bc155600 2024-05-17T21:57:40.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fcf55b98640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fcf34077700 0x7fcf34079b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35c40c5e70 con 0x7f35c0077650 2024-05-17T21:57:40.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84bc1551e0 con 0x7f84bc155600 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b280b3070 con 0x7f9b380b4990 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b34ff9640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b2406f4d0 0x7f9b240719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b28128990 con 0x7f9b380b4990 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b34ff9640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f9b240776c0 0x7f9b24079b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b2800af30 con 0x7f9b240776c0 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b377fe640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b2406f4d0 0x7f9b240719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b3cddf640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f9b240776c0 0x7f9b24079b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b3cddf640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f9b240776c0 0x7f9b24079b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7f9b377fe640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b2406f4d0 0x7f9b240719e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b38160b30 tx=0x7f9b2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9b280f1840 con 0x7f9b380b4990 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848017070 con 0x7f78540b4cb0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78480052d0 con 0x7f78540b4cb0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848003840 con 0x7f78540b4cb0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785414dc80 con 0x7f78540b4cb0 2024-05-17T21:57:40.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785414e240 con 0x7f78540b4cb0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.673+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6700b4950 con 0x7fd66c00b8a0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd66a7fc640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd65806f400 0x7fd658071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd670129d30 con 0x7fd66c00b8a0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd66a7fc640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7fd6580775f0 0x7fd658079a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6700c8e50 con 0x7fd6580775f0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd67012a120 con 0x7fd66c00b8a0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd67a759640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7fd6580775f0 0x7fd658079a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fed780c0020 con 0x7fed74003a60 2024-05-17T21:57:40.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fed6c06f400 0x7fed6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed7812a400 con 0x7fed74003a60 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7fed6c077670 0x7fed6c079ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed6c07a190 con 0x7fed6c077670 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fed7812a7f0 con 0x7fed74003a60 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed8c95a640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fed6c06f400 0x7fed6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed8d95c640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7fed6c077670 0x7fed6c079ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed8c95a640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fed6c06f400 0x7fed6c071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed881544c0 tx=0x7fed8000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b2800af30 con 0x7f9b240776c0 2024-05-17T21:57:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd679f58640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd65806f400 0x7fd658071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd67a759640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7fd6580775f0 0x7fd658079a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fd679f58640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd65806f400 0x7fd658071910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd67406b900 tx=0x7fd67406b3a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7fed8d95c640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7fed6c077670 0x7fed6c079ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.677+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f784801a070 con 0x7f78540b4cb0 2024-05-17T21:57:40.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f783406f4d0 0x7f78340719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f84b80a6720 con 0x7f84bc155600 2024-05-17T21:57:40.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f785406e920 con 0x7f78540b4cb0 2024-05-17T21:57:40.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f784808e630 con 0x7f78540b4cb0 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f7834077720 0x7f7834079b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7848008e50 con 0x7f7834077720 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7852d9d640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f7834077720 0x7f7834079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f84a006f500 0x7f84a0071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f785259c640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f783406f4d0 0x7f78340719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f84b8129d30 con 0x7f84bc155600 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84c176d640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f84a006f500 0x7f84a0071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7f84a00777d0 0x7f84a0079c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84a007a2f0 con 0x7f84a00777d0 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f84b812a120 con 0x7f84bc155600 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84c0f6c640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7f84a00777d0 0x7f84a0079c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7852d9d640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f7834077720 0x7f7834079b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f785259c640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f783406f4d0 0x7f78340719e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ae000 tx=0x7f784c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f78480574e0 con 0x7f78540b4cb0 2024-05-17T21:57:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf3407a220 con 0x7fcf34077700 2024-05-17T21:57:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84c0f6c640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7f84a00777d0 0x7f84a0079c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed6c07a190 con 0x7fed6c077670 2024-05-17T21:57:40.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7848008e50 con 0x7f7834077720 2024-05-17T21:57:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6700c8e50 con 0x7fd6580775f0 2024-05-17T21:57:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84c176d640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f84a006f500 0x7f84a0071a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84b4000c10 tx=0x7f84b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.681+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84a007a2f0 con 0x7f84a00777d0 2024-05-17T21:57:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.697+0000 7fdbeeb95640 1 -- 172.21.15.83:0/1035545994 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdbc8002de0 con 0x7fdbcc077740 2024-05-17T21:57:40.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.697+0000 7fdbe57fa640 1 -- 172.21.15.83:0/1035545994 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdbc8002de0 con 0x7fdbcc077740 2024-05-17T21:57:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.697+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b380739f0 con 0x7f9b240776c0 2024-05-17T21:57:40.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b34ff9640 1 -- 172.21.15.83:0/738127275 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b380739f0 con 0x7f9b240776c0 2024-05-17T21:57:40.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f9b240776c0 msgr2=0x7f9b24079b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f9b240776c0 0x7f9b24079b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b2406f4d0 msgr2=0x7f9b240719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b2406f4d0 0x7f9b240719e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b38160b30 tx=0x7f9b2c009040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 msgr2=0x7f9b38159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 --2- 172.21.15.83:0/738127275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b380b4990 0x7f9b38159830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b280190b0 tx=0x7f9b28019240 comp rx=0 tx=0).stop 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3cddf640 1 -- 172.21.15.83:0/738127275 reap_dead start 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 shutdown_connections 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 >> 172.21.15.83:0/738127275 conn(0x7f9b380d1440 msgr2=0x7f9b3815f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 shutdown_connections 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f9b3e842640 1 -- 172.21.15.83:0/738127275 wait complete. 2024-05-17T21:57:40.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35d41fdfd0 con 0x7f35c0077650 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7fdbcc077740 msgr2=0x7fdbcc079b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7fdbcc077740 0x7fdbcc079b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdbcc06f4d0 msgr2=0x7fdbcc0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdbcc06f4d0 0x7fdbcc0719e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80b69c0 tx=0x7fdbdc009410 comp rx=0 tx=0).stop 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 msgr2=0x7fdbe80aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 --2- 172.21.15.83:0/1035545994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe8201870 0x7fdbe80aa310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4000c10 tx=0x7fdbd8005840 comp rx=0 tx=0).stop 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbed132640 1 -- 172.21.15.83:0/1035545994 reap_dead start 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 shutdown_connections 2024-05-17T21:57:40.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 >> 172.21.15.83:0/1035545994 conn(0x7fdbe806c790 msgr2=0x7fdbe81630d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 shutdown_connections 2024-05-17T21:57:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7f35d17fa640 1 -- 172.21.15.83:0/4014949206 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35d41fdfd0 con 0x7f35c0077650 2024-05-17T21:57:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.701+0000 7fdbd2ffd640 1 -- 172.21.15.83:0/1035545994 wait complete. 2024-05-17T21:57:40.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd66c04b190 con 0x7fd6580775f0 2024-05-17T21:57:40.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf30002de0 con 0x7fcf34077700 2024-05-17T21:57:40.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fd66a7fc640 1 -- 172.21.15.83:0/2891371965 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd66c04b190 con 0x7fd6580775f0 2024-05-17T21:57:40.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf45ffb640 1 -- 172.21.15.83:0/1809958656 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcf30002de0 con 0x7fcf34077700 2024-05-17T21:57:40.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7fd6580775f0 msgr2=0x7fd658079a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7fd6580775f0 0x7fd658079a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fcf34077700 msgr2=0x7fcf34079b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fcf34077700 0x7fcf34079b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcf3406f430 msgr2=0x7fcf34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcf3406f430 0x7fcf34071940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf480012f0 tx=0x7fcf48009040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 msgr2=0x7fcf500b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 --2- 172.21.15.83:0/1809958656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5015fc40 0x7fcf500b10a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c420 tx=0x7fcf4c00c8e0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf55b98640 1 -- 172.21.15.83:0/1809958656 reap_dead start 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 shutdown_connections 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 >> 172.21.15.83:0/1809958656 conn(0x7fcf500d1440 msgr2=0x7fcf50157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 shutdown_connections 2024-05-17T21:57:40.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.705+0000 7fcf575fb640 1 -- 172.21.15.83:0/1809958656 wait complete. 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd65806f400 msgr2=0x7fd658071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd65806f400 0x7fd658071910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd67406b900 tx=0x7fd67406b3a0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 msgr2=0x7fd66c04a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 --2- 172.21.15.83:0/2891371965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd66c00b8a0 0x7fd66c04a800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd670009870 tx=0x7fd6700191e0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67a759640 1 -- 172.21.15.83:0/2891371965 reap_dead start 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 shutdown_connections 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 >> 172.21.15.83:0/2891371965 conn(0x7fd66c01dfc0 msgr2=0x7fd66c006cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 shutdown_connections 2024-05-17T21:57:40.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7fd67c1bc640 1 -- 172.21.15.83:0/2891371965 wait complete. 2024-05-17T21:57:40.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84bc154310 con 0x7f84a00777d0 2024-05-17T21:57:40.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f35c0077650 msgr2=0x7f35c0079aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f35db1e9640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f35c0077650 0x7f35c0079aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f84adffb640 1 -- 172.21.15.83:0/1032291215 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84bc154310 con 0x7f84a00777d0 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7f84a00777d0 msgr2=0x7f84a0079c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.709+0000 7f84c39d1640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7f84a00777d0 0x7f84a0079c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f84a006f500 msgr2=0x7f84a0071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f84a006f500 0x7f84a0071a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84b4000c10 tx=0x7f84b4009040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84bc155600 msgr2=0x7f84bc13d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 --2- 172.21.15.83:0/1032291215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84bc155600 0x7f84bc13d6f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f84b80ac000 tx=0x7f84b8019510 comp rx=0 tx=0).stop 2024-05-17T21:57:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f35c006f400 msgr2=0x7f35c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f35c006f400 0x7f35c0071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35d4073770 tx=0x7f35c8008040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 msgr2=0x7f35d415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 --2- 172.21.15.83:0/4014949206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d406eea0 0x7f35d415d3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f35c401c8d0 tx=0x7f35c40191b0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c1f6e640 1 -- 172.21.15.83:0/1032291215 reap_dead start 2024-05-17T21:57:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 shutdown_connections 2024-05-17T21:57:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 >> 172.21.15.83:0/1032291215 conn(0x7f84bc0b3600 msgr2=0x7f84bc06f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35d9786640 1 -- 172.21.15.83:0/4014949206 reap_dead start 2024-05-17T21:57:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 shutdown_connections 2024-05-17T21:57:40.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 >> 172.21.15.83:0/4014949206 conn(0x7f35d40d1440 msgr2=0x7f35d4161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 shutdown_connections 2024-05-17T21:57:40.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 shutdown_connections 2024-05-17T21:57:40.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f84c39d1640 1 -- 172.21.15.83:0/1032291215 wait complete. 2024-05-17T21:57:40.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7f35db1e9640 1 -- 172.21.15.83:0/4014949206 wait complete. 2024-05-17T21:57:40.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7fed8f3bf640 1 -- 172.21.15.83:0/1018896181 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed88069ab0 con 0x7fed6c077670 2024-05-17T21:57:40.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.713+0000 7fed7dffb640 1 -- 172.21.15.83:0/1018896181 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed88069ab0 con 0x7fed6c077670 2024-05-17T21:57:40.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.717+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78540b46f0 con 0x7f7834077720 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.717+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7fed6c077670 msgr2=0x7fed6c079ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.717+0000 7fed737fe640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7fed6c077670 0x7fed6c079ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.717+0000 7f7842ffd640 1 -- 172.21.15.83:0/1559471027 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78540b46f0 con 0x7f7834077720 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fed6c06f400 msgr2=0x7fed6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fed6c06f400 0x7fed6c071910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fed881544c0 tx=0x7fed8000b040 comp rx=0 tx=0).stop 2024-05-17T21:57:40.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 msgr2=0x7fed88146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 --2- 172.21.15.83:0/1018896181 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed74003a60 0x7fed88146990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c10 tx=0x7fed74005840 comp rx=0 tx=0).stop 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed8d95c640 1 -- 172.21.15.83:0/1018896181 reap_dead start 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 shutdown_connections 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 >> 172.21.15.83:0/1018896181 conn(0x7fed8806c790 msgr2=0x7fed88070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f7834077720 msgr2=0x7f7834079b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 shutdown_connections 2024-05-17T21:57:40.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f7834077720 0x7f7834079b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7fed737fe640 1 -- 172.21.15.83:0/1018896181 wait complete. 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f783406f4d0 msgr2=0x7f78340719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f783406f4d0 0x7f78340719e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ae000 tx=0x7f784c0a7070 comp rx=0 tx=0).stop 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 msgr2=0x7f7854150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 --2- 172.21.15.83:0/1559471027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78540b4cb0 0x7f7854150d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f784800c430 tx=0x7f784800c8f0 comp rx=0 tx=0).stop 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7852d9d640 1 -- 172.21.15.83:0/1559471027 reap_dead start 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 shutdown_connections 2024-05-17T21:57:40.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 >> 172.21.15.83:0/1559471027 conn(0x7f785406c790 msgr2=0x7f785415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:40.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 shutdown_connections 2024-05-17T21:57:40.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:40.721+0000 7f7858c42640 1 -- 172.21.15.83:0/1559471027 wait complete. 2024-05-17T21:57:40.752 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-17T21:57:40.753 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-05-17T21:57:40.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-17T21:57:40.754 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-17T21:57:40.756 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-05-17T21:57:40.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-17T21:57:40.757 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-17T21:57:40.757 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-17T21:57:40.758 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-17T21:57:40.759 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:40.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-17T21:57:41.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f79cd664640 1 Processor -- start 2024-05-17T21:57:41.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f79cd664640 1 -- start start 2024-05-17T21:57:41.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7efe5e958640 1 Processor -- start 2024-05-17T21:57:41.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f9d048ca640 1 Processor -- start 2024-05-17T21:57:41.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7efe5e958640 1 -- start start 2024-05-17T21:57:41.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f3c7d6e9640 1 Processor -- start 2024-05-17T21:57:41.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7fd3a956a640 1 Processor -- start 2024-05-17T21:57:41.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f3c7d6e9640 1 -- start start 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f4e4e59a640 1 Processor -- start 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79cd664640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c80b5bd0 0x7f79c815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79cd664640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79c815d810 0x7f79c806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79cd664640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80b53e0 con 0x7f79c80b44f0 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79cd664640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f79c80b4900 con 0x7f79c815d810 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79cd664640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80b4b40 con 0x7f79c80b5bd0 2024-05-17T21:57:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79c815d810 0x7f79c806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79c815d810 0x7f79c806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39624/0 (socket says 172.21.15.83:39624) 2024-05-17T21:57:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 -- 172.21.15.83:0/3307035385 learned_addr learned my addr 172.21.15.83:0/3307035385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e4e59a640 1 -- start start 2024-05-17T21:57:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7f9d048ca640 1 -- start start 2024-05-17T21:57:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9d048ca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d0015c980 0x7f9d000b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c67fc640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c80b5bd0 0x7f79c815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 -- 172.21.15.83:0/3307035385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c80b5bd0 msgr2=0x7f79c815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c80b5bd0 0x7f79c815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 -- 172.21.15.83:0/3307035385 >> v1:172.21.15.155:6789/0 conn(0x7f79c80b44f0 legacy=0x7f79c80b3b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f79c77fe640 1 -- 172.21.15.83:0/3307035385 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80b4e20 con 0x7f79c815d810 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e4e59a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e4000a1e0 0x7f4e40053b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e4e59a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4000a960 con 0x7f4e400540c0 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e4e59a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4000ac40 con 0x7f4e400df5f0 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e4e59a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4e40007110 con 0x7f4e4000a1e0 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9d048ca640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b3210 0x7f9d0006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9d048ca640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d000b0980 con 0x7f9d000b1d20 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9d048ca640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9d0015cd90 con 0x7f9d0015c980 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9d048ca640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d000b0c60 con 0x7f9d000b3210 2024-05-17T21:57:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e47fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f4e400df5f0 0x7f4e400313d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48768/0 (socket says 172.21.15.83:48768) 2024-05-17T21:57:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e47fff640 1 -- 172.21.15.83:0/4039441010 learned_addr learned my addr 172.21.15.83:0/4039441010 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5e958640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe50007d70 0x7efe500ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 Processor -- start 2024-05-17T21:57:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 -- start start 2024-05-17T21:57:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.021+0000 7fd3a956a640 1 -- start start 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a956a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a956a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a4160380 con 0x7fd3a406dec0 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a956a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a40b5e90 con 0x7fd3a4155d50 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a956a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a40b5a00 con 0x7fd3a415fa30 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfed9d640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9d000b1d20 0x7f9d000aff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48774/0 (socket says 172.21.15.83:48774) 2024-05-17T21:57:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfed9d640 1 -- 172.21.15.83:0/713020174 learned_addr learned my addr 172.21.15.83:0/713020174 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c7d6e9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5e958640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe5003a850 con 0x7efe50031c60 2024-05-17T21:57:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5e958640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe50007650 con 0x7efe500df930 2024-05-17T21:57:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5e958640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efe50052bb0 con 0x7efe50007d70 2024-05-17T21:57:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe577fe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7efe500df930 0x7efe50006f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48782/0 (socket says 172.21.15.83:48782) 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe577fe640 1 -- 172.21.15.83:0/43056340 learned_addr learned my addr 172.21.15.83:0/43056340 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48b80b64d0 0x7f48b80703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b80736b0 0x7f48b81642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b80b5010 con 0x7f48b8163ea0 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f48b80b68e0 con 0x7f48b80b64d0 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bd126640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f48b8070900 con 0x7f48b80736b0 2024-05-17T21:57:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c7d6e9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700e3630 con 0x7f3c700e3220 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c7d6e9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c70053d40 con 0x7f3c700533f0 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c7d6e9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c70053800 con 0x7f3c70054150 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c767fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49460/0 (socket says 172.21.15.83:49460) 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 -- 172.21.15.83:0/662543028 learned_addr learned my addr 172.21.15.83:0/662543028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e477fe640 1 --2- 172.21.15.83:0/4039441010 >> v2:172.21.15.83:3300/0 conn(0x7f4e4000a1e0 0x7f4e40053b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c767fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49474/0 (socket says 172.21.15.83:49474) 2024-05-17T21:57:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f3c767fc640 1 -- 172.21.15.83:0/1512248871 learned_addr learned my addr 172.21.15.83:0/1512248871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e46ffd640 1 -- 172.21.15.83:0/4039441010 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841196613 0 0) 0x7f4e4000ac40 con 0x7f4e400df5f0 2024-05-17T21:57:41.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e46ffd640 1 -- 172.21.15.83:0/4039441010 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e38003530 con 0x7f4e400df5f0 2024-05-17T21:57:41.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5cef5640 1 --2- 172.21.15.83:0/43056340 >> v2:172.21.15.155:3300/0 conn(0x7efe50007d70 0x7efe500ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfe59c640 1 --2- 172.21.15.83:0/713020174 >> v2:172.21.15.83:3300/0 conn(0x7f9d0015c980 0x7f9d000b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48b77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f48b8163ea0 0x7f48b8072fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33690/0 (socket says 172.21.15.83:33690) 2024-05-17T21:57:41.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cff59e640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b3210 0x7f9d0006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48b77fe640 1 -- 172.21.15.83:0/1323769895 learned_addr learned my addr 172.21.15.83:0/1323769895 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48bc925640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b80736b0 0x7f48b81642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f48b7fff640 1 --2- 172.21.15.83:0/1323769895 >> v2:172.21.15.155:3300/0 conn(0x7f48b80b64d0 0x7f48b80703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfe59c640 1 -- 172.21.15.83:0/713020174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b3210 msgr2=0x7f9d0006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfe59c640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b3210 0x7f9d0006ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfe59c640 1 -- 172.21.15.83:0/713020174 >> v1:172.21.15.155:6789/0 conn(0x7f9d000b1d20 legacy=0x7f9d000aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 -- 172.21.15.83:0/662543028 >> v1:172.21.15.155:6789/0 conn(0x7fd3a4155d50 legacy=0x7fd3a4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 -- 172.21.15.83:0/662543028 >> v1:172.21.15.83:6789/0 conn(0x7fd3a406dec0 legacy=0x7fd3a4070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e477fe640 1 -- 172.21.15.83:0/4039441010 >> v1:172.21.15.155:6789/0 conn(0x7f4e400df5f0 legacy=0x7f4e400313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e477fe640 1 -- 172.21.15.83:0/4039441010 >> v1:172.21.15.83:6789/0 conn(0x7f4e400540c0 legacy=0x7f4e40036260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f4e477fe640 1 -- 172.21.15.83:0/4039441010 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e400530a0 con 0x7f4e4000a1e0 2024-05-17T21:57:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7fd3a37fe640 1 -- 172.21.15.83:0/662543028 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40b4b10 con 0x7fd3a415fa30 2024-05-17T21:57:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7f9cfe59c640 1 -- 172.21.15.83:0/713020174 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d000b0f40 con 0x7f9d0015c980 2024-05-17T21:57:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b6ffd640 1 -- 172.21.15.83:0/1323769895 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095356009 0 0) 0x7f48b80b5010 con 0x7f48b8163ea0 2024-05-17T21:57:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b6ffd640 1 -- 172.21.15.83:0/1323769895 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48a0003530 con 0x7f48b8163ea0 2024-05-17T21:57:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f79c77fe640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c815d810 0x7f79c806f6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d1e0 tx=0x7f79c001ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5210b63e46180458 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5cef5640 1 -- 172.21.15.83:0/43056340 >> v1:172.21.15.155:6789/0 conn(0x7efe500df930 legacy=0x7efe50006f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5cef5640 1 -- 172.21.15.83:0/43056340 >> v1:172.21.15.83:6789/0 conn(0x7efe50031c60 legacy=0x7efe50006830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.025+0000 7efe5cef5640 1 -- 172.21.15.83:0/43056340 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50052e00 con 0x7efe50007d70 2024-05-17T21:57:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f15998e0640 1 Processor -- start 2024-05-17T21:57:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b7fff640 1 -- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b80736b0 msgr2=0x7f48b81642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b7fff640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b80736b0 0x7f48b81642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b7fff640 1 -- 172.21.15.83:0/1323769895 >> v1:172.21.15.83:6789/0 conn(0x7f48b8163ea0 legacy=0x7f48b8072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f48b7fff640 1 -- 172.21.15.83:0/1323769895 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b80b52f0 con 0x7f48b80b64d0 2024-05-17T21:57:41.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f79c5ffb640 1 -- 172.21.15.83:0/3307035385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00ad070 con 0x7f79c815d810 2024-05-17T21:57:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f79c5ffb640 1 -- 172.21.15.83:0/3307035385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c00a7d90 con 0x7f79c815d810 2024-05-17T21:57:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c767fc640 1 -- 172.21.15.83:0/1512248871 >> v1:172.21.15.155:6789/0 conn(0x7f3c700533f0 legacy=0x7f3c7001c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c767fc640 1 -- 172.21.15.83:0/1512248871 >> v1:172.21.15.83:6789/0 conn(0x7f3c700e3220 legacy=0x7f3c70006a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c767fc640 1 -- 172.21.15.83:0/1512248871 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c70053a60 con 0x7f3c70054150 2024-05-17T21:57:41.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f4e477fe640 1 --2- 172.21.15.83:0/4039441010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4000a1e0 0x7f4e40053b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0097c0 tx=0x7f4e3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39d43d945305cbe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7efe5cef5640 1 --2- 172.21.15.83:0/43056340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50007d70 0x7efe500ea2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe58052290 tx=0x7efe5809b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05edb2c0595a0d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f9cfe59c640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d000b2cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00097c0 tx=0x7f9cf001e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86743f14d9e40ee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7fd3a37fe640 1 --2- 172.21.15.83:0/662543028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd39400bb20 tx=0x7fd39401e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2849f3f07d3541a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f4e46ffd640 1 -- 172.21.15.83:0/4039441010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0b0070 con 0x7f4e4000a1e0 2024-05-17T21:57:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f79c5ffb640 1 -- 172.21.15.83:0/3307035385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00b1640 con 0x7f79c815d810 2024-05-17T21:57:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f15998e0640 1 -- start start 2024-05-17T21:57:41.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c767fc640 1 --2- 172.21.15.83:0/1512248871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3c70032ad0 tx=0x7f3c7809a920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2903f06b5f2b2ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f9cfdd9b640 1 -- 172.21.15.83:0/713020174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00ae070 con 0x7f9d0015c980 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f9cfdd9b640 1 -- 172.21.15.83:0/713020174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf00a8650 con 0x7f9d0015c980 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f9cfdd9b640 1 -- 172.21.15.83:0/713020174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00a8a20 con 0x7f9d0015c980 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7fd3a2ffd640 1 -- 172.21.15.83:0/662543028 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394004230 con 0x7fd3a415fa30 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7fd3a2ffd640 1 -- 172.21.15.83:0/662543028 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3940043d0 con 0x7fd3a415fa30 2024-05-17T21:57:41.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7fd3a2ffd640 1 -- 172.21.15.83:0/662543028 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394003820 con 0x7fd3a415fa30 2024-05-17T21:57:41.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f4e46ffd640 1 -- 172.21.15.83:0/4039441010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3c0ab600 con 0x7f4e4000a1e0 2024-05-17T21:57:41.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f4e46ffd640 1 -- 172.21.15.83:0/4039441010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0b0070 con 0x7f4e4000a1e0 2024-05-17T21:57:41.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c75ffb640 1 -- 172.21.15.83:0/1512248871 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78130070 con 0x7f3c70054150 2024-05-17T21:57:41.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.029+0000 7f3c75ffb640 1 -- 172.21.15.83:0/1512248871 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c781238e0 con 0x7f3c70054150 2024-05-17T21:57:41.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c75ffb640 1 -- 172.21.15.83:0/1512248871 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78073040 con 0x7f3c70054150 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48b7fff640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b80703c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009d90 tx=0x7f48b001c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26df82639c7e0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe56ffd640 1 -- 172.21.15.83:0/43056340 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe581294e0 con 0x7efe50007d70 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe56ffd640 1 -- 172.21.15.83:0/43056340 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe58129680 con 0x7efe50007d70 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe56ffd640 1 -- 172.21.15.83:0/43056340 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58129af0 con 0x7efe50007d70 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- 172.21.15.83:0/4039441010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4000a1e0 msgr2=0x7f4e40053b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/4039441010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4000a1e0 0x7f4e40053b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0097c0 tx=0x7f4e3c01c4a0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- 172.21.15.83:0/713020174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 msgr2=0x7f9d000b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d000b2cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00097c0 tx=0x7f9cf001e680 comp rx=0 tx=0).stop 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- 172.21.15.83:0/4039441010 shutdown_connections 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/4039441010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e4000a1e0 0x7f4e40053b80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- 172.21.15.83:0/4039441010 >> 172.21.15.83:0/4039441010 conn(0x7f4e4001db70 msgr2=0x7f4e40039610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- 172.21.15.83:0/4039441010 shutdown_connections 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48b6ffd640 1 -- 172.21.15.83:0/1323769895 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00b2070 con 0x7f48b80b64d0 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- 172.21.15.83:0/713020174 shutdown_connections 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b3210 0x7f9d0006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 --2- 172.21.15.83:0/713020174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d000b2cd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- 172.21.15.83:0/713020174 >> 172.21.15.83:0/713020174 conn(0x7f9d00074140 msgr2=0x7f9d000b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48b6ffd640 1 -- 172.21.15.83:0/1323769895 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b00a5a00 con 0x7f48b80b64d0 2024-05-17T21:57:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48b6ffd640 1 -- 172.21.15.83:0/1323769895 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00a5db0 con 0x7f48b80b64d0 2024-05-17T21:57:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- 172.21.15.83:0/713020174 shutdown_connections 2024-05-17T21:57:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f15998e0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15940cef10 0x7f159415e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 -- 172.21.15.83:0/43056340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50007d70 msgr2=0x7efe500ea2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 --2- 172.21.15.83:0/43056340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50007d70 0x7efe500ea2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe58052290 tx=0x7efe5809b8e0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 -- 172.21.15.83:0/3307035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c815d810 msgr2=0x7f79c806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c815d810 0x7f79c806f6f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d1e0 tx=0x7f79c001ed20 comp rx=0 tx=0).stop 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 -- 172.21.15.83:0/3307035385 shutdown_connections 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c815d810 0x7f79c806f6f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 --2- 172.21.15.83:0/3307035385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c80b5bd0 0x7f79c815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 -- 172.21.15.83:0/3307035385 >> 172.21.15.83:0/3307035385 conn(0x7f79c806c790 msgr2=0x7f79c806cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f15998e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159406cea0 0x7f15940b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f15998e0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1594167b70 con 0x7f1594164e40 2024-05-17T21:57:41.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f15998e0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1594168780 con 0x7f15940cef10 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f15998e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1594167e50 con 0x7f159406cea0 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 -- 172.21.15.83:0/43056340 shutdown_connections 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 --2- 172.21.15.83:0/43056340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50007d70 0x7efe500ea2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 -- 172.21.15.83:0/43056340 >> 172.21.15.83:0/43056340 conn(0x7efe5001e400 msgr2=0x7efe50039ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 -- 172.21.15.83:0/662543028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 msgr2=0x7fd3a415fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 --2- 172.21.15.83:0/662543028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd39400bb20 tx=0x7fd39401e7c0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 -- 172.21.15.83:0/43056340 shutdown_connections 2024-05-17T21:57:41.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7efe5e958640 1 -- 172.21.15.83:0/43056340 wait complete. 2024-05-17T21:57:41.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1512248871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 msgr2=0x7f3c700e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 --2- 172.21.15.83:0/1512248871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3c70032ad0 tx=0x7f3c7809a920 comp rx=0 tx=0).stop 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 -- 172.21.15.83:0/662543028 shutdown_connections 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 --2- 172.21.15.83:0/662543028 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a415fa30 0x7fd3a415fe40 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd39400bb20 tx=0x7fd39401e7c0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 -- 172.21.15.83:0/662543028 >> 172.21.15.83:0/662543028 conn(0x7fd3a40d1cb0 msgr2=0x7fd3a40d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1512248871 shutdown_connections 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 --2- 172.21.15.83:0/1512248871 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c70054150 0x7f3c700e2ce0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1512248871 >> 172.21.15.83:0/1512248871 conn(0x7f3c700362d0 msgr2=0x7f3c700366e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15940cef10 0x7f159415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15940cef10 0x7f159415e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37876/0 (socket says 172.21.15.83:37876) 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 -- 172.21.15.83:0/748411960 learned_addr learned my addr 172.21.15.83:0/748411960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 -- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159406cea0 msgr2=0x7f15940b1380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 --2- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159406cea0 0x7f15940b1380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 -- 172.21.15.83:0/748411960 >> v1:172.21.15.155:6789/0 conn(0x7f1594164e40 legacy=0x7f159406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f1592ffd640 1 -- 172.21.15.83:0/748411960 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1594168130 con 0x7f15940cef10 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 -- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 msgr2=0x7f48b80703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b80703c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009d90 tx=0x7f48b001c980 comp rx=0 tx=0).stop 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 -- 172.21.15.83:0/1323769895 shutdown_connections 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b80736b0 0x7f48b81642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 --2- 172.21.15.83:0/1323769895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b80703c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f48bd126640 1 -- 172.21.15.83:0/1323769895 >> 172.21.15.83:0/1323769895 conn(0x7f48b806c790 msgr2=0x7f48b806cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 -- 172.21.15.83:0/3307035385 shutdown_connections 2024-05-17T21:57:41.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- 172.21.15.83:0/4039441010 wait complete. 2024-05-17T21:57:41.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f79cd664640 1 -- 172.21.15.83:0/3307035385 wait complete. 2024-05-17T21:57:41.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1512248871 shutdown_connections 2024-05-17T21:57:41.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- 172.21.15.83:0/713020174 wait complete. 2024-05-17T21:57:41.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 Processor -- start 2024-05-17T21:57:41.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f9d048ca640 1 -- start start 2024-05-17T21:57:41.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 Processor -- start 2024-05-17T21:57:41.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7fd3a956a640 1 -- 172.21.15.83:0/662543028 shutdown_connections 2024-05-17T21:57:41.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f4e4e59a640 1 -- start start 2024-05-17T21:57:41.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.033+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1512248871 wait complete. 2024-05-17T21:57:41.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7efe5e958640 1 Processor -- start 2024-05-17T21:57:41.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 Processor -- start 2024-05-17T21:57:41.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 -- start start 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e400df5f0 0x7f4e4004a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e40051910 0x7f4e4018a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400ed160 con 0x7f4e400540c0 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400ecfe0 con 0x7f4e40051910 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4e59a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400ed2e0 con 0x7f4e400df5f0 2024-05-17T21:57:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7efe5e958640 1 -- start start 2024-05-17T21:57:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39658/0 (socket says 172.21.15.83:39658) 2024-05-17T21:57:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 -- 172.21.15.83:0/3777843113 learned_addr learned my addr 172.21.15.83:0/3777843113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f79cd664640 1 Processor -- start 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b1d20 0x7f9d0014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d000b3210 0x7f9d0014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e4cb37640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e40051910 0x7f4e4018a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e477fe640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e400df5f0 0x7f4e4004a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f1592ffd640 1 --2- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15940cef10 0x7f159415e400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1584009870 tx=0x7f158401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a981bd466371c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f79cd664640 1 -- start start 2024-05-17T21:57:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39666/0 (socket says 172.21.15.83:39666) 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cfe59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d000b3210 0x7f9d0014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 -- 172.21.15.83:0/4284595194 learned_addr learned my addr 172.21.15.83:0/4284595194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0006da00 con 0x7f9d0015c980 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d000b0690 con 0x7f9d000b3210 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 -- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e400df5f0 msgr2=0x7f4e4004a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e400df5f0 0x7f4e4004a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 -- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e40051910 msgr2=0x7f4e4018a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e40051910 0x7f4e4018a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9d048ca640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0006db80 con 0x7f9d000b1d20 2024-05-17T21:57:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 -- 172.21.15.83:0/3777843113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e400511d0 con 0x7f4e400540c0 2024-05-17T21:57:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f1591ffb640 1 -- 172.21.15.83:0/748411960 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840b1070 con 0x7f15940cef10 2024-05-17T21:57:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f1591ffb640 1 -- 172.21.15.83:0/748411960 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15840ac600 con 0x7f15940cef10 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f1591ffb640 1 -- 172.21.15.83:0/748411960 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840ac9d0 con 0x7f15940cef10 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f4e47fff640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d320 tx=0x7f4e3400d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 -- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b1d20 msgr2=0x7f9d0014ec90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b1d20 0x7f9d0014ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 -- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d000b3210 msgr2=0x7f9d0014e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d000b3210 0x7f9d0014e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d00157730 con 0x7f9d0015c980 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 0x7f3c700da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700e3220 0x7f3c70172ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70054560 con 0x7f3c70054150 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7001ce70 con 0x7f3c700533f0 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c7d6e9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700df3f0 con 0x7f3c700e3220 2024-05-17T21:57:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f48bd126640 1 -- 172.21.15.83:0/1323769895 shutdown_connections 2024-05-17T21:57:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f48bd126640 1 -- 172.21.15.83:0/1323769895 wait complete. 2024-05-17T21:57:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9cff59e640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007e90 tx=0x7f9cf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f15998e0640 1 -- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15940cef10 msgr2=0x7f159415e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f15998e0640 1 --2- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15940cef10 0x7f159415e400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1584009870 tx=0x7f158401c4f0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7efe5e958640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50007d70 0x7efe50035180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7efe5e958640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe500df930 0x7efe5004ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe500ec9c0 con 0x7efe50007d70 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe500ec840 con 0x7efe50031c60 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe500ecb40 con 0x7efe500df930 2024-05-17T21:57:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c80b44f0 0x7f79c814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37922/0 (socket says 172.21.15.83:37922) 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 -- 172.21.15.83:0/1793942799 learned_addr learned my addr 172.21.15.83:0/1793942799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f15998e0640 1 -- 172.21.15.83:0/748411960 shutdown_connections 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f15998e0640 1 --2- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159406cea0 0x7f15940b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f15998e0640 1 --2- 172.21.15.83:0/748411960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15940cef10 0x7f159415e400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f15998e0640 1 -- 172.21.15.83:0/748411960 >> 172.21.15.83:0/748411960 conn(0x7f15940b4250 msgr2=0x7f15940b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c815d810 0x7f79c8150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5cef5640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe500df930 0x7efe5004ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80b5fe0 con 0x7f79c80b44f0 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8070040 con 0x7f79c80b5bd0 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37930/0 (socket says 172.21.15.83:37930) 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 -- 172.21.15.83:0/556672701 learned_addr learned my addr 172.21.15.83:0/556672701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c77fe640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c815d810 0x7f79c8150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79cd664640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80b6160 con 0x7f79c815d810 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8014420 con 0x7f9d0015c980 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf8019070 con 0x7f9d0015c980 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8014800 con 0x7f9d0015c980 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c6ffd640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c80b44f0 0x7f79c814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 -- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe500df930 msgr2=0x7efe5004ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe500df930 0x7efe5004ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 -- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50007d70 msgr2=0x7efe50035180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50007d70 0x7efe50035180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 -- 172.21.15.83:0/1793942799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50037990 con 0x7efe50031c60 2024-05-17T21:57:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f48bd126640 1 Processor -- start 2024-05-17T21:57:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34014020 con 0x7f4e400540c0 2024-05-17T21:57:41.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3400f8c0 con 0x7f4e400540c0 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400fcb0 con 0x7f4e400540c0 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e400495d0 con 0x7f4e400540c0 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e40049b90 con 0x7f4e400540c0 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 -- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c815d810 msgr2=0x7f79c8150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c815d810 0x7f79c8150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 -- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c80b44f0 msgr2=0x7f79c814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c80b44f0 0x7f79c814f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c814d100 con 0x7f79c80b5bd0 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe577fe640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c4d0 tx=0x7efe4c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c017070 con 0x7efe50031c60 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c005cf0 con 0x7efe50031c60 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c007270 con 0x7efe50031c60 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 -- 172.21.15.83:0/1793942799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5004a360 con 0x7efe50031c60 2024-05-17T21:57:41.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe5e958640 1 -- 172.21.15.83:0/1793942799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe500dc7f0 con 0x7efe50031c60 2024-05-17T21:57:41.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f79c67fc640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e8a0 tx=0x7f79bc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f9d048ca640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d00157a10 con 0x7f9d0015c980 2024-05-17T21:57:41.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f48bd126640 1 -- start start 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f48bd126640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b8204ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f48bd126640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 0x7f48b8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f48bd126640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39668/0 (socket says 172.21.15.83:39668) 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 -- 172.21.15.83:0/1019447525 learned_addr learned my addr 172.21.15.83:0/1019447525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f9d048ca640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0014f260 con 0x7f9d0015c980 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c777fe640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700e3220 0x7f3c70172ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c76ffd640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 0x7f3c700da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 -- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700e3220 msgr2=0x7f3c70172ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700e3220 0x7f3c70172ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 -- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 msgr2=0x7f3c700da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 0x7f3c700da3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 -- 172.21.15.83:0/1019447525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700da930 con 0x7f3c70054150 2024-05-17T21:57:41.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c76ffd640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 0x7f3c700da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.037+0000 7f3c767fc640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c78052290 tx=0x7f3c780991b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78130040 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7812b680 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7812ba70 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1019447525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c700dca80 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1019447525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c700dcd40 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1019447525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7017d9d0 con 0x7f3c70054150 2024-05-17T21:57:41.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300052a0 con 0x7efe50031c60 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.041+0000 7fd3a956a640 1 -- 172.21.15.83:0/662543028 wait complete. 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7fd3a956a640 1 Processor -- start 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48b77fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 0x7f48b8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48b77fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 0x7f48b8153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:49542/0 (socket says 172.21.15.83:49542) 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48b7fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b8204ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48b7fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b8204ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37932/0 (socket says 172.21.15.83:37932) 2024-05-17T21:57:41.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e34017020 con 0x7f4e400540c0 2024-05-17T21:57:41.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f15998e0640 1 -- 172.21.15.83:0/748411960 shutdown_connections 2024-05-17T21:57:41.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f4e44ff9640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 0x7f4e2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e3408fab0 con 0x7f4e400540c0 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f4e477fe640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 0x7f4e2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f4e477fe640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 0x7f4e2c071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0ad000 tx=0x7f4e3c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe4c01a070 con 0x7efe50031c60 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7efe54ff9640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 0x7efe3c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe4c08eb70 con 0x7efe50031c60 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7efe57fff640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 0x7efe3c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7efe57fff640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 0x7efe3c071940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe44007fd0 tx=0x7efe44007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c7812bc10 con 0x7f3c70054150 2024-05-17T21:57:41.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f15998e0640 1 -- 172.21.15.83:0/748411960 wait complete. 2024-05-17T21:57:41.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f3c5f7fe640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 0x7f3c64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f15998e0640 1 Processor -- start 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd80052a0 con 0x7f9d0015c980 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cf80149a0 con 0x7f9d0015c980 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9ce77fe640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 0x7f9cec071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9cfed9d640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 0x7f9cec071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9cf808fa90 con 0x7f9d0015c980 2024-05-17T21:57:41.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48bc925640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48bc925640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39708/0 (socket says 172.21.15.83:39708) 2024-05-17T21:57:41.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3c781a5940 con 0x7f3c70054150 2024-05-17T21:57:41.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48b77fe640 1 -- 172.21.15.83:0/3724430390 learned_addr learned my addr 172.21.15.83:0/3724430390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f3c76ffd640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 0x7f3c64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48b80d0ce0 con 0x7f48b8160520 2024-05-17T21:57:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f48b80d0b60 con 0x7f48b80b64d0 2024-05-17T21:57:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f48b80d0e60 con 0x7f48b8163ea0 2024-05-17T21:57:41.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f9cfed9d640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 0x7f9cec071a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d00072980 tx=0x7f9cf4007a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7f15998e0640 1 -- start start 2024-05-17T21:57:41.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f3c76ffd640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 0x7f3c64071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000bdb0 tx=0x7f3c68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 -- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 msgr2=0x7f48b8153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 0x7f48b8153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 -- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 msgr2=0x7f48b8204ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b8204ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b8160e70 con 0x7f48b8160520 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f48bc925640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e7c0 tx=0x7f48ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1594164e40 0x7f1594161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 0x7f15941616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15940b5a90 con 0x7f15940cef10 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15940b5910 con 0x7f1594164e40 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15998e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15940b5c10 con 0x7f159415ee10 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39722/0 (socket says 172.21.15.83:39722) 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 -- 172.21.15.83:0/842650485 learned_addr learned my addr 172.21.15.83:0/842650485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc017070 con 0x7f79c80b5bd0 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15927fc640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1594164e40 0x7f1594161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15937fe640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 0x7f15941616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 -- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 msgr2=0x7f15941616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 0x7f15941616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 -- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1594164e40 msgr2=0x7f1594161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1594164e40 0x7f1594161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 -- 172.21.15.83:0/842650485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159415f560 con 0x7f15940cef10 2024-05-17T21:57:41.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79bc00c8e0 con 0x7f79c80b5bd0 2024-05-17T21:57:41.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f15937fe640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 0x7f15941616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:41.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.045+0000 7fd3a956a640 1 -- start start 2024-05-17T21:57:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f1592ffd640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f158401aba0 tx=0x7f158401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4000ac40 con 0x7f4e400540c0 2024-05-17T21:57:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a4155d50 0x7fd3a40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a41558f0 0x7fd3a40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40711c0 con 0x7fd3a406dec0 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4156870 con 0x7fd3a4155d50 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a956a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a4071340 con 0x7fd3a41558f0 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79cd664640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c80cd380 con 0x7f79c80b5bd0 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc00ccb0 con 0x7f79c80b5bd0 2024-05-17T21:57:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79cd664640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c80ccdc0 con 0x7f79c80b5bd0 2024-05-17T21:57:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39730/0 (socket says 172.21.15.83:39730) 2024-05-17T21:57:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 -- 172.21.15.83:0/729210336 learned_addr learned my addr 172.21.15.83:0/729210336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a8d69640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a41558f0 0x7fd3a40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a00096b0 con 0x7f79c80b5bd0 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a37fe640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a4155d50 0x7fd3a40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 -- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a41558f0 msgr2=0x7fd3a40cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a41558f0 0x7fd3a40cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 -- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a4155d50 msgr2=0x7fd3a40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a4155d50 0x7fd3a40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 -- 172.21.15.83:0/729210336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a41fd010 con 0x7fd3a406dec0 2024-05-17T21:57:41.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.049+0000 7fd3a3fff640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00ba50 tx=0x7fd39c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.053+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9cf805c960 con 0x7f9d0015c980 2024-05-17T21:57:41.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3c700e3630 con 0x7f3c70054150 2024-05-17T21:57:41.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840b1040 con 0x7f15940cef10 2024-05-17T21:57:41.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c019070 con 0x7fd3a406dec0 2024-05-17T21:57:41.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac017070 con 0x7f48b8160520 2024-05-17T21:57:41.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b81542a0 con 0x7f48b8160520 2024-05-17T21:57:41.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a414f030 con 0x7fd3a406dec0 2024-05-17T21:57:41.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159415f840 con 0x7f15940cef10 2024-05-17T21:57:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159415cb10 con 0x7f15940cef10 2024-05-17T21:57:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a41509a0 con 0x7fd3a406dec0 2024-05-17T21:57:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b8153040 con 0x7f48b8160520 2024-05-17T21:57:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a4069a50 con 0x7fd3a406dec0 2024-05-17T21:57:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15940b1380 con 0x7f15940cef10 2024-05-17T21:57:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b81642b0 con 0x7f48b8160520 2024-05-17T21:57:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.057+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efe50037990 con 0x7efe50031c60 2024-05-17T21:57:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79bc01a070 con 0x7f79c80b5bd0 2024-05-17T21:57:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1584000e50 con 0x7f15940cef10 2024-05-17T21:57:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840b7620 con 0x7f15940cef10 2024-05-17T21:57:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac004020 con 0x7f48b8160520 2024-05-17T21:57:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac021060 con 0x7f48b8160520 2024-05-17T21:57:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f79af7fe640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 0x7f79b4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f79c6ffd640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 0x7f79b4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f79c6ffd640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 0x7f79b4071a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79b8006fd0 tx=0x7f79b8009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4e3401e020 con 0x7f4e400540c0 2024-05-17T21:57:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c00cb80 con 0x7fd3a406dec0 2024-05-17T21:57:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.061+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c003980 con 0x7fd3a406dec0 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79bc08f7f0 con 0x7f79c80b5bd0 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15840b77c0 con 0x7f15940cef10 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f48ac01a070 con 0x7f48b8160520 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f48b4ff9640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 0x7f48a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48ac08f420 con 0x7f48b8160520 2024-05-17T21:57:41.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f48b7fff640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 0x7f48a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f157bfff640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 0x7f1580071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f158412a050 con 0x7f15940cef10 2024-05-17T21:57:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f15927fc640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 0x7f1580071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f15927fc640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 0x7f1580071910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1594163d80 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7f48b7fff640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 0x7f48a4071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48b00af000 tx=0x7f48b00a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd39c01c070 con 0x7fd3a406dec0 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7fd3a0ff9640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 0x7fd390071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd39c08f470 con 0x7fd3a406dec0 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7fd3a37fe640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 0x7fd390071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.065+0000 7fd3a37fe640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 0x7fd390071940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3940b1000 tx=0x7fd3940a9000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.069+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f48b8160e70 con 0x7f48b8160520 2024-05-17T21:57:41.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.077+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f79bc05c070 con 0x7f79c80b5bd0 2024-05-17T21:57:41.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.077+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f15840c6020 con 0x7f15940cef10 2024-05-17T21:57:41.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.077+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd39c023070 con 0x7fd3a406dec0 2024-05-17T21:57:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.377+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9cd8005590 con 0x7f9d0015c980 2024-05-17T21:57:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce77fe640 1 -- 172.21.15.83:0/4284595194 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9cf805c260 con 0x7f9d0015c980 2024-05-17T21:57:41.371 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-05-17T21:57:41.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f3c7d6e9640 1 -- 172.21.15.83:0/1019447525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3c700e2ce0 con 0x7f3c70054150 2024-05-17T21:57:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 msgr2=0x7f9cec071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 0x7f9cec071a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9d00072980 tx=0x7f9cf4007a30 comp rx=0 tx=0).stop 2024-05-17T21:57:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f3c5f7fe640 1 -- 172.21.15.83:0/1019447525 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c78172820 con 0x7f3c70054150 2024-05-17T21:57:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 msgr2=0x7f9d0014d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007e90 tx=0x7f9cf800e4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 shutdown_connections 2024-05-17T21:57:41.375 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.381+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015c980 0x7f9d0014d6c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9cec06f500 0x7f9cec071a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d000b3210 0x7f9d0014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 --2- 172.21.15.83:0/4284595194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d000b1d20 0x7f9d0014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 >> 172.21.15.83:0/4284595194 conn(0x7f9d00074140 msgr2=0x7f9d00071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 shutdown_connections 2024-05-17T21:57:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f9ce57fa640 1 -- 172.21.15.83:0/4284595194 wait complete. 2024-05-17T21:57:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efe30005860 con 0x7efe50031c60 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 msgr2=0x7f3c64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 0x7f3c64071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000bdb0 tx=0x7f3c68009040 comp rx=0 tx=0).stop 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 msgr2=0x7f3c70172570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c78052290 tx=0x7f3c780991b0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 shutdown_connections 2024-05-17T21:57:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c700e3220 0x7f3c70172ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c70054150 0x7f3c70172570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3c6406f400 0x7f3c64071910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 --2- 172.21.15.83:0/1019447525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c700533f0 0x7f3c700da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 >> 172.21.15.83:0/1019447525 conn(0x7f3c700362d0 msgr2=0x7f3c7000a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 shutdown_connections 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7f3c5d7fa640 1 -- 172.21.15.83:0/1019447525 wait complete. 2024-05-17T21:57:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.385+0000 7efe54ff9640 1 -- 172.21.15.83:0/1793942799 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efe4c05b580 con 0x7efe50031c60 2024-05-17T21:57:41.380 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.389+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4e40047c60 con 0x7f4e400540c0 2024-05-17T21:57:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.389+0000 7f4e44ff9640 1 -- 172.21.15.83:0/3777843113 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e3405ca30 con 0x7f4e400540c0 2024-05-17T21:57:41.381 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 msgr2=0x7f4e2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 0x7f4e2c071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0ad000 tx=0x7f4e3c0a6000 comp rx=0 tx=0).stop 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 msgr2=0x7efe3c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 0x7efe3c071940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe44007fd0 tx=0x7efe44007960 comp rx=0 tx=0).stop 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 msgr2=0x7f4e400506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d320 tx=0x7f4e3400d7e0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 msgr2=0x7efe50037450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c4d0 tx=0x7efe4c00c990 comp rx=0 tx=0).stop 2024-05-17T21:57:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 shutdown_connections 2024-05-17T21:57:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 shutdown_connections 2024-05-17T21:57:41.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e2c06f400 0x7f4e2c071910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe500df930 0x7efe5004ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe3c06f430 0x7efe3c071940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe50031c60 0x7efe50037450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 --2- 172.21.15.83:0/1793942799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50007d70 0x7efe50035180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 >> 172.21.15.83:0/1793942799 conn(0x7efe5001e400 msgr2=0x7efe5004c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e40051910 0x7f4e4018a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 shutdown_connections 2024-05-17T21:57:41.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e400df5f0 0x7f4e4004a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 --2- 172.21.15.83:0/3777843113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e400540c0 0x7f4e400506a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 >> 172.21.15.83:0/3777843113 conn(0x7f4e4001db70 msgr2=0x7f4e40032550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7efe427fc640 1 -- 172.21.15.83:0/1793942799 wait complete. 2024-05-17T21:57:41.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 shutdown_connections 2024-05-17T21:57:41.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f4e4e59a640 1 -- 172.21.15.83:0/3777843113 wait complete. 2024-05-17T21:57:41.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f48b8152ae0 con 0x7f48b8160520 2024-05-17T21:57:41.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.393+0000 7f48b4ff9640 1 -- 172.21.15.83:0/3724430390 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48b8152ae0 con 0x7f48b8160520 2024-05-17T21:57:41.389 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 msgr2=0x7f48a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 0x7f48a4071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f48b00af000 tx=0x7f48b00a7070 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 msgr2=0x7f48b8160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e7c0 tx=0x7f48ac00ec80 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 shutdown_connections 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48b8160520 0x7f48b8160930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48b8163ea0 0x7f48b8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f48a406f400 0x7f48a4071910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 --2- 172.21.15.83:0/3724430390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b80b64d0 0x7f48b8204ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 >> 172.21.15.83:0/3724430390 conn(0x7f48b806c790 msgr2=0x7f48b8161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 shutdown_connections 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f48bd126640 1 -- 172.21.15.83:0/3724430390 wait complete. 2024-05-17T21:57:41.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.397+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f79a0009c70 con 0x7f79c80b5bd0 2024-05-17T21:57:41.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.401+0000 7f79af7fe640 1 -- 172.21.15.83:0/556672701 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f79bc060370 con 0x7f79c80b5bd0 2024-05-17T21:57:41.392 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.401+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f15940b4660 con 0x7f15940cef10 2024-05-17T21:57:41.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.401+0000 7f157bfff640 1 -- 172.21.15.83:0/842650485 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1584016170 con 0x7f15940cef10 2024-05-17T21:57:41.393 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 msgr2=0x7f79b4071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 0x7f79b4071a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79b8006fd0 tx=0x7f79b8009060 comp rx=0 tx=0).stop 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 msgr2=0x7f79c814c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e8a0 tx=0x7f79bc00ed60 comp rx=0 tx=0).stop 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 shutdown_connections 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79c815d810 0x7f79c8150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f79b406f500 0x7f79b4071a10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79c80b5bd0 0x7f79c814c1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 --2- 172.21.15.83:0/556672701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c80b44f0 0x7f79c814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 >> 172.21.15.83:0/556672701 conn(0x7f79c806c790 msgr2=0x7f79c815dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 shutdown_connections 2024-05-17T21:57:41.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f79ad7fa640 1 -- 172.21.15.83:0/556672701 wait complete. 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 msgr2=0x7f1580071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 0x7f1580071910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1594163d80 tx=0x7f1588009040 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 msgr2=0x7f15941623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f158401aba0 tx=0x7f158401ad30 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 shutdown_connections 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f159415ee10 0x7f15941616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f158006f400 0x7f1580071910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1594164e40 0x7f1594161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 --2- 172.21.15.83:0/842650485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15940cef10 0x7f15941623d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 >> 172.21.15.83:0/842650485 conn(0x7f15940b4250 msgr2=0x7f15940d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 shutdown_connections 2024-05-17T21:57:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7f15998e0640 1 -- 172.21.15.83:0/842650485 wait complete. 2024-05-17T21:57:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd3a40b0ad0 con 0x7fd3a406dec0 2024-05-17T21:57:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.405+0000 7fd3a0ff9640 1 -- 172.21.15.83:0/729210336 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd39c023070 con 0x7fd3a406dec0 2024-05-17T21:57:41.410 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2024-05-17T21:57:41.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 msgr2=0x7fd390071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 0x7fd390071940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3940b1000 tx=0x7fd3940a9000 comp rx=0 tx=0).stop 2024-05-17T21:57:41.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 msgr2=0x7fd3a41fcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00ba50 tx=0x7fd39c00bf10 comp rx=0 tx=0).stop 2024-05-17T21:57:41.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 shutdown_connections 2024-05-17T21:57:41.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3a41558f0 0x7fd3a40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd39006f430 0x7fd390071940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3a4155d50 0x7fd3a40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 --2- 172.21.15.83:0/729210336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a406dec0 0x7fd3a41fcad0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 >> 172.21.15.83:0/729210336 conn(0x7fd3a40d1cb0 msgr2=0x7fd3a415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-05-17T21:57:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 shutdown_connections 2024-05-17T21:57:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.421+0000 7fd3a956a640 1 -- 172.21.15.83:0/729210336 wait complete. 2024-05-17T21:57:41.414 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-05-17T21:57:41.423 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-05-17T21:57:41.423 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-05-17T21:57:41.431 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-05-17T21:57:41.432 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-05-17T21:57:41.433 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-05-17T21:57:41.433 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-05-17T21:57:41.440 DEBUG:teuthology.parallel:result is None 2024-05-17T21:57:41.440 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T21:57:41.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 Processor -- start 2024-05-17T21:57:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 -- start start 2024-05-17T21:57:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9ec06dfb0 0x7fa9ec167830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ec0d0030 con 0x7fa9ec06f5e0 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ec0b5670 con 0x7fa9ec0b1120 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9f18d2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ec0b5950 con 0x7fa9ec06dfb0 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eaffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa9ec06f5e0 0x7fa9ec0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33710/0 (socket says 172.21.15.83:33710) 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eaffd640 1 -- 172.21.15.83:0/3076871933 learned_addr learned my addr 172.21.15.83:0/3076871933 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eb7fe640 1 --2- 172.21.15.83:0/3076871933 >> v2:172.21.15.83:3300/0 conn(0x7fa9ec06dfb0 0x7fa9ec167830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9e9ffb640 1 -- 172.21.15.83:0/3076871933 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591102617 0 0) 0x7fa9ec0d0030 con 0x7fa9ec06f5e0 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9e9ffb640 1 -- 172.21.15.83:0/3076871933 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9d4003530 con 0x7fa9ec06f5e0 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/3076871933 >> v1:172.21.15.155:6789/0 conn(0x7fa9ec0b1120 legacy=0x7fa9ec06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/3076871933 >> v1:172.21.15.83:6789/0 conn(0x7fa9ec06f5e0 legacy=0x7fa9ec0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.625+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/3076871933 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec0b5c30 con 0x7fa9ec06dfb0 2024-05-17T21:57:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9eb7fe640 1 --2- 172.21.15.83:0/3076871933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec06dfb0 0x7fa9ec167830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009d90 tx=0x7fa9e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee7cde5ad60b570 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9e9ffb640 1 -- 172.21.15.83:0/3076871933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40b2070 con 0x7fa9ec06dfb0 2024-05-17T21:57:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9e9ffb640 1 -- 172.21.15.83:0/3076871933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e40a59c0 con 0x7fa9ec06dfb0 2024-05-17T21:57:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9e9ffb640 1 -- 172.21.15.83:0/3076871933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40a5d70 con 0x7fa9ec06dfb0 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 -- 172.21.15.83:0/3076871933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec06dfb0 msgr2=0x7fa9ec167830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/3076871933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec06dfb0 0x7fa9ec167830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009d90 tx=0x7fa9e401c8e0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 -- 172.21.15.83:0/3076871933 shutdown_connections 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/3076871933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec06dfb0 0x7fa9ec167830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 -- 172.21.15.83:0/3076871933 >> 172.21.15.83:0/3076871933 conn(0x7fa9ec0d1ca0 msgr2=0x7fa9ec0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 -- 172.21.15.83:0/3076871933 shutdown_connections 2024-05-17T21:57:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.629+0000 7fa9f18d2640 1 -- 172.21.15.83:0/3076871933 wait complete. 2024-05-17T21:57:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 Processor -- start 2024-05-17T21:57:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- start start 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9ec06f5e0 0x7fa9ec153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 0x7fa9ec0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec16af40 con 0x7fa9ec158b20 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec16adc0 con 0x7fa9ec0b1120 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec16b0c0 con 0x7fa9ec06f5e0 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39740/0 (socket says 172.21.15.83:39740) 2024-05-17T21:57:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/44031414 learned_addr learned my addr 172.21.15.83:0/44031414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eaffd640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9ec06f5e0 0x7fa9ec153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9ea7fc640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 0x7fa9ec0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9ec06f5e0 msgr2=0x7fa9ec153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9ec06f5e0 0x7fa9ec153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 msgr2=0x7fa9ec0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 0x7fa9ec0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 -- 172.21.15.83:0/44031414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec159470 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9ea7fc640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 0x7fa9ec0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9eb7fe640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40b0040 tx=0x7fa9e4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40b2050 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e40be070 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec153be0 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4004040 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec1561e0 con 0x7fa9ec158b20 2024-05-17T21:57:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.633+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ec0cf870 con 0x7fa9ec158b20 2024-05-17T21:57:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.637+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9e40041e0 con 0x7fa9ec158b20 2024-05-17T21:57:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.637+0000 7fa9d3fff640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 0x7fa9d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.637+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e412acc0 con 0x7fa9ec158b20 2024-05-17T21:57:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.637+0000 7fa9eaffd640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 0x7fa9d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.637+0000 7fa9eaffd640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 0x7fa9d8071940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007a70 tx=0x7fa9dc0079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.641+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa9ec159470 con 0x7fa9ec158b20 2024-05-17T21:57:41.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.785+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9ec068730 con 0x7fa9d806f430 2024-05-17T21:57:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.797+0000 7fa9d3fff640 1 -- 172.21.15.83:0/44031414 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fa9ec068730 con 0x7fa9d806f430 2024-05-17T21:57:41.790 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:41.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.797+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 msgr2=0x7fa9d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.797+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 0x7fa9d8071940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007a70 tx=0x7fa9dc0079c0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.797+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 msgr2=0x7fa9ec158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.797+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40b0040 tx=0x7fa9e4019680 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 shutdown_connections 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9ec158b20 0x7fa9ec158f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9d806f430 0x7fa9d8071940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9ec0b1120 0x7fa9ec0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 --2- 172.21.15.83:0/44031414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9ec06f5e0 0x7fa9ec153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 >> 172.21.15.83:0/44031414 conn(0x7fa9ec0d1ca0 msgr2=0x7fa9ec159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 shutdown_connections 2024-05-17T21:57:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.801+0000 7fa9f18d2640 1 -- 172.21.15.83:0/44031414 wait complete. 2024-05-17T21:57:41.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-17T21:57:40.968123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003112"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200427+0000","last_change":"2024-05-17T21:57:30.334206+0000","last_active":"2024-05-17T21:57:32.200427+0000","last_peered":"2024-05-17T21:57:32.200427+0000","last_clean":"2024-05-17T21:57:32.200427+0000","last_became_active":"2024-05-17T21:57:30.333625+0000","last_became_peered":"2024-05-17T21:57:30.333625+0000","last_unstale":"2024-05-17T21:57:32.200427+0000","last_undegraded":"2024-05-17T21:57:32.200427+0000","last_fullsized":"2024-05-17T21:57:32.200427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268827+0000","last_change":"2024-05-17T21:57:30.282372+0000","last_active":"2024-05-17T21:57:31.268827+0000","last_peered":"2024-05-17T21:57:31.268827+0000","last_clean":"2024-05-17T21:57:31.268827+0000","last_became_active":"2024-05-17T21:57:30.282085+0000","last_became_peered":"2024-05-17T21:57:30.282085+0000","last_unstale":"2024-05-17T21:57:31.268827+0000","last_undegraded":"2024-05-17T21:57:31.268827+0000","last_fullsized":"2024-05-17T21:57:31.268827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201211+0000","last_change":"2024-05-17T21:57:30.274388+0000","last_active":"2024-05-17T21:57:32.201211+0000","last_peered":"2024-05-17T21:57:32.201211+0000","last_clean":"2024-05-17T21:57:32.201211+0000","last_became_active":"2024-05-17T21:57:30.274079+0000","last_became_peered":"2024-05-17T21:57:30.274079+0000","last_unstale":"2024-05-17T21:57:32.201211+0000","last_undegraded":"2024-05-17T21:57:32.201211+0000","last_fullsized":"2024-05-17T21:57:32.201211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:10.441970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262187+0000","last_change":"2024-05-17T21:57:30.299694+0000","last_active":"2024-05-17T21:57:31.262187+0000","last_peered":"2024-05-17T21:57:31.262187+0000","last_clean":"2024-05-17T21:57:31.262187+0000","last_became_active":"2024-05-17T21:57:30.298884+0000","last_became_peered":"2024-05-17T21:57:30.298884+0000","last_unstale":"2024-05-17T21:57:31.262187+0000","last_undegraded":"2024-05-17T21:57:31.262187+0000","last_fullsized":"2024-05-17T21:57:31.262187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199587+0000","last_change":"2024-05-17T21:57:30.316073+0000","last_active":"2024-05-17T21:57:32.199587+0000","last_peered":"2024-05-17T21:57:32.199587+0000","last_clean":"2024-05-17T21:57:32.199587+0000","last_became_active":"2024-05-17T21:57:30.315439+0000","last_became_peered":"2024-05-17T21:57:30.315439+0000","last_unstale":"2024-05-17T21:57:32.199587+0000","last_undegraded":"2024-05-17T21:57:32.199587+0000","last_fullsized":"2024-05-17T21:57:32.199587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261267+0000","last_change":"2024-05-17T21:57:30.295598+0000","last_active":"2024-05-17T21:57:31.261267+0000","last_peered":"2024-05-17T21:57:31.261267+0000","last_clean":"2024-05-17T21:57:31.261267+0000","last_became_active":"2024-05-17T21:57:30.294722+0000","last_became_peered":"2024-05-17T21:57:30.294722+0000","last_unstale":"2024-05-17T21:57:31.261267+0000","last_undegraded":"2024-05-17T21:57:31.261267+0000","last_fullsized":"2024-05-17T21:57:31.261267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199251+0000","last_change":"2024-05-17T21:57:30.324100+0000","last_active":"2024-05-17T21:57:32.199251+0000","last_peered":"2024-05-17T21:57:32.199251+0000","last_clean":"2024-05-17T21:57:32.199251+0000","last_became_active":"2024-05-17T21:57:30.320444+0000","last_became_peered":"2024-05-17T21:57:30.320444+0000","last_unstale":"2024-05-17T21:57:32.199251+0000","last_undegraded":"2024-05-17T21:57:32.199251+0000","last_fullsized":"2024-05-17T21:57:32.199251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200972+0000","last_change":"2024-05-17T21:57:30.302526+0000","last_active":"2024-05-17T21:57:32.200972+0000","last_peered":"2024-05-17T21:57:32.200972+0000","last_clean":"2024-05-17T21:57:32.200972+0000","last_became_active":"2024-05-17T21:57:30.302136+0000","last_became_peered":"2024-05-17T21:57:30.302136+0000","last_unstale":"2024-05-17T21:57:32.200972+0000","last_undegraded":"2024-05-17T21:57:32.200972+0000","last_fullsized":"2024-05-17T21:57:32.200972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200607+0000","last_change":"2024-05-17T21:57:30.332944+0000","last_active":"2024-05-17T21:57:32.200607+0000","last_peered":"2024-05-17T21:57:32.200607+0000","last_clean":"2024-05-17T21:57:32.200607+0000","last_became_active":"2024-05-17T21:57:30.329174+0000","last_became_peered":"2024-05-17T21:57:30.329174+0000","last_unstale":"2024-05-17T21:57:32.200607+0000","last_undegraded":"2024-05-17T21:57:32.200607+0000","last_fullsized":"2024-05-17T21:57:32.200607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:10.454507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272598+0000","last_change":"2024-05-17T21:57:30.314980+0000","last_active":"2024-05-17T21:57:31.272598+0000","last_peered":"2024-05-17T21:57:31.272598+0000","last_clean":"2024-05-17T21:57:31.272598+0000","last_became_active":"2024-05-17T21:57:30.314085+0000","last_became_peered":"2024-05-17T21:57:30.314085+0000","last_unstale":"2024-05-17T21:57:31.272598+0000","last_undegraded":"2024-05-17T21:57:31.272598+0000","last_fullsized":"2024-05-17T21:57:31.272598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272161+0000","last_change":"2024-05-17T21:57:30.316071+0000","last_active":"2024-05-17T21:57:31.272161+0000","last_peered":"2024-05-17T21:57:31.272161+0000","last_clean":"2024-05-17T21:57:31.272161+0000","last_became_active":"2024-05-17T21:57:30.314772+0000","last_became_peered":"2024-05-17T21:57:30.314772+0000","last_unstale":"2024-05-17T21:57:31.272161+0000","last_undegraded":"2024-05-17T21:57:31.272161+0000","last_fullsized":"2024-05-17T21:57:31.272161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260695+0000","last_change":"2024-05-17T21:57:30.319379+0000","last_active":"2024-05-17T21:57:31.260695+0000","last_peered":"2024-05-17T21:57:31.260695+0000","last_clean":"2024-05-17T21:57:31.260695+0000","last_became_active":"2024-05-17T21:57:30.318510+0000","last_became_peered":"2024-05-17T21:57:30.318510+0000","last_unstale":"2024-05-17T21:57:31.260695+0000","last_undegraded":"2024-05-17T21:57:31.260695+0000","last_fullsized":"2024-05-17T21:57:31.260695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:22.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203239+0000","last_change":"2024-05-17T21:57:30.315381+0000","last_active":"2024-05-17T21:57:32.203239+0000","last_peered":"2024-05-17T21:57:32.203239+0000","last_clean":"2024-05-17T21:57:32.203239+0000","last_became_active":"2024-05-17T21:57:30.315076+0000","last_became_peered":"2024-05-17T21:57:30.315076+0000","last_unstale":"2024-05-17T21:57:32.203239+0000","last_undegraded":"2024-05-17T21:57:32.203239+0000","last_fullsized":"2024-05-17T21:57:32.203239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200631+0000","last_change":"2024-05-17T21:57:30.274361+0000","last_active":"2024-05-17T21:57:32.200631+0000","last_peered":"2024-05-17T21:57:32.200631+0000","last_clean":"2024-05-17T21:57:32.200631+0000","last_became_active":"2024-05-17T21:57:30.273916+0000","last_became_peered":"2024-05-17T21:57:30.273916+0000","last_unstale":"2024-05-17T21:57:32.200631+0000","last_undegraded":"2024-05-17T21:57:32.200631+0000","last_fullsized":"2024-05-17T21:57:32.200631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:47.494853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268312+0000","last_change":"2024-05-17T21:57:30.283163+0000","last_active":"2024-05-17T21:57:31.268312+0000","last_peered":"2024-05-17T21:57:31.268312+0000","last_clean":"2024-05-17T21:57:31.268312+0000","last_became_active":"2024-05-17T21:57:30.282704+0000","last_became_peered":"2024-05-17T21:57:30.282704+0000","last_unstale":"2024-05-17T21:57:31.268312+0000","last_undegraded":"2024-05-17T21:57:31.268312+0000","last_fullsized":"2024-05-17T21:57:31.268312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260004+0000","last_change":"2024-05-17T21:57:30.296030+0000","last_active":"2024-05-17T21:57:31.260004+0000","last_peered":"2024-05-17T21:57:31.260004+0000","last_clean":"2024-05-17T21:57:31.260004+0000","last_became_active":"2024-05-17T21:57:30.294696+0000","last_became_peered":"2024-05-17T21:57:30.294696+0000","last_unstale":"2024-05-17T21:57:31.260004+0000","last_undegraded":"2024-05-17T21:57:31.260004+0000","last_fullsized":"2024-05-17T21:57:31.260004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263697+0000","last_change":"2024-05-17T21:57:30.323553+0000","last_active":"2024-05-17T21:57:31.263697+0000","last_peered":"2024-05-17T21:57:31.263697+0000","last_clean":"2024-05-17T21:57:31.263697+0000","last_became_active":"2024-05-17T21:57:30.323019+0000","last_became_peered":"2024-05-17T21:57:30.323019+0000","last_unstale":"2024-05-17T21:57:31.263697+0000","last_undegraded":"2024-05-17T21:57:31.263697+0000","last_fullsized":"2024-05-17T21:57:31.263697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262779+0000","last_change":"2024-05-17T21:57:30.319719+0000","last_active":"2024-05-17T21:57:31.262779+0000","last_peered":"2024-05-17T21:57:31.262779+0000","last_clean":"2024-05-17T21:57:31.262779+0000","last_became_active":"2024-05-17T21:57:30.319335+0000","last_became_peered":"2024-05-17T21:57:30.319335+0000","last_unstale":"2024-05-17T21:57:31.262779+0000","last_undegraded":"2024-05-17T21:57:31.262779+0000","last_fullsized":"2024-05-17T21:57:31.262779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200408+0000","last_change":"2024-05-17T21:57:30.321777+0000","last_active":"2024-05-17T21:57:32.200408+0000","last_peered":"2024-05-17T21:57:32.200408+0000","last_clean":"2024-05-17T21:57:32.200408+0000","last_became_active":"2024-05-17T21:57:30.321011+0000","last_became_peered":"2024-05-17T21:57:30.321011+0000","last_unstale":"2024-05-17T21:57:32.200408+0000","last_undegraded":"2024-05-17T21:57:32.200408+0000","last_fullsized":"2024-05-17T21:57:32.200408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272173+0000","last_change":"2024-05-17T21:57:30.314965+0000","last_active":"2024-05-17T21:57:31.272173+0000","last_peered":"2024-05-17T21:57:31.272173+0000","last_clean":"2024-05-17T21:57:31.272173+0000","last_became_active":"2024-05-17T21:57:30.314249+0000","last_became_peered":"2024-05-17T21:57:30.314249+0000","last_unstale":"2024-05-17T21:57:31.272173+0000","last_undegraded":"2024-05-17T21:57:31.272173+0000","last_fullsized":"2024-05-17T21:57:31.272173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201572+0000","last_change":"2024-05-17T21:57:30.318172+0000","last_active":"2024-05-17T21:57:32.201572+0000","last_peered":"2024-05-17T21:57:32.201572+0000","last_clean":"2024-05-17T21:57:32.201572+0000","last_became_active":"2024-05-17T21:57:30.317761+0000","last_became_peered":"2024-05-17T21:57:30.317761+0000","last_unstale":"2024-05-17T21:57:32.201572+0000","last_undegraded":"2024-05-17T21:57:32.201572+0000","last_fullsized":"2024-05-17T21:57:32.201572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262717+0000","last_change":"2024-05-17T21:57:30.320885+0000","last_active":"2024-05-17T21:57:31.262717+0000","last_peered":"2024-05-17T21:57:31.262717+0000","last_clean":"2024-05-17T21:57:31.262717+0000","last_became_active":"2024-05-17T21:57:30.320268+0000","last_became_peered":"2024-05-17T21:57:30.320268+0000","last_unstale":"2024-05-17T21:57:31.262717+0000","last_undegraded":"2024-05-17T21:57:31.262717+0000","last_fullsized":"2024-05-17T21:57:31.262717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201655+0000","last_change":"2024-05-17T21:57:30.306877+0000","last_active":"2024-05-17T21:57:32.201655+0000","last_peered":"2024-05-17T21:57:32.201655+0000","last_clean":"2024-05-17T21:57:32.201655+0000","last_became_active":"2024-05-17T21:57:30.306541+0000","last_became_peered":"2024-05-17T21:57:30.306541+0000","last_unstale":"2024-05-17T21:57:32.201655+0000","last_undegraded":"2024-05-17T21:57:32.201655+0000","last_fullsized":"2024-05-17T21:57:32.201655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260191+0000","last_change":"2024-05-17T21:57:30.285039+0000","last_active":"2024-05-17T21:57:31.260191+0000","last_peered":"2024-05-17T21:57:31.260191+0000","last_clean":"2024-05-17T21:57:31.260191+0000","last_became_active":"2024-05-17T21:57:30.284132+0000","last_became_peered":"2024-05-17T21:57:30.284132+0000","last_unstale":"2024-05-17T21:57:31.260191+0000","last_undegraded":"2024-05-17T21:57:31.260191+0000","last_fullsized":"2024-05-17T21:57:31.260191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202612+0000","last_change":"2024-05-17T21:57:30.285554+0000","last_active":"2024-05-17T21:57:32.202612+0000","last_peered":"2024-05-17T21:57:32.202612+0000","last_clean":"2024-05-17T21:57:32.202612+0000","last_became_active":"2024-05-17T21:57:30.285106+0000","last_became_peered":"2024-05-17T21:57:30.285106+0000","last_unstale":"2024-05-17T21:57:32.202612+0000","last_undegraded":"2024-05-17T21:57:32.202612+0000","last_fullsized":"2024-05-17T21:57:32.202612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272595+0000","last_change":"2024-05-17T21:57:30.318707+0000","last_active":"2024-05-17T21:57:31.272595+0000","last_peered":"2024-05-17T21:57:31.272595+0000","last_clean":"2024-05-17T21:57:31.272595+0000","last_became_active":"2024-05-17T21:57:30.317977+0000","last_became_peered":"2024-05-17T21:57:30.317977+0000","last_unstale":"2024-05-17T21:57:31.272595+0000","last_undegraded":"2024-05-17T21:57:31.272595+0000","last_fullsized":"2024-05-17T21:57:31.272595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.269288+0000","last_change":"2024-05-17T21:57:30.279948+0000","last_active":"2024-05-17T21:57:31.269288+0000","last_peered":"2024-05-17T21:57:31.269288+0000","last_clean":"2024-05-17T21:57:31.269288+0000","last_became_active":"2024-05-17T21:57:30.279346+0000","last_became_peered":"2024-05-17T21:57:30.279346+0000","last_unstale":"2024-05-17T21:57:31.269288+0000","last_undegraded":"2024-05-17T21:57:31.269288+0000","last_fullsized":"2024-05-17T21:57:31.269288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203266+0000","last_change":"2024-05-17T21:57:30.289588+0000","last_active":"2024-05-17T21:57:32.203266+0000","last_peered":"2024-05-17T21:57:32.203266+0000","last_clean":"2024-05-17T21:57:32.203266+0000","last_became_active":"2024-05-17T21:57:30.289189+0000","last_became_peered":"2024-05-17T21:57:30.289189+0000","last_unstale":"2024-05-17T21:57:32.203266+0000","last_undegraded":"2024-05-17T21:57:32.203266+0000","last_fullsized":"2024-05-17T21:57:32.203266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202018+0000","last_change":"2024-05-17T21:57:30.282815+0000","last_active":"2024-05-17T21:57:32.202018+0000","last_peered":"2024-05-17T21:57:32.202018+0000","last_clean":"2024-05-17T21:57:32.202018+0000","last_became_active":"2024-05-17T21:57:30.282056+0000","last_became_peered":"2024-05-17T21:57:30.282056+0000","last_unstale":"2024-05-17T21:57:32.202018+0000","last_undegraded":"2024-05-17T21:57:32.202018+0000","last_fullsized":"2024-05-17T21:57:32.202018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261817+0000","last_change":"2024-05-17T21:57:30.281437+0000","last_active":"2024-05-17T21:57:31.261817+0000","last_peered":"2024-05-17T21:57:31.261817+0000","last_clean":"2024-05-17T21:57:31.261817+0000","last_became_active":"2024-05-17T21:57:30.280235+0000","last_became_peered":"2024-05-17T21:57:30.280235+0000","last_unstale":"2024-05-17T21:57:31.261817+0000","last_undegraded":"2024-05-17T21:57:31.261817+0000","last_fullsized":"2024-05-17T21:57:31.261817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262790+0000","last_change":"2024-05-17T21:57:30.274855+0000","last_active":"2024-05-17T21:57:31.262790+0000","last_peered":"2024-05-17T21:57:31.262790+0000","last_clean":"2024-05-17T21:57:31.262790+0000","last_became_active":"2024-05-17T21:57:30.274300+0000","last_became_peered":"2024-05-17T21:57:30.274300+0000","last_unstale":"2024-05-17T21:57:31.262790+0000","last_undegraded":"2024-05-17T21:57:31.262790+0000","last_fullsized":"2024-05-17T21:57:31.262790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:33.958058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271627+0000","last_change":"2024-05-17T21:57:30.305394+0000","last_active":"2024-05-17T21:57:31.271627+0000","last_peered":"2024-05-17T21:57:31.271627+0000","last_clean":"2024-05-17T21:57:31.271627+0000","last_became_active":"2024-05-17T21:57:30.304545+0000","last_became_peered":"2024-05-17T21:57:30.304545+0000","last_unstale":"2024-05-17T21:57:31.271627+0000","last_undegraded":"2024-05-17T21:57:31.271627+0000","last_fullsized":"2024-05-17T21:57:31.271627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260730+0000","last_change":"2024-05-17T21:57:30.271395+0000","last_active":"2024-05-17T21:57:31.260730+0000","last_peered":"2024-05-17T21:57:31.260730+0000","last_clean":"2024-05-17T21:57:31.260730+0000","last_became_active":"2024-05-17T21:57:30.270232+0000","last_became_peered":"2024-05-17T21:57:30.270232+0000","last_unstale":"2024-05-17T21:57:31.260730+0000","last_undegraded":"2024-05-17T21:57:31.260730+0000","last_fullsized":"2024-05-17T21:57:31.260730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:33.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203332+0000","last_change":"2024-05-17T21:57:29.306016+0000","last_active":"2024-05-17T21:57:32.203332+0000","last_peered":"2024-05-17T21:57:32.203332+0000","last_clean":"2024-05-17T21:57:32.203332+0000","last_became_active":"2024-05-17T21:57:29.305576+0000","last_became_peered":"2024-05-17T21:57:29.305576+0000","last_unstale":"2024-05-17T21:57:32.203332+0000","last_undegraded":"2024-05-17T21:57:32.203332+0000","last_fullsized":"2024-05-17T21:57:32.203332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202752+0000","last_change":"2024-05-17T21:57:30.283060+0000","last_active":"2024-05-17T21:57:32.202752+0000","last_peered":"2024-05-17T21:57:32.202752+0000","last_clean":"2024-05-17T21:57:32.202752+0000","last_became_active":"2024-05-17T21:57:30.282526+0000","last_became_peered":"2024-05-17T21:57:30.282526+0000","last_unstale":"2024-05-17T21:57:32.202752+0000","last_undegraded":"2024-05-17T21:57:32.202752+0000","last_fullsized":"2024-05-17T21:57:32.202752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271871+0000","last_change":"2024-05-17T21:57:29.299174+0000","last_active":"2024-05-17T21:57:31.271871+0000","last_peered":"2024-05-17T21:57:31.271871+0000","last_clean":"2024-05-17T21:57:31.271871+0000","last_became_active":"2024-05-17T21:57:29.297800+0000","last_became_peered":"2024-05-17T21:57:29.297800+0000","last_unstale":"2024-05-17T21:57:31.271871+0000","last_undegraded":"2024-05-17T21:57:31.271871+0000","last_fullsized":"2024-05-17T21:57:31.271871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202014+0000","last_change":"2024-05-17T21:57:29.290665+0000","last_active":"2024-05-17T21:57:32.202014+0000","last_peered":"2024-05-17T21:57:32.202014+0000","last_clean":"2024-05-17T21:57:32.202014+0000","last_became_active":"2024-05-17T21:57:29.290298+0000","last_became_peered":"2024-05-17T21:57:29.290298+0000","last_unstale":"2024-05-17T21:57:32.202014+0000","last_undegraded":"2024-05-17T21:57:32.202014+0000","last_fullsized":"2024-05-17T21:57:32.202014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201127+0000","last_change":"2024-05-17T21:57:30.283851+0000","last_active":"2024-05-17T21:57:32.201127+0000","last_peered":"2024-05-17T21:57:32.201127+0000","last_clean":"2024-05-17T21:57:32.201127+0000","last_became_active":"2024-05-17T21:57:30.283547+0000","last_became_peered":"2024-05-17T21:57:30.283547+0000","last_unstale":"2024-05-17T21:57:32.201127+0000","last_undegraded":"2024-05-17T21:57:32.201127+0000","last_fullsized":"2024-05-17T21:57:32.201127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260166+0000","last_change":"2024-05-17T21:57:29.297693+0000","last_active":"2024-05-17T21:57:31.260166+0000","last_peered":"2024-05-17T21:57:31.260166+0000","last_clean":"2024-05-17T21:57:31.260166+0000","last_became_active":"2024-05-17T21:57:29.296843+0000","last_became_peered":"2024-05-17T21:57:29.296843+0000","last_unstale":"2024-05-17T21:57:31.260166+0000","last_undegraded":"2024-05-17T21:57:31.260166+0000","last_fullsized":"2024-05-17T21:57:31.260166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268162+0000","last_change":"2024-05-17T21:57:30.278227+0000","last_active":"2024-05-17T21:57:31.268162+0000","last_peered":"2024-05-17T21:57:31.268162+0000","last_clean":"2024-05-17T21:57:31.268162+0000","last_became_active":"2024-05-17T21:57:30.277784+0000","last_became_peered":"2024-05-17T21:57:30.277784+0000","last_unstale":"2024-05-17T21:57:31.268162+0000","last_undegraded":"2024-05-17T21:57:31.268162+0000","last_fullsized":"2024-05-17T21:57:31.268162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271960+0000","last_change":"2024-05-17T21:57:29.290141+0000","last_active":"2024-05-17T21:57:31.271960+0000","last_peered":"2024-05-17T21:57:31.271960+0000","last_clean":"2024-05-17T21:57:31.271960+0000","last_became_active":"2024-05-17T21:57:29.288975+0000","last_became_peered":"2024-05-17T21:57:29.288975+0000","last_unstale":"2024-05-17T21:57:31.271960+0000","last_undegraded":"2024-05-17T21:57:31.271960+0000","last_fullsized":"2024-05-17T21:57:31.271960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262047+0000","last_change":"2024-05-17T21:57:30.322244+0000","last_active":"2024-05-17T21:57:31.262047+0000","last_peered":"2024-05-17T21:57:31.262047+0000","last_clean":"2024-05-17T21:57:31.262047+0000","last_became_active":"2024-05-17T21:57:30.321701+0000","last_became_peered":"2024-05-17T21:57:30.321701+0000","last_unstale":"2024-05-17T21:57:31.262047+0000","last_undegraded":"2024-05-17T21:57:31.262047+0000","last_fullsized":"2024-05-17T21:57:31.262047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262204+0000","last_change":"2024-05-17T21:57:29.293457+0000","last_active":"2024-05-17T21:57:31.262204+0000","last_peered":"2024-05-17T21:57:31.262204+0000","last_clean":"2024-05-17T21:57:31.262204+0000","last_became_active":"2024-05-17T21:57:29.292418+0000","last_became_peered":"2024-05-17T21:57:29.292418+0000","last_unstale":"2024-05-17T21:57:31.262204+0000","last_undegraded":"2024-05-17T21:57:31.262204+0000","last_fullsized":"2024-05-17T21:57:31.262204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202960+0000","last_change":"2024-05-17T21:57:30.277503+0000","last_active":"2024-05-17T21:57:32.202960+0000","last_peered":"2024-05-17T21:57:32.202960+0000","last_clean":"2024-05-17T21:57:32.202960+0000","last_became_active":"2024-05-17T21:57:30.276765+0000","last_became_peered":"2024-05-17T21:57:30.276765+0000","last_unstale":"2024-05-17T21:57:32.202960+0000","last_undegraded":"2024-05-17T21:57:32.202960+0000","last_fullsized":"2024-05-17T21:57:32.202960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268442+0000","last_change":"2024-05-17T21:57:29.284139+0000","last_active":"2024-05-17T21:57:31.268442+0000","last_peered":"2024-05-17T21:57:31.268442+0000","last_clean":"2024-05-17T21:57:31.268442+0000","last_became_active":"2024-05-17T21:57:29.283786+0000","last_became_peered":"2024-05-17T21:57:29.283786+0000","last_unstale":"2024-05-17T21:57:31.268442+0000","last_undegraded":"2024-05-17T21:57:31.268442+0000","last_fullsized":"2024-05-17T21:57:31.268442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260839+0000","last_change":"2024-05-17T21:57:30.322236+0000","last_active":"2024-05-17T21:57:31.260839+0000","last_peered":"2024-05-17T21:57:31.260839+0000","last_clean":"2024-05-17T21:57:31.260839+0000","last_became_active":"2024-05-17T21:57:30.321651+0000","last_became_peered":"2024-05-17T21:57:30.321651+0000","last_unstale":"2024-05-17T21:57:31.260839+0000","last_undegraded":"2024-05-17T21:57:31.260839+0000","last_fullsized":"2024-05-17T21:57:31.260839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:11.129074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271878+0000","last_change":"2024-05-17T21:57:29.286191+0000","last_active":"2024-05-17T21:57:31.271878+0000","last_peered":"2024-05-17T21:57:31.271878+0000","last_clean":"2024-05-17T21:57:31.271878+0000","last_became_active":"2024-05-17T21:57:29.285202+0000","last_became_peered":"2024-05-17T21:57:29.285202+0000","last_unstale":"2024-05-17T21:57:31.271878+0000","last_undegraded":"2024-05-17T21:57:31.271878+0000","last_fullsized":"2024-05-17T21:57:31.271878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199606+0000","last_change":"2024-05-17T21:57:30.323488+0000","last_active":"2024-05-17T21:57:32.199606+0000","last_peered":"2024-05-17T21:57:32.199606+0000","last_clean":"2024-05-17T21:57:32.199606+0000","last_became_active":"2024-05-17T21:57:30.322790+0000","last_became_peered":"2024-05-17T21:57:30.322790+0000","last_unstale":"2024-05-17T21:57:32.199606+0000","last_undegraded":"2024-05-17T21:57:32.199606+0000","last_fullsized":"2024-05-17T21:57:32.199606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202865+0000","last_change":"2024-05-17T21:57:29.298078+0000","last_active":"2024-05-17T21:57:32.202865+0000","last_peered":"2024-05-17T21:57:32.202865+0000","last_clean":"2024-05-17T21:57:32.202865+0000","last_became_active":"2024-05-17T21:57:29.297807+0000","last_became_peered":"2024-05-17T21:57:29.297807+0000","last_unstale":"2024-05-17T21:57:32.202865+0000","last_undegraded":"2024-05-17T21:57:32.202865+0000","last_fullsized":"2024-05-17T21:57:32.202865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:24:33.244279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201096+0000","last_change":"2024-05-17T21:57:30.318056+0000","last_active":"2024-05-17T21:57:32.201096+0000","last_peered":"2024-05-17T21:57:32.201096+0000","last_clean":"2024-05-17T21:57:32.201096+0000","last_became_active":"2024-05-17T21:57:30.317771+0000","last_became_peered":"2024-05-17T21:57:30.317771+0000","last_unstale":"2024-05-17T21:57:32.201096+0000","last_undegraded":"2024-05-17T21:57:32.201096+0000","last_fullsized":"2024-05-17T21:57:32.201096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:42.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261966+0000","last_change":"2024-05-17T21:57:29.284218+0000","last_active":"2024-05-17T21:57:31.261966+0000","last_peered":"2024-05-17T21:57:31.261966+0000","last_clean":"2024-05-17T21:57:31.261966+0000","last_became_active":"2024-05-17T21:57:29.283386+0000","last_became_peered":"2024-05-17T21:57:29.283386+0000","last_unstale":"2024-05-17T21:57:31.261966+0000","last_undegraded":"2024-05-17T21:57:31.261966+0000","last_fullsized":"2024-05-17T21:57:31.261966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200000+0000","last_change":"2024-05-17T21:57:30.321543+0000","last_active":"2024-05-17T21:57:32.200000+0000","last_peered":"2024-05-17T21:57:32.200000+0000","last_clean":"2024-05-17T21:57:32.200000+0000","last_became_active":"2024-05-17T21:57:30.320566+0000","last_became_peered":"2024-05-17T21:57:30.320566+0000","last_unstale":"2024-05-17T21:57:32.200000+0000","last_undegraded":"2024-05-17T21:57:32.200000+0000","last_fullsized":"2024-05-17T21:57:32.200000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271412+0000","last_change":"2024-05-17T21:57:29.297525+0000","last_active":"2024-05-17T21:57:31.271412+0000","last_peered":"2024-05-17T21:57:31.271412+0000","last_clean":"2024-05-17T21:57:31.271412+0000","last_became_active":"2024-05-17T21:57:29.296129+0000","last_became_peered":"2024-05-17T21:57:29.296129+0000","last_unstale":"2024-05-17T21:57:31.271412+0000","last_undegraded":"2024-05-17T21:57:31.271412+0000","last_fullsized":"2024-05-17T21:57:31.271412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271732+0000","last_change":"2024-05-17T21:57:30.306121+0000","last_active":"2024-05-17T21:57:31.271732+0000","last_peered":"2024-05-17T21:57:31.271732+0000","last_clean":"2024-05-17T21:57:31.271732+0000","last_became_active":"2024-05-17T21:57:30.304496+0000","last_became_peered":"2024-05-17T21:57:30.304496+0000","last_unstale":"2024-05-17T21:57:31.271732+0000","last_undegraded":"2024-05-17T21:57:31.271732+0000","last_fullsized":"2024-05-17T21:57:31.271732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262030+0000","last_change":"2024-05-17T21:57:29.273347+0000","last_active":"2024-05-17T21:57:31.262030+0000","last_peered":"2024-05-17T21:57:31.262030+0000","last_clean":"2024-05-17T21:57:31.262030+0000","last_became_active":"2024-05-17T21:57:29.272913+0000","last_became_peered":"2024-05-17T21:57:29.272913+0000","last_unstale":"2024-05-17T21:57:31.262030+0000","last_undegraded":"2024-05-17T21:57:31.262030+0000","last_fullsized":"2024-05-17T21:57:31.262030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:34:49.617602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199328+0000","last_change":"2024-05-17T21:57:30.322675+0000","last_active":"2024-05-17T21:57:32.199328+0000","last_peered":"2024-05-17T21:57:32.199328+0000","last_clean":"2024-05-17T21:57:32.199328+0000","last_became_active":"2024-05-17T21:57:30.321924+0000","last_became_peered":"2024-05-17T21:57:30.321924+0000","last_unstale":"2024-05-17T21:57:32.199328+0000","last_undegraded":"2024-05-17T21:57:32.199328+0000","last_fullsized":"2024-05-17T21:57:32.199328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262737+0000","last_change":"2024-05-17T21:57:29.294110+0000","last_active":"2024-05-17T21:57:31.262737+0000","last_peered":"2024-05-17T21:57:31.262737+0000","last_clean":"2024-05-17T21:57:31.262737+0000","last_became_active":"2024-05-17T21:57:29.293170+0000","last_became_peered":"2024-05-17T21:57:29.293170+0000","last_unstale":"2024-05-17T21:57:31.262737+0000","last_undegraded":"2024-05-17T21:57:31.262737+0000","last_fullsized":"2024-05-17T21:57:31.262737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:28:34.887309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200251+0000","last_change":"2024-05-17T21:57:30.315778+0000","last_active":"2024-05-17T21:57:32.200251+0000","last_peered":"2024-05-17T21:57:32.200251+0000","last_clean":"2024-05-17T21:57:32.200251+0000","last_became_active":"2024-05-17T21:57:30.314758+0000","last_became_peered":"2024-05-17T21:57:30.314758+0000","last_unstale":"2024-05-17T21:57:32.200251+0000","last_undegraded":"2024-05-17T21:57:32.200251+0000","last_fullsized":"2024-05-17T21:57:32.200251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263385+0000","last_change":"2024-05-17T21:57:29.281454+0000","last_active":"2024-05-17T21:57:31.263385+0000","last_peered":"2024-05-17T21:57:31.263385+0000","last_clean":"2024-05-17T21:57:31.263385+0000","last_became_active":"2024-05-17T21:57:29.280983+0000","last_became_peered":"2024-05-17T21:57:29.280983+0000","last_unstale":"2024-05-17T21:57:31.263385+0000","last_undegraded":"2024-05-17T21:57:31.263385+0000","last_fullsized":"2024-05-17T21:57:31.263385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200703+0000","last_change":"2024-05-17T21:57:30.334810+0000","last_active":"2024-05-17T21:57:32.200703+0000","last_peered":"2024-05-17T21:57:32.200703+0000","last_clean":"2024-05-17T21:57:32.200703+0000","last_became_active":"2024-05-17T21:57:30.332951+0000","last_became_peered":"2024-05-17T21:57:30.332951+0000","last_unstale":"2024-05-17T21:57:32.200703+0000","last_undegraded":"2024-05-17T21:57:32.200703+0000","last_fullsized":"2024-05-17T21:57:32.200703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:52.659720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268772+0000","last_change":"2024-05-17T21:57:29.296523+0000","last_active":"2024-05-17T21:57:31.268772+0000","last_peered":"2024-05-17T21:57:31.268772+0000","last_clean":"2024-05-17T21:57:31.268772+0000","last_became_active":"2024-05-17T21:57:29.296297+0000","last_became_peered":"2024-05-17T21:57:29.296297+0000","last_unstale":"2024-05-17T21:57:31.268772+0000","last_undegraded":"2024-05-17T21:57:31.268772+0000","last_fullsized":"2024-05-17T21:57:31.268772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263237+0000","last_change":"2024-05-17T21:57:30.297086+0000","last_active":"2024-05-17T21:57:31.263237+0000","last_peered":"2024-05-17T21:57:31.263237+0000","last_clean":"2024-05-17T21:57:31.263237+0000","last_became_active":"2024-05-17T21:57:30.296276+0000","last_became_peered":"2024-05-17T21:57:30.296276+0000","last_unstale":"2024-05-17T21:57:31.263237+0000","last_undegraded":"2024-05-17T21:57:31.263237+0000","last_fullsized":"2024-05-17T21:57:31.263237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262701+0000","last_change":"2024-05-17T21:57:29.282987+0000","last_active":"2024-05-17T21:57:31.262701+0000","last_peered":"2024-05-17T21:57:31.262701+0000","last_clean":"2024-05-17T21:57:31.262701+0000","last_became_active":"2024-05-17T21:57:29.282594+0000","last_became_peered":"2024-05-17T21:57:29.282594+0000","last_unstale":"2024-05-17T21:57:31.262701+0000","last_undegraded":"2024-05-17T21:57:31.262701+0000","last_fullsized":"2024-05-17T21:57:31.262701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200040+0000","last_change":"2024-05-17T21:57:30.328305+0000","last_active":"2024-05-17T21:57:32.200040+0000","last_peered":"2024-05-17T21:57:32.200040+0000","last_clean":"2024-05-17T21:57:32.200040+0000","last_became_active":"2024-05-17T21:57:30.324651+0000","last_became_peered":"2024-05-17T21:57:30.324651+0000","last_unstale":"2024-05-17T21:57:32.200040+0000","last_undegraded":"2024-05-17T21:57:32.200040+0000","last_fullsized":"2024-05-17T21:57:32.200040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199220+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:32.199220+0000","last_peered":"2024-05-17T21:57:32.199220+0000","last_clean":"2024-05-17T21:57:32.199220+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:32.199220+0000","last_undegraded":"2024-05-17T21:57:32.199220+0000","last_fullsized":"2024-05-17T21:57:32.199220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261146+0000","last_change":"2024-05-17T21:57:29.277871+0000","last_active":"2024-05-17T21:57:31.261146+0000","last_peered":"2024-05-17T21:57:31.261146+0000","last_clean":"2024-05-17T21:57:31.261146+0000","last_became_active":"2024-05-17T21:57:29.276986+0000","last_became_peered":"2024-05-17T21:57:29.276986+0000","last_unstale":"2024-05-17T21:57:31.261146+0000","last_undegraded":"2024-05-17T21:57:31.261146+0000","last_fullsized":"2024-05-17T21:57:31.261146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201481+0000","last_change":"2024-05-17T21:57:30.281160+0000","last_active":"2024-05-17T21:57:32.201481+0000","last_peered":"2024-05-17T21:57:32.201481+0000","last_clean":"2024-05-17T21:57:32.201481+0000","last_became_active":"2024-05-17T21:57:30.280004+0000","last_became_peered":"2024-05-17T21:57:30.280004+0000","last_unstale":"2024-05-17T21:57:32.201481+0000","last_undegraded":"2024-05-17T21:57:32.201481+0000","last_fullsized":"2024-05-17T21:57:32.201481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198887+0000","last_change":"2024-05-17T21:57:29.308082+0000","last_active":"2024-05-17T21:57:32.198887+0000","last_peered":"2024-05-17T21:57:32.198887+0000","last_clean":"2024-05-17T21:57:32.198887+0000","last_became_active":"2024-05-17T21:57:29.306695+0000","last_became_peered":"2024-05-17T21:57:29.306695+0000","last_unstale":"2024-05-17T21:57:32.198887+0000","last_undegraded":"2024-05-17T21:57:32.198887+0000","last_fullsized":"2024-05-17T21:57:32.198887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200656+0000","last_change":"2024-05-17T21:57:30.281759+0000","last_active":"2024-05-17T21:57:32.200656+0000","last_peered":"2024-05-17T21:57:32.200656+0000","last_clean":"2024-05-17T21:57:32.200656+0000","last_became_active":"2024-05-17T21:57:30.281449+0000","last_became_peered":"2024-05-17T21:57:30.281449+0000","last_unstale":"2024-05-17T21:57:32.200656+0000","last_undegraded":"2024-05-17T21:57:32.200656+0000","last_fullsized":"2024-05-17T21:57:32.200656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:53:58.249441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262812+0000","last_change":"2024-05-17T21:57:29.280664+0000","last_active":"2024-05-17T21:57:31.262812+0000","last_peered":"2024-05-17T21:57:31.262812+0000","last_clean":"2024-05-17T21:57:31.262812+0000","last_became_active":"2024-05-17T21:57:29.280118+0000","last_became_peered":"2024-05-17T21:57:29.280118+0000","last_unstale":"2024-05-17T21:57:31.262812+0000","last_undegraded":"2024-05-17T21:57:31.262812+0000","last_fullsized":"2024-05-17T21:57:31.262812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268285+0000","last_change":"2024-05-17T21:57:30.287528+0000","last_active":"2024-05-17T21:57:31.268285+0000","last_peered":"2024-05-17T21:57:31.268285+0000","last_clean":"2024-05-17T21:57:31.268285+0000","last_became_active":"2024-05-17T21:57:30.285405+0000","last_became_peered":"2024-05-17T21:57:30.285405+0000","last_unstale":"2024-05-17T21:57:31.268285+0000","last_undegraded":"2024-05-17T21:57:31.268285+0000","last_fullsized":"2024-05-17T21:57:31.268285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262287+0000","last_change":"2024-05-17T21:57:29.281356+0000","last_active":"2024-05-17T21:57:31.262287+0000","last_peered":"2024-05-17T21:57:31.262287+0000","last_clean":"2024-05-17T21:57:31.262287+0000","last_became_active":"2024-05-17T21:57:29.279906+0000","last_became_peered":"2024-05-17T21:57:29.279906+0000","last_unstale":"2024-05-17T21:57:31.262287+0000","last_undegraded":"2024-05-17T21:57:31.262287+0000","last_fullsized":"2024-05-17T21:57:31.262287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261918+0000","last_change":"2024-05-17T21:57:30.277518+0000","last_active":"2024-05-17T21:57:31.261918+0000","last_peered":"2024-05-17T21:57:31.261918+0000","last_clean":"2024-05-17T21:57:31.261918+0000","last_became_active":"2024-05-17T21:57:30.275892+0000","last_became_peered":"2024-05-17T21:57:30.275892+0000","last_unstale":"2024-05-17T21:57:31.261918+0000","last_undegraded":"2024-05-17T21:57:31.261918+0000","last_fullsized":"2024-05-17T21:57:31.261918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268304+0000","last_change":"2024-05-17T21:57:29.270571+0000","last_active":"2024-05-17T21:57:31.268304+0000","last_peered":"2024-05-17T21:57:31.268304+0000","last_clean":"2024-05-17T21:57:31.268304+0000","last_became_active":"2024-05-17T21:57:29.270219+0000","last_became_peered":"2024-05-17T21:57:29.270219+0000","last_unstale":"2024-05-17T21:57:31.268304+0000","last_undegraded":"2024-05-17T21:57:31.268304+0000","last_fullsized":"2024-05-17T21:57:31.268304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262482+0000","last_change":"2024-05-17T21:57:30.281497+0000","last_active":"2024-05-17T21:57:31.262482+0000","last_peered":"2024-05-17T21:57:31.262482+0000","last_clean":"2024-05-17T21:57:31.262482+0000","last_became_active":"2024-05-17T21:57:30.280473+0000","last_became_peered":"2024-05-17T21:57:30.280473+0000","last_unstale":"2024-05-17T21:57:31.262482+0000","last_undegraded":"2024-05-17T21:57:31.262482+0000","last_fullsized":"2024-05-17T21:57:31.262482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200226+0000","last_change":"2024-05-17T21:57:29.310998+0000","last_active":"2024-05-17T21:57:32.200226+0000","last_peered":"2024-05-17T21:57:32.200226+0000","last_clean":"2024-05-17T21:57:32.200226+0000","last_became_active":"2024-05-17T21:57:29.310445+0000","last_became_peered":"2024-05-17T21:57:29.310445+0000","last_unstale":"2024-05-17T21:57:32.200226+0000","last_undegraded":"2024-05-17T21:57:32.200226+0000","last_fullsized":"2024-05-17T21:57:32.200226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.270156+0000","last_change":"2024-05-17T21:57:30.286938+0000","last_active":"2024-05-17T21:57:31.270156+0000","last_peered":"2024-05-17T21:57:31.270156+0000","last_clean":"2024-05-17T21:57:31.270156+0000","last_became_active":"2024-05-17T21:57:30.286141+0000","last_became_peered":"2024-05-17T21:57:30.286141+0000","last_unstale":"2024-05-17T21:57:31.270156+0000","last_undegraded":"2024-05-17T21:57:31.270156+0000","last_fullsized":"2024-05-17T21:57:31.270156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268967+0000","last_change":"2024-05-17T21:57:29.291111+0000","last_active":"2024-05-17T21:57:31.268967+0000","last_peered":"2024-05-17T21:57:31.268967+0000","last_clean":"2024-05-17T21:57:31.268967+0000","last_became_active":"2024-05-17T21:57:29.290818+0000","last_became_peered":"2024-05-17T21:57:29.290818+0000","last_unstale":"2024-05-17T21:57:31.268967+0000","last_undegraded":"2024-05-17T21:57:31.268967+0000","last_fullsized":"2024-05-17T21:57:31.268967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202605+0000","last_change":"2024-05-17T21:57:30.321709+0000","last_active":"2024-05-17T21:57:32.202605+0000","last_peered":"2024-05-17T21:57:32.202605+0000","last_clean":"2024-05-17T21:57:32.202605+0000","last_became_active":"2024-05-17T21:57:30.321312+0000","last_became_peered":"2024-05-17T21:57:30.321312+0000","last_unstale":"2024-05-17T21:57:32.202605+0000","last_undegraded":"2024-05-17T21:57:32.202605+0000","last_fullsized":"2024-05-17T21:57:32.202605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272058+0000","last_change":"2024-05-17T21:57:29.310186+0000","last_active":"2024-05-17T21:57:31.272058+0000","last_peered":"2024-05-17T21:57:31.272058+0000","last_clean":"2024-05-17T21:57:31.272058+0000","last_became_active":"2024-05-17T21:57:29.309480+0000","last_became_peered":"2024-05-17T21:57:29.309480+0000","last_unstale":"2024-05-17T21:57:31.272058+0000","last_undegraded":"2024-05-17T21:57:31.272058+0000","last_fullsized":"2024-05-17T21:57:31.272058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202172+0000","last_change":"2024-05-17T21:57:30.276347+0000","last_active":"2024-05-17T21:57:32.202172+0000","last_peered":"2024-05-17T21:57:32.202172+0000","last_clean":"2024-05-17T21:57:32.202172+0000","last_became_active":"2024-05-17T21:57:30.275932+0000","last_became_peered":"2024-05-17T21:57:30.275932+0000","last_unstale":"2024-05-17T21:57:32.202172+0000","last_undegraded":"2024-05-17T21:57:32.202172+0000","last_fullsized":"2024-05-17T21:57:32.202172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198846+0000","last_change":"2024-05-17T21:57:29.307483+0000","last_active":"2024-05-17T21:57:32.198846+0000","last_peered":"2024-05-17T21:57:32.198846+0000","last_clean":"2024-05-17T21:57:32.198846+0000","last_became_active":"2024-05-17T21:57:29.305855+0000","last_became_peered":"2024-05-17T21:57:29.305855+0000","last_unstale":"2024-05-17T21:57:32.198846+0000","last_undegraded":"2024-05-17T21:57:32.198846+0000","last_fullsized":"2024-05-17T21:57:32.198846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261305+0000","last_change":"2024-05-17T21:57:30.287920+0000","last_active":"2024-05-17T21:57:31.261305+0000","last_peered":"2024-05-17T21:57:31.261305+0000","last_clean":"2024-05-17T21:57:31.261305+0000","last_became_active":"2024-05-17T21:57:30.285382+0000","last_became_peered":"2024-05-17T21:57:30.285382+0000","last_unstale":"2024-05-17T21:57:31.261305+0000","last_undegraded":"2024-05-17T21:57:31.261305+0000","last_fullsized":"2024-05-17T21:57:31.261305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:06.097854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268342+0000","last_change":"2024-05-17T21:57:29.291983+0000","last_active":"2024-05-17T21:57:31.268342+0000","last_peered":"2024-05-17T21:57:31.268342+0000","last_clean":"2024-05-17T21:57:31.268342+0000","last_became_active":"2024-05-17T21:57:29.291695+0000","last_became_peered":"2024-05-17T21:57:29.291695+0000","last_unstale":"2024-05-17T21:57:31.268342+0000","last_undegraded":"2024-05-17T21:57:31.268342+0000","last_fullsized":"2024-05-17T21:57:31.268342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201964+0000","last_change":"2024-05-17T21:57:30.274417+0000","last_active":"2024-05-17T21:57:32.201964+0000","last_peered":"2024-05-17T21:57:32.201964+0000","last_clean":"2024-05-17T21:57:32.201964+0000","last_became_active":"2024-05-17T21:57:30.272167+0000","last_became_peered":"2024-05-17T21:57:30.272167+0000","last_unstale":"2024-05-17T21:57:32.201964+0000","last_undegraded":"2024-05-17T21:57:32.201964+0000","last_fullsized":"2024-05-17T21:57:32.201964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268209+0000","last_change":"2024-05-17T21:57:29.281250+0000","last_active":"2024-05-17T21:57:31.268209+0000","last_peered":"2024-05-17T21:57:31.268209+0000","last_clean":"2024-05-17T21:57:31.268209+0000","last_became_active":"2024-05-17T21:57:29.280913+0000","last_became_peered":"2024-05-17T21:57:29.280913+0000","last_unstale":"2024-05-17T21:57:31.268209+0000","last_undegraded":"2024-05-17T21:57:31.268209+0000","last_fullsized":"2024-05-17T21:57:31.268209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203516+0000","last_change":"2024-05-17T21:57:30.288741+0000","last_active":"2024-05-17T21:57:32.203516+0000","last_peered":"2024-05-17T21:57:32.203516+0000","last_clean":"2024-05-17T21:57:32.203516+0000","last_became_active":"2024-05-17T21:57:30.288383+0000","last_became_peered":"2024-05-17T21:57:30.288383+0000","last_unstale":"2024-05-17T21:57:32.203516+0000","last_undegraded":"2024-05-17T21:57:32.203516+0000","last_fullsized":"2024-05-17T21:57:32.203516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199760+0000","last_change":"2024-05-17T21:57:29.307532+0000","last_active":"2024-05-17T21:57:32.199760+0000","last_peered":"2024-05-17T21:57:32.199760+0000","last_clean":"2024-05-17T21:57:32.199760+0000","last_became_active":"2024-05-17T21:57:29.306783+0000","last_became_peered":"2024-05-17T21:57:29.306783+0000","last_unstale":"2024-05-17T21:57:32.199760+0000","last_undegraded":"2024-05-17T21:57:32.199760+0000","last_fullsized":"2024-05-17T21:57:32.199760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272292+0000","last_change":"2024-05-17T21:57:30.316509+0000","last_active":"2024-05-17T21:57:31.272292+0000","last_peered":"2024-05-17T21:57:31.272292+0000","last_clean":"2024-05-17T21:57:31.272292+0000","last_became_active":"2024-05-17T21:57:30.315773+0000","last_became_peered":"2024-05-17T21:57:30.315773+0000","last_unstale":"2024-05-17T21:57:31.272292+0000","last_undegraded":"2024-05-17T21:57:31.272292+0000","last_fullsized":"2024-05-17T21:57:31.272292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261905+0000","last_change":"2024-05-17T21:57:30.280585+0000","last_active":"2024-05-17T21:57:31.261905+0000","last_peered":"2024-05-17T21:57:31.261905+0000","last_clean":"2024-05-17T21:57:31.261905+0000","last_became_active":"2024-05-17T21:57:30.279214+0000","last_became_peered":"2024-05-17T21:57:30.279214+0000","last_unstale":"2024-05-17T21:57:31.261905+0000","last_undegraded":"2024-05-17T21:57:31.261905+0000","last_fullsized":"2024-05-17T21:57:31.261905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199406+0000","last_change":"2024-05-17T21:57:29.310082+0000","last_active":"2024-05-17T21:57:32.199406+0000","last_peered":"2024-05-17T21:57:32.199406+0000","last_clean":"2024-05-17T21:57:32.199406+0000","last_became_active":"2024-05-17T21:57:29.309288+0000","last_became_peered":"2024-05-17T21:57:29.309288+0000","last_unstale":"2024-05-17T21:57:32.199406+0000","last_undegraded":"2024-05-17T21:57:32.199406+0000","last_fullsized":"2024-05-17T21:57:32.199406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262800+0000","last_change":"2024-05-17T21:57:30.302983+0000","last_active":"2024-05-17T21:57:31.262800+0000","last_peered":"2024-05-17T21:57:31.262800+0000","last_clean":"2024-05-17T21:57:31.262800+0000","last_became_active":"2024-05-17T21:57:30.302146+0000","last_became_peered":"2024-05-17T21:57:30.302146+0000","last_unstale":"2024-05-17T21:57:31.262800+0000","last_undegraded":"2024-05-17T21:57:31.262800+0000","last_fullsized":"2024-05-17T21:57:31.262800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199774+0000","last_change":"2024-05-17T21:57:29.308164+0000","last_active":"2024-05-17T21:57:32.199774+0000","last_peered":"2024-05-17T21:57:32.199774+0000","last_clean":"2024-05-17T21:57:32.199774+0000","last_became_active":"2024-05-17T21:57:29.307552+0000","last_became_peered":"2024-05-17T21:57:29.307552+0000","last_unstale":"2024-05-17T21:57:32.199774+0000","last_undegraded":"2024-05-17T21:57:32.199774+0000","last_fullsized":"2024-05-17T21:57:32.199774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:46.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202699+0000","last_change":"2024-05-17T21:57:30.277781+0000","last_active":"2024-05-17T21:57:32.202699+0000","last_peered":"2024-05-17T21:57:32.202699+0000","last_clean":"2024-05-17T21:57:32.202699+0000","last_became_active":"2024-05-17T21:57:30.277206+0000","last_became_peered":"2024-05-17T21:57:30.277206+0000","last_unstale":"2024-05-17T21:57:32.202699+0000","last_undegraded":"2024-05-17T21:57:32.202699+0000","last_fullsized":"2024-05-17T21:57:32.202699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201872+0000","last_change":"2024-05-17T21:57:29.280172+0000","last_active":"2024-05-17T21:57:32.201872+0000","last_peered":"2024-05-17T21:57:32.201872+0000","last_clean":"2024-05-17T21:57:32.201872+0000","last_became_active":"2024-05-17T21:57:29.279827+0000","last_became_peered":"2024-05-17T21:57:29.279827+0000","last_unstale":"2024-05-17T21:57:32.201872+0000","last_undegraded":"2024-05-17T21:57:32.201872+0000","last_fullsized":"2024-05-17T21:57:32.201872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201468+0000","last_change":"2024-05-17T21:57:30.316828+0000","last_active":"2024-05-17T21:57:32.201468+0000","last_peered":"2024-05-17T21:57:32.201468+0000","last_clean":"2024-05-17T21:57:32.201468+0000","last_became_active":"2024-05-17T21:57:30.316471+0000","last_became_peered":"2024-05-17T21:57:30.316471+0000","last_unstale":"2024-05-17T21:57:32.201468+0000","last_undegraded":"2024-05-17T21:57:32.201468+0000","last_fullsized":"2024-05-17T21:57:32.201468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202822+0000","last_change":"2024-05-17T21:57:29.305595+0000","last_active":"2024-05-17T21:57:32.202822+0000","last_peered":"2024-05-17T21:57:32.202822+0000","last_clean":"2024-05-17T21:57:32.202822+0000","last_became_active":"2024-05-17T21:57:29.305272+0000","last_became_peered":"2024-05-17T21:57:29.305272+0000","last_unstale":"2024-05-17T21:57:32.202822+0000","last_undegraded":"2024-05-17T21:57:32.202822+0000","last_fullsized":"2024-05-17T21:57:32.202822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:33.665928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:41.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:57:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.997+0000 7f4b41371640 1 Processor -- start 2024-05-17T21:57:41.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:41.997+0000 7f4b41371640 1 -- start start 2024-05-17T21:57:41.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b41371640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b3c0d1040 0x7f4b3c144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b41371640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 0x7f4b3c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b41371640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b3c15ea40 con 0x7f4b3c15e490 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b41371640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3c15ed20 con 0x7f4b3c0d1040 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b41371640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4b3c15f000 con 0x7f4b3c15e080 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b3c0d1040 0x7f4b3c144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4b3c15e490 0x7f4b3c1616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33714/0 (socket says 172.21.15.83:33714) 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3b7fe640 1 -- 172.21.15.83:0/1552299631 learned_addr learned my addr 172.21.15.83:0/1552299631 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b33fff640 1 --2- 172.21.15.83:0/1552299631 >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 0x7f4b3c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 -- 172.21.15.83:0/1552299631 >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 msgr2=0x7f4b3c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 --2- 172.21.15.83:0/1552299631 >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 0x7f4b3c072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 -- 172.21.15.83:0/1552299631 >> v1:172.21.15.83:6789/0 conn(0x7f4b3c15e490 legacy=0x7f4b3c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 -- 172.21.15.83:0/1552299631 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c06ef30 con 0x7f4b3c0d1040 2024-05-17T21:57:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b33fff640 1 --2- 172.21.15.83:0/1552299631 >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 0x7f4b3c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.001+0000 7f4b3affd640 1 --2- 172.21.15.83:0/1552299631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c144080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0b6b10 tx=0x7f4b2c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74fe570bf839cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b3a7fc640 1 -- 172.21.15.83:0/1552299631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0b1070 con 0x7f4b3c0d1040 2024-05-17T21:57:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b3a7fc640 1 -- 172.21.15.83:0/1552299631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0ac5f0 con 0x7f4b3c0d1040 2024-05-17T21:57:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b3a7fc640 1 -- 172.21.15.83:0/1552299631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0ac9c0 con 0x7f4b3c0d1040 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- 172.21.15.83:0/1552299631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 msgr2=0x7f4b3c144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 --2- 172.21.15.83:0/1552299631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c144080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0b6b10 tx=0x7f4b2c01c480 comp rx=0 tx=0).stop 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- 172.21.15.83:0/1552299631 shutdown_connections 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 --2- 172.21.15.83:0/1552299631 >> v2:172.21.15.155:3300/0 conn(0x7f4b3c15e080 0x7f4b3c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 --2- 172.21.15.83:0/1552299631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c144080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- 172.21.15.83:0/1552299631 >> 172.21.15.83:0/1552299631 conn(0x7f4b3c06cd90 msgr2=0x7f4b3c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- 172.21.15.83:0/1552299631 shutdown_connections 2024-05-17T21:57:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- 172.21.15.83:0/1552299631 wait complete. 2024-05-17T21:57:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 Processor -- start 2024-05-17T21:57:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.005+0000 7f4b41371640 1 -- start start 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b3c15e080 0x7f4b3c13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 0x7f4b3c1461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c16abe0 con 0x7f4b3c0d1040 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c16aa60 con 0x7f4b3c15e490 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c16ad60 con 0x7f4b3c15e080 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39766/0 (socket says 172.21.15.83:39766) 2024-05-17T21:57:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 -- 172.21.15.83:0/4546232 learned_addr learned my addr 172.21.15.83:0/4546232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 -- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b3c15e080 msgr2=0x7f4b3c13b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b3c15e080 0x7f4b3c13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3b7fe640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 0x7f4b3c1461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 -- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 msgr2=0x7f4b3c1461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 0x7f4b3c1461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 -- 172.21.15.83:0/4546232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c15c8e0 con 0x7f4b3c0d1040 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3b7fe640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 0x7f4b3c1461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b3affd640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0af040 tx=0x7f4b2c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0b9020 con 0x7f4b3c0d1040 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0b5410 con 0x7f4b3c0d1040 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c15cbc0 con 0x7f4b3c0d1040 2024-05-17T21:57:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0b1040 con 0x7f4b3c0d1040 2024-05-17T21:57:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c15d180 con 0x7f4b3c0d1040 2024-05-17T21:57:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c144080 con 0x7f4b3c0d1040 2024-05-17T21:57:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b2c0bc020 con 0x7f4b3c0d1040 2024-05-17T21:57:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 0x7f4b1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b2c128dd0 con 0x7f4b3c0d1040 2024-05-17T21:57:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.009+0000 7f4b33fff640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 0x7f4b1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.013+0000 7f4b33fff640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 0x7f4b1c071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c137120 tx=0x7f4b24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.017+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4b2c0c3030 con 0x7f4b3c0d1040 2024-05-17T21:57:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.161+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b3c0b35b0 con 0x7f4b1c06f400 2024-05-17T21:57:42.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.173+0000 7f4b337fe640 1 -- 172.21.15.83:0/4546232 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f4b3c0b35b0 con 0x7f4b1c06f400 2024-05-17T21:57:42.167 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:42.169 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:57:42.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 msgr2=0x7f4b1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 0x7f4b1c071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c137120 tx=0x7f4b24008040 comp rx=0 tx=0).stop 2024-05-17T21:57:42.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 msgr2=0x7f4b3c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0af040 tx=0x7f4b2c019220 comp rx=0 tx=0).stop 2024-05-17T21:57:42.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 shutdown_connections 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4b1c06f400 0x7f4b1c071910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b3c15e490 0x7f4b3c1461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b3c15e080 0x7f4b3c13b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 --2- 172.21.15.83:0/4546232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c0d1040 0x7f4b3c15bff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 >> 172.21.15.83:0/4546232 conn(0x7f4b3c06cd90 msgr2=0x7f4b3c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 shutdown_connections 2024-05-17T21:57:42.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.177+0000 7f4b41371640 1 -- 172.21.15.83:0/4546232 wait complete. 2024-05-17T21:57:42.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-17T21:57:40.968123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003112"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200427+0000","last_change":"2024-05-17T21:57:30.334206+0000","last_active":"2024-05-17T21:57:32.200427+0000","last_peered":"2024-05-17T21:57:32.200427+0000","last_clean":"2024-05-17T21:57:32.200427+0000","last_became_active":"2024-05-17T21:57:30.333625+0000","last_became_peered":"2024-05-17T21:57:30.333625+0000","last_unstale":"2024-05-17T21:57:32.200427+0000","last_undegraded":"2024-05-17T21:57:32.200427+0000","last_fullsized":"2024-05-17T21:57:32.200427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268827+0000","last_change":"2024-05-17T21:57:30.282372+0000","last_active":"2024-05-17T21:57:31.268827+0000","last_peered":"2024-05-17T21:57:31.268827+0000","last_clean":"2024-05-17T21:57:31.268827+0000","last_became_active":"2024-05-17T21:57:30.282085+0000","last_became_peered":"2024-05-17T21:57:30.282085+0000","last_unstale":"2024-05-17T21:57:31.268827+0000","last_undegraded":"2024-05-17T21:57:31.268827+0000","last_fullsized":"2024-05-17T21:57:31.268827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201211+0000","last_change":"2024-05-17T21:57:30.274388+0000","last_active":"2024-05-17T21:57:32.201211+0000","last_peered":"2024-05-17T21:57:32.201211+0000","last_clean":"2024-05-17T21:57:32.201211+0000","last_became_active":"2024-05-17T21:57:30.274079+0000","last_became_peered":"2024-05-17T21:57:30.274079+0000","last_unstale":"2024-05-17T21:57:32.201211+0000","last_undegraded":"2024-05-17T21:57:32.201211+0000","last_fullsized":"2024-05-17T21:57:32.201211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:10.441970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262187+0000","last_change":"2024-05-17T21:57:30.299694+0000","last_active":"2024-05-17T21:57:31.262187+0000","last_peered":"2024-05-17T21:57:31.262187+0000","last_clean":"2024-05-17T21:57:31.262187+0000","last_became_active":"2024-05-17T21:57:30.298884+0000","last_became_peered":"2024-05-17T21:57:30.298884+0000","last_unstale":"2024-05-17T21:57:31.262187+0000","last_undegraded":"2024-05-17T21:57:31.262187+0000","last_fullsized":"2024-05-17T21:57:31.262187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199587+0000","last_change":"2024-05-17T21:57:30.316073+0000","last_active":"2024-05-17T21:57:32.199587+0000","last_peered":"2024-05-17T21:57:32.199587+0000","last_clean":"2024-05-17T21:57:32.199587+0000","last_became_active":"2024-05-17T21:57:30.315439+0000","last_became_peered":"2024-05-17T21:57:30.315439+0000","last_unstale":"2024-05-17T21:57:32.199587+0000","last_undegraded":"2024-05-17T21:57:32.199587+0000","last_fullsized":"2024-05-17T21:57:32.199587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261267+0000","last_change":"2024-05-17T21:57:30.295598+0000","last_active":"2024-05-17T21:57:31.261267+0000","last_peered":"2024-05-17T21:57:31.261267+0000","last_clean":"2024-05-17T21:57:31.261267+0000","last_became_active":"2024-05-17T21:57:30.294722+0000","last_became_peered":"2024-05-17T21:57:30.294722+0000","last_unstale":"2024-05-17T21:57:31.261267+0000","last_undegraded":"2024-05-17T21:57:31.261267+0000","last_fullsized":"2024-05-17T21:57:31.261267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199251+0000","last_change":"2024-05-17T21:57:30.324100+0000","last_active":"2024-05-17T21:57:32.199251+0000","last_peered":"2024-05-17T21:57:32.199251+0000","last_clean":"2024-05-17T21:57:32.199251+0000","last_became_active":"2024-05-17T21:57:30.320444+0000","last_became_peered":"2024-05-17T21:57:30.320444+0000","last_unstale":"2024-05-17T21:57:32.199251+0000","last_undegraded":"2024-05-17T21:57:32.199251+0000","last_fullsized":"2024-05-17T21:57:32.199251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200972+0000","last_change":"2024-05-17T21:57:30.302526+0000","last_active":"2024-05-17T21:57:32.200972+0000","last_peered":"2024-05-17T21:57:32.200972+0000","last_clean":"2024-05-17T21:57:32.200972+0000","last_became_active":"2024-05-17T21:57:30.302136+0000","last_became_peered":"2024-05-17T21:57:30.302136+0000","last_unstale":"2024-05-17T21:57:32.200972+0000","last_undegraded":"2024-05-17T21:57:32.200972+0000","last_fullsized":"2024-05-17T21:57:32.200972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200607+0000","last_change":"2024-05-17T21:57:30.332944+0000","last_active":"2024-05-17T21:57:32.200607+0000","last_peered":"2024-05-17T21:57:32.200607+0000","last_clean":"2024-05-17T21:57:32.200607+0000","last_became_active":"2024-05-17T21:57:30.329174+0000","last_became_peered":"2024-05-17T21:57:30.329174+0000","last_unstale":"2024-05-17T21:57:32.200607+0000","last_undegraded":"2024-05-17T21:57:32.200607+0000","last_fullsized":"2024-05-17T21:57:32.200607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:10.454507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272598+0000","last_change":"2024-05-17T21:57:30.314980+0000","last_active":"2024-05-17T21:57:31.272598+0000","last_peered":"2024-05-17T21:57:31.272598+0000","last_clean":"2024-05-17T21:57:31.272598+0000","last_became_active":"2024-05-17T21:57:30.314085+0000","last_became_peered":"2024-05-17T21:57:30.314085+0000","last_unstale":"2024-05-17T21:57:31.272598+0000","last_undegraded":"2024-05-17T21:57:31.272598+0000","last_fullsized":"2024-05-17T21:57:31.272598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272161+0000","last_change":"2024-05-17T21:57:30.316071+0000","last_active":"2024-05-17T21:57:31.272161+0000","last_peered":"2024-05-17T21:57:31.272161+0000","last_clean":"2024-05-17T21:57:31.272161+0000","last_became_active":"2024-05-17T21:57:30.314772+0000","last_became_peered":"2024-05-17T21:57:30.314772+0000","last_unstale":"2024-05-17T21:57:31.272161+0000","last_undegraded":"2024-05-17T21:57:31.272161+0000","last_fullsized":"2024-05-17T21:57:31.272161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260695+0000","last_change":"2024-05-17T21:57:30.319379+0000","last_active":"2024-05-17T21:57:31.260695+0000","last_peered":"2024-05-17T21:57:31.260695+0000","last_clean":"2024-05-17T21:57:31.260695+0000","last_became_active":"2024-05-17T21:57:30.318510+0000","last_became_peered":"2024-05-17T21:57:30.318510+0000","last_unstale":"2024-05-17T21:57:31.260695+0000","last_undegraded":"2024-05-17T21:57:31.260695+0000","last_fullsized":"2024-05-17T21:57:31.260695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:22.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203239+0000","last_change":"2024-05-17T21:57:30.315381+0000","last_active":"2024-05-17T21:57:32.203239+0000","last_peered":"2024-05-17T21:57:32.203239+0000","last_clean":"2024-05-17T21:57:32.203239+0000","last_became_active":"2024-05-17T21:57:30.315076+0000","last_became_peered":"2024-05-17T21:57:30.315076+0000","last_unstale":"2024-05-17T21:57:32.203239+0000","last_undegraded":"2024-05-17T21:57:32.203239+0000","last_fullsized":"2024-05-17T21:57:32.203239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200631+0000","last_change":"2024-05-17T21:57:30.274361+0000","last_active":"2024-05-17T21:57:32.200631+0000","last_peered":"2024-05-17T21:57:32.200631+0000","last_clean":"2024-05-17T21:57:32.200631+0000","last_became_active":"2024-05-17T21:57:30.273916+0000","last_became_peered":"2024-05-17T21:57:30.273916+0000","last_unstale":"2024-05-17T21:57:32.200631+0000","last_undegraded":"2024-05-17T21:57:32.200631+0000","last_fullsized":"2024-05-17T21:57:32.200631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:47.494853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268312+0000","last_change":"2024-05-17T21:57:30.283163+0000","last_active":"2024-05-17T21:57:31.268312+0000","last_peered":"2024-05-17T21:57:31.268312+0000","last_clean":"2024-05-17T21:57:31.268312+0000","last_became_active":"2024-05-17T21:57:30.282704+0000","last_became_peered":"2024-05-17T21:57:30.282704+0000","last_unstale":"2024-05-17T21:57:31.268312+0000","last_undegraded":"2024-05-17T21:57:31.268312+0000","last_fullsized":"2024-05-17T21:57:31.268312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260004+0000","last_change":"2024-05-17T21:57:30.296030+0000","last_active":"2024-05-17T21:57:31.260004+0000","last_peered":"2024-05-17T21:57:31.260004+0000","last_clean":"2024-05-17T21:57:31.260004+0000","last_became_active":"2024-05-17T21:57:30.294696+0000","last_became_peered":"2024-05-17T21:57:30.294696+0000","last_unstale":"2024-05-17T21:57:31.260004+0000","last_undegraded":"2024-05-17T21:57:31.260004+0000","last_fullsized":"2024-05-17T21:57:31.260004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263697+0000","last_change":"2024-05-17T21:57:30.323553+0000","last_active":"2024-05-17T21:57:31.263697+0000","last_peered":"2024-05-17T21:57:31.263697+0000","last_clean":"2024-05-17T21:57:31.263697+0000","last_became_active":"2024-05-17T21:57:30.323019+0000","last_became_peered":"2024-05-17T21:57:30.323019+0000","last_unstale":"2024-05-17T21:57:31.263697+0000","last_undegraded":"2024-05-17T21:57:31.263697+0000","last_fullsized":"2024-05-17T21:57:31.263697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:01:08.803813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262779+0000","last_change":"2024-05-17T21:57:30.319719+0000","last_active":"2024-05-17T21:57:31.262779+0000","last_peered":"2024-05-17T21:57:31.262779+0000","last_clean":"2024-05-17T21:57:31.262779+0000","last_became_active":"2024-05-17T21:57:30.319335+0000","last_became_peered":"2024-05-17T21:57:30.319335+0000","last_unstale":"2024-05-17T21:57:31.262779+0000","last_undegraded":"2024-05-17T21:57:31.262779+0000","last_fullsized":"2024-05-17T21:57:31.262779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200408+0000","last_change":"2024-05-17T21:57:30.321777+0000","last_active":"2024-05-17T21:57:32.200408+0000","last_peered":"2024-05-17T21:57:32.200408+0000","last_clean":"2024-05-17T21:57:32.200408+0000","last_became_active":"2024-05-17T21:57:30.321011+0000","last_became_peered":"2024-05-17T21:57:30.321011+0000","last_unstale":"2024-05-17T21:57:32.200408+0000","last_undegraded":"2024-05-17T21:57:32.200408+0000","last_fullsized":"2024-05-17T21:57:32.200408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272173+0000","last_change":"2024-05-17T21:57:30.314965+0000","last_active":"2024-05-17T21:57:31.272173+0000","last_peered":"2024-05-17T21:57:31.272173+0000","last_clean":"2024-05-17T21:57:31.272173+0000","last_became_active":"2024-05-17T21:57:30.314249+0000","last_became_peered":"2024-05-17T21:57:30.314249+0000","last_unstale":"2024-05-17T21:57:31.272173+0000","last_undegraded":"2024-05-17T21:57:31.272173+0000","last_fullsized":"2024-05-17T21:57:31.272173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201572+0000","last_change":"2024-05-17T21:57:30.318172+0000","last_active":"2024-05-17T21:57:32.201572+0000","last_peered":"2024-05-17T21:57:32.201572+0000","last_clean":"2024-05-17T21:57:32.201572+0000","last_became_active":"2024-05-17T21:57:30.317761+0000","last_became_peered":"2024-05-17T21:57:30.317761+0000","last_unstale":"2024-05-17T21:57:32.201572+0000","last_undegraded":"2024-05-17T21:57:32.201572+0000","last_fullsized":"2024-05-17T21:57:32.201572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262717+0000","last_change":"2024-05-17T21:57:30.320885+0000","last_active":"2024-05-17T21:57:31.262717+0000","last_peered":"2024-05-17T21:57:31.262717+0000","last_clean":"2024-05-17T21:57:31.262717+0000","last_became_active":"2024-05-17T21:57:30.320268+0000","last_became_peered":"2024-05-17T21:57:30.320268+0000","last_unstale":"2024-05-17T21:57:31.262717+0000","last_undegraded":"2024-05-17T21:57:31.262717+0000","last_fullsized":"2024-05-17T21:57:31.262717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201655+0000","last_change":"2024-05-17T21:57:30.306877+0000","last_active":"2024-05-17T21:57:32.201655+0000","last_peered":"2024-05-17T21:57:32.201655+0000","last_clean":"2024-05-17T21:57:32.201655+0000","last_became_active":"2024-05-17T21:57:30.306541+0000","last_became_peered":"2024-05-17T21:57:30.306541+0000","last_unstale":"2024-05-17T21:57:32.201655+0000","last_undegraded":"2024-05-17T21:57:32.201655+0000","last_fullsized":"2024-05-17T21:57:32.201655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260191+0000","last_change":"2024-05-17T21:57:30.285039+0000","last_active":"2024-05-17T21:57:31.260191+0000","last_peered":"2024-05-17T21:57:31.260191+0000","last_clean":"2024-05-17T21:57:31.260191+0000","last_became_active":"2024-05-17T21:57:30.284132+0000","last_became_peered":"2024-05-17T21:57:30.284132+0000","last_unstale":"2024-05-17T21:57:31.260191+0000","last_undegraded":"2024-05-17T21:57:31.260191+0000","last_fullsized":"2024-05-17T21:57:31.260191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202612+0000","last_change":"2024-05-17T21:57:30.285554+0000","last_active":"2024-05-17T21:57:32.202612+0000","last_peered":"2024-05-17T21:57:32.202612+0000","last_clean":"2024-05-17T21:57:32.202612+0000","last_became_active":"2024-05-17T21:57:30.285106+0000","last_became_peered":"2024-05-17T21:57:30.285106+0000","last_unstale":"2024-05-17T21:57:32.202612+0000","last_undegraded":"2024-05-17T21:57:32.202612+0000","last_fullsized":"2024-05-17T21:57:32.202612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272595+0000","last_change":"2024-05-17T21:57:30.318707+0000","last_active":"2024-05-17T21:57:31.272595+0000","last_peered":"2024-05-17T21:57:31.272595+0000","last_clean":"2024-05-17T21:57:31.272595+0000","last_became_active":"2024-05-17T21:57:30.317977+0000","last_became_peered":"2024-05-17T21:57:30.317977+0000","last_unstale":"2024-05-17T21:57:31.272595+0000","last_undegraded":"2024-05-17T21:57:31.272595+0000","last_fullsized":"2024-05-17T21:57:31.272595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.269288+0000","last_change":"2024-05-17T21:57:30.279948+0000","last_active":"2024-05-17T21:57:31.269288+0000","last_peered":"2024-05-17T21:57:31.269288+0000","last_clean":"2024-05-17T21:57:31.269288+0000","last_became_active":"2024-05-17T21:57:30.279346+0000","last_became_peered":"2024-05-17T21:57:30.279346+0000","last_unstale":"2024-05-17T21:57:31.269288+0000","last_undegraded":"2024-05-17T21:57:31.269288+0000","last_fullsized":"2024-05-17T21:57:31.269288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203266+0000","last_change":"2024-05-17T21:57:30.289588+0000","last_active":"2024-05-17T21:57:32.203266+0000","last_peered":"2024-05-17T21:57:32.203266+0000","last_clean":"2024-05-17T21:57:32.203266+0000","last_became_active":"2024-05-17T21:57:30.289189+0000","last_became_peered":"2024-05-17T21:57:30.289189+0000","last_unstale":"2024-05-17T21:57:32.203266+0000","last_undegraded":"2024-05-17T21:57:32.203266+0000","last_fullsized":"2024-05-17T21:57:32.203266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202018+0000","last_change":"2024-05-17T21:57:30.282815+0000","last_active":"2024-05-17T21:57:32.202018+0000","last_peered":"2024-05-17T21:57:32.202018+0000","last_clean":"2024-05-17T21:57:32.202018+0000","last_became_active":"2024-05-17T21:57:30.282056+0000","last_became_peered":"2024-05-17T21:57:30.282056+0000","last_unstale":"2024-05-17T21:57:32.202018+0000","last_undegraded":"2024-05-17T21:57:32.202018+0000","last_fullsized":"2024-05-17T21:57:32.202018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261817+0000","last_change":"2024-05-17T21:57:30.281437+0000","last_active":"2024-05-17T21:57:31.261817+0000","last_peered":"2024-05-17T21:57:31.261817+0000","last_clean":"2024-05-17T21:57:31.261817+0000","last_became_active":"2024-05-17T21:57:30.280235+0000","last_became_peered":"2024-05-17T21:57:30.280235+0000","last_unstale":"2024-05-17T21:57:31.261817+0000","last_undegraded":"2024-05-17T21:57:31.261817+0000","last_fullsized":"2024-05-17T21:57:31.261817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:27:48.125319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262790+0000","last_change":"2024-05-17T21:57:30.274855+0000","last_active":"2024-05-17T21:57:31.262790+0000","last_peered":"2024-05-17T21:57:31.262790+0000","last_clean":"2024-05-17T21:57:31.262790+0000","last_became_active":"2024-05-17T21:57:30.274300+0000","last_became_peered":"2024-05-17T21:57:30.274300+0000","last_unstale":"2024-05-17T21:57:31.262790+0000","last_undegraded":"2024-05-17T21:57:31.262790+0000","last_fullsized":"2024-05-17T21:57:31.262790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:33.958058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271627+0000","last_change":"2024-05-17T21:57:30.305394+0000","last_active":"2024-05-17T21:57:31.271627+0000","last_peered":"2024-05-17T21:57:31.271627+0000","last_clean":"2024-05-17T21:57:31.271627+0000","last_became_active":"2024-05-17T21:57:30.304545+0000","last_became_peered":"2024-05-17T21:57:30.304545+0000","last_unstale":"2024-05-17T21:57:31.271627+0000","last_undegraded":"2024-05-17T21:57:31.271627+0000","last_fullsized":"2024-05-17T21:57:31.271627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:19.160093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260730+0000","last_change":"2024-05-17T21:57:30.271395+0000","last_active":"2024-05-17T21:57:31.260730+0000","last_peered":"2024-05-17T21:57:31.260730+0000","last_clean":"2024-05-17T21:57:31.260730+0000","last_became_active":"2024-05-17T21:57:30.270232+0000","last_became_peered":"2024-05-17T21:57:30.270232+0000","last_unstale":"2024-05-17T21:57:31.260730+0000","last_undegraded":"2024-05-17T21:57:31.260730+0000","last_fullsized":"2024-05-17T21:57:31.260730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:33.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203332+0000","last_change":"2024-05-17T21:57:29.306016+0000","last_active":"2024-05-17T21:57:32.203332+0000","last_peered":"2024-05-17T21:57:32.203332+0000","last_clean":"2024-05-17T21:57:32.203332+0000","last_became_active":"2024-05-17T21:57:29.305576+0000","last_became_peered":"2024-05-17T21:57:29.305576+0000","last_unstale":"2024-05-17T21:57:32.203332+0000","last_undegraded":"2024-05-17T21:57:32.203332+0000","last_fullsized":"2024-05-17T21:57:32.203332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202752+0000","last_change":"2024-05-17T21:57:30.283060+0000","last_active":"2024-05-17T21:57:32.202752+0000","last_peered":"2024-05-17T21:57:32.202752+0000","last_clean":"2024-05-17T21:57:32.202752+0000","last_became_active":"2024-05-17T21:57:30.282526+0000","last_became_peered":"2024-05-17T21:57:30.282526+0000","last_unstale":"2024-05-17T21:57:32.202752+0000","last_undegraded":"2024-05-17T21:57:32.202752+0000","last_fullsized":"2024-05-17T21:57:32.202752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271871+0000","last_change":"2024-05-17T21:57:29.299174+0000","last_active":"2024-05-17T21:57:31.271871+0000","last_peered":"2024-05-17T21:57:31.271871+0000","last_clean":"2024-05-17T21:57:31.271871+0000","last_became_active":"2024-05-17T21:57:29.297800+0000","last_became_peered":"2024-05-17T21:57:29.297800+0000","last_unstale":"2024-05-17T21:57:31.271871+0000","last_undegraded":"2024-05-17T21:57:31.271871+0000","last_fullsized":"2024-05-17T21:57:31.271871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202014+0000","last_change":"2024-05-17T21:57:29.290665+0000","last_active":"2024-05-17T21:57:32.202014+0000","last_peered":"2024-05-17T21:57:32.202014+0000","last_clean":"2024-05-17T21:57:32.202014+0000","last_became_active":"2024-05-17T21:57:29.290298+0000","last_became_peered":"2024-05-17T21:57:29.290298+0000","last_unstale":"2024-05-17T21:57:32.202014+0000","last_undegraded":"2024-05-17T21:57:32.202014+0000","last_fullsized":"2024-05-17T21:57:32.202014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201127+0000","last_change":"2024-05-17T21:57:30.283851+0000","last_active":"2024-05-17T21:57:32.201127+0000","last_peered":"2024-05-17T21:57:32.201127+0000","last_clean":"2024-05-17T21:57:32.201127+0000","last_became_active":"2024-05-17T21:57:30.283547+0000","last_became_peered":"2024-05-17T21:57:30.283547+0000","last_unstale":"2024-05-17T21:57:32.201127+0000","last_undegraded":"2024-05-17T21:57:32.201127+0000","last_fullsized":"2024-05-17T21:57:32.201127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260166+0000","last_change":"2024-05-17T21:57:29.297693+0000","last_active":"2024-05-17T21:57:31.260166+0000","last_peered":"2024-05-17T21:57:31.260166+0000","last_clean":"2024-05-17T21:57:31.260166+0000","last_became_active":"2024-05-17T21:57:29.296843+0000","last_became_peered":"2024-05-17T21:57:29.296843+0000","last_unstale":"2024-05-17T21:57:31.260166+0000","last_undegraded":"2024-05-17T21:57:31.260166+0000","last_fullsized":"2024-05-17T21:57:31.260166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268162+0000","last_change":"2024-05-17T21:57:30.278227+0000","last_active":"2024-05-17T21:57:31.268162+0000","last_peered":"2024-05-17T21:57:31.268162+0000","last_clean":"2024-05-17T21:57:31.268162+0000","last_became_active":"2024-05-17T21:57:30.277784+0000","last_became_peered":"2024-05-17T21:57:30.277784+0000","last_unstale":"2024-05-17T21:57:31.268162+0000","last_undegraded":"2024-05-17T21:57:31.268162+0000","last_fullsized":"2024-05-17T21:57:31.268162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271960+0000","last_change":"2024-05-17T21:57:29.290141+0000","last_active":"2024-05-17T21:57:31.271960+0000","last_peered":"2024-05-17T21:57:31.271960+0000","last_clean":"2024-05-17T21:57:31.271960+0000","last_became_active":"2024-05-17T21:57:29.288975+0000","last_became_peered":"2024-05-17T21:57:29.288975+0000","last_unstale":"2024-05-17T21:57:31.271960+0000","last_undegraded":"2024-05-17T21:57:31.271960+0000","last_fullsized":"2024-05-17T21:57:31.271960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262047+0000","last_change":"2024-05-17T21:57:30.322244+0000","last_active":"2024-05-17T21:57:31.262047+0000","last_peered":"2024-05-17T21:57:31.262047+0000","last_clean":"2024-05-17T21:57:31.262047+0000","last_became_active":"2024-05-17T21:57:30.321701+0000","last_became_peered":"2024-05-17T21:57:30.321701+0000","last_unstale":"2024-05-17T21:57:31.262047+0000","last_undegraded":"2024-05-17T21:57:31.262047+0000","last_fullsized":"2024-05-17T21:57:31.262047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:13.526524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262204+0000","last_change":"2024-05-17T21:57:29.293457+0000","last_active":"2024-05-17T21:57:31.262204+0000","last_peered":"2024-05-17T21:57:31.262204+0000","last_clean":"2024-05-17T21:57:31.262204+0000","last_became_active":"2024-05-17T21:57:29.292418+0000","last_became_peered":"2024-05-17T21:57:29.292418+0000","last_unstale":"2024-05-17T21:57:31.262204+0000","last_undegraded":"2024-05-17T21:57:31.262204+0000","last_fullsized":"2024-05-17T21:57:31.262204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202960+0000","last_change":"2024-05-17T21:57:30.277503+0000","last_active":"2024-05-17T21:57:32.202960+0000","last_peered":"2024-05-17T21:57:32.202960+0000","last_clean":"2024-05-17T21:57:32.202960+0000","last_became_active":"2024-05-17T21:57:30.276765+0000","last_became_peered":"2024-05-17T21:57:30.276765+0000","last_unstale":"2024-05-17T21:57:32.202960+0000","last_undegraded":"2024-05-17T21:57:32.202960+0000","last_fullsized":"2024-05-17T21:57:32.202960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:00:26.498023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268442+0000","last_change":"2024-05-17T21:57:29.284139+0000","last_active":"2024-05-17T21:57:31.268442+0000","last_peered":"2024-05-17T21:57:31.268442+0000","last_clean":"2024-05-17T21:57:31.268442+0000","last_became_active":"2024-05-17T21:57:29.283786+0000","last_became_peered":"2024-05-17T21:57:29.283786+0000","last_unstale":"2024-05-17T21:57:31.268442+0000","last_undegraded":"2024-05-17T21:57:31.268442+0000","last_fullsized":"2024-05-17T21:57:31.268442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.260839+0000","last_change":"2024-05-17T21:57:30.322236+0000","last_active":"2024-05-17T21:57:31.260839+0000","last_peered":"2024-05-17T21:57:31.260839+0000","last_clean":"2024-05-17T21:57:31.260839+0000","last_became_active":"2024-05-17T21:57:30.321651+0000","last_became_peered":"2024-05-17T21:57:30.321651+0000","last_unstale":"2024-05-17T21:57:31.260839+0000","last_undegraded":"2024-05-17T21:57:31.260839+0000","last_fullsized":"2024-05-17T21:57:31.260839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:11.129074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271878+0000","last_change":"2024-05-17T21:57:29.286191+0000","last_active":"2024-05-17T21:57:31.271878+0000","last_peered":"2024-05-17T21:57:31.271878+0000","last_clean":"2024-05-17T21:57:31.271878+0000","last_became_active":"2024-05-17T21:57:29.285202+0000","last_became_peered":"2024-05-17T21:57:29.285202+0000","last_unstale":"2024-05-17T21:57:31.271878+0000","last_undegraded":"2024-05-17T21:57:31.271878+0000","last_fullsized":"2024-05-17T21:57:31.271878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199606+0000","last_change":"2024-05-17T21:57:30.323488+0000","last_active":"2024-05-17T21:57:32.199606+0000","last_peered":"2024-05-17T21:57:32.199606+0000","last_clean":"2024-05-17T21:57:32.199606+0000","last_became_active":"2024-05-17T21:57:30.322790+0000","last_became_peered":"2024-05-17T21:57:30.322790+0000","last_unstale":"2024-05-17T21:57:32.199606+0000","last_undegraded":"2024-05-17T21:57:32.199606+0000","last_fullsized":"2024-05-17T21:57:32.199606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202865+0000","last_change":"2024-05-17T21:57:29.298078+0000","last_active":"2024-05-17T21:57:32.202865+0000","last_peered":"2024-05-17T21:57:32.202865+0000","last_clean":"2024-05-17T21:57:32.202865+0000","last_became_active":"2024-05-17T21:57:29.297807+0000","last_became_peered":"2024-05-17T21:57:29.297807+0000","last_unstale":"2024-05-17T21:57:32.202865+0000","last_undegraded":"2024-05-17T21:57:32.202865+0000","last_fullsized":"2024-05-17T21:57:32.202865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:24:33.244279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201096+0000","last_change":"2024-05-17T21:57:30.318056+0000","last_active":"2024-05-17T21:57:32.201096+0000","last_peered":"2024-05-17T21:57:32.201096+0000","last_clean":"2024-05-17T21:57:32.201096+0000","last_became_active":"2024-05-17T21:57:30.317771+0000","last_became_peered":"2024-05-17T21:57:30.317771+0000","last_unstale":"2024-05-17T21:57:32.201096+0000","last_undegraded":"2024-05-17T21:57:32.201096+0000","last_fullsized":"2024-05-17T21:57:32.201096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:42.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261966+0000","last_change":"2024-05-17T21:57:29.284218+0000","last_active":"2024-05-17T21:57:31.261966+0000","last_peered":"2024-05-17T21:57:31.261966+0000","last_clean":"2024-05-17T21:57:31.261966+0000","last_became_active":"2024-05-17T21:57:29.283386+0000","last_became_peered":"2024-05-17T21:57:29.283386+0000","last_unstale":"2024-05-17T21:57:31.261966+0000","last_undegraded":"2024-05-17T21:57:31.261966+0000","last_fullsized":"2024-05-17T21:57:31.261966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200000+0000","last_change":"2024-05-17T21:57:30.321543+0000","last_active":"2024-05-17T21:57:32.200000+0000","last_peered":"2024-05-17T21:57:32.200000+0000","last_clean":"2024-05-17T21:57:32.200000+0000","last_became_active":"2024-05-17T21:57:30.320566+0000","last_became_peered":"2024-05-17T21:57:30.320566+0000","last_unstale":"2024-05-17T21:57:32.200000+0000","last_undegraded":"2024-05-17T21:57:32.200000+0000","last_fullsized":"2024-05-17T21:57:32.200000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271412+0000","last_change":"2024-05-17T21:57:29.297525+0000","last_active":"2024-05-17T21:57:31.271412+0000","last_peered":"2024-05-17T21:57:31.271412+0000","last_clean":"2024-05-17T21:57:31.271412+0000","last_became_active":"2024-05-17T21:57:29.296129+0000","last_became_peered":"2024-05-17T21:57:29.296129+0000","last_unstale":"2024-05-17T21:57:31.271412+0000","last_undegraded":"2024-05-17T21:57:31.271412+0000","last_fullsized":"2024-05-17T21:57:31.271412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.271732+0000","last_change":"2024-05-17T21:57:30.306121+0000","last_active":"2024-05-17T21:57:31.271732+0000","last_peered":"2024-05-17T21:57:31.271732+0000","last_clean":"2024-05-17T21:57:31.271732+0000","last_became_active":"2024-05-17T21:57:30.304496+0000","last_became_peered":"2024-05-17T21:57:30.304496+0000","last_unstale":"2024-05-17T21:57:31.271732+0000","last_undegraded":"2024-05-17T21:57:31.271732+0000","last_fullsized":"2024-05-17T21:57:31.271732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:06:53.601643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262030+0000","last_change":"2024-05-17T21:57:29.273347+0000","last_active":"2024-05-17T21:57:31.262030+0000","last_peered":"2024-05-17T21:57:31.262030+0000","last_clean":"2024-05-17T21:57:31.262030+0000","last_became_active":"2024-05-17T21:57:29.272913+0000","last_became_peered":"2024-05-17T21:57:29.272913+0000","last_unstale":"2024-05-17T21:57:31.262030+0000","last_undegraded":"2024-05-17T21:57:31.262030+0000","last_fullsized":"2024-05-17T21:57:31.262030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:34:49.617602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199328+0000","last_change":"2024-05-17T21:57:30.322675+0000","last_active":"2024-05-17T21:57:32.199328+0000","last_peered":"2024-05-17T21:57:32.199328+0000","last_clean":"2024-05-17T21:57:32.199328+0000","last_became_active":"2024-05-17T21:57:30.321924+0000","last_became_peered":"2024-05-17T21:57:30.321924+0000","last_unstale":"2024-05-17T21:57:32.199328+0000","last_undegraded":"2024-05-17T21:57:32.199328+0000","last_fullsized":"2024-05-17T21:57:32.199328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262737+0000","last_change":"2024-05-17T21:57:29.294110+0000","last_active":"2024-05-17T21:57:31.262737+0000","last_peered":"2024-05-17T21:57:31.262737+0000","last_clean":"2024-05-17T21:57:31.262737+0000","last_became_active":"2024-05-17T21:57:29.293170+0000","last_became_peered":"2024-05-17T21:57:29.293170+0000","last_unstale":"2024-05-17T21:57:31.262737+0000","last_undegraded":"2024-05-17T21:57:31.262737+0000","last_fullsized":"2024-05-17T21:57:31.262737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:28:34.887309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200251+0000","last_change":"2024-05-17T21:57:30.315778+0000","last_active":"2024-05-17T21:57:32.200251+0000","last_peered":"2024-05-17T21:57:32.200251+0000","last_clean":"2024-05-17T21:57:32.200251+0000","last_became_active":"2024-05-17T21:57:30.314758+0000","last_became_peered":"2024-05-17T21:57:30.314758+0000","last_unstale":"2024-05-17T21:57:32.200251+0000","last_undegraded":"2024-05-17T21:57:32.200251+0000","last_fullsized":"2024-05-17T21:57:32.200251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263385+0000","last_change":"2024-05-17T21:57:29.281454+0000","last_active":"2024-05-17T21:57:31.263385+0000","last_peered":"2024-05-17T21:57:31.263385+0000","last_clean":"2024-05-17T21:57:31.263385+0000","last_became_active":"2024-05-17T21:57:29.280983+0000","last_became_peered":"2024-05-17T21:57:29.280983+0000","last_unstale":"2024-05-17T21:57:31.263385+0000","last_undegraded":"2024-05-17T21:57:31.263385+0000","last_fullsized":"2024-05-17T21:57:31.263385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200703+0000","last_change":"2024-05-17T21:57:30.334810+0000","last_active":"2024-05-17T21:57:32.200703+0000","last_peered":"2024-05-17T21:57:32.200703+0000","last_clean":"2024-05-17T21:57:32.200703+0000","last_became_active":"2024-05-17T21:57:30.332951+0000","last_became_peered":"2024-05-17T21:57:30.332951+0000","last_unstale":"2024-05-17T21:57:32.200703+0000","last_undegraded":"2024-05-17T21:57:32.200703+0000","last_fullsized":"2024-05-17T21:57:32.200703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:52.659720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268772+0000","last_change":"2024-05-17T21:57:29.296523+0000","last_active":"2024-05-17T21:57:31.268772+0000","last_peered":"2024-05-17T21:57:31.268772+0000","last_clean":"2024-05-17T21:57:31.268772+0000","last_became_active":"2024-05-17T21:57:29.296297+0000","last_became_peered":"2024-05-17T21:57:29.296297+0000","last_unstale":"2024-05-17T21:57:31.268772+0000","last_undegraded":"2024-05-17T21:57:31.268772+0000","last_fullsized":"2024-05-17T21:57:31.268772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.263237+0000","last_change":"2024-05-17T21:57:30.297086+0000","last_active":"2024-05-17T21:57:31.263237+0000","last_peered":"2024-05-17T21:57:31.263237+0000","last_clean":"2024-05-17T21:57:31.263237+0000","last_became_active":"2024-05-17T21:57:30.296276+0000","last_became_peered":"2024-05-17T21:57:30.296276+0000","last_unstale":"2024-05-17T21:57:31.263237+0000","last_undegraded":"2024-05-17T21:57:31.263237+0000","last_fullsized":"2024-05-17T21:57:31.263237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:10:28.582008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262701+0000","last_change":"2024-05-17T21:57:29.282987+0000","last_active":"2024-05-17T21:57:31.262701+0000","last_peered":"2024-05-17T21:57:31.262701+0000","last_clean":"2024-05-17T21:57:31.262701+0000","last_became_active":"2024-05-17T21:57:29.282594+0000","last_became_peered":"2024-05-17T21:57:29.282594+0000","last_unstale":"2024-05-17T21:57:31.262701+0000","last_undegraded":"2024-05-17T21:57:31.262701+0000","last_fullsized":"2024-05-17T21:57:31.262701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:43:52.306565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200040+0000","last_change":"2024-05-17T21:57:30.328305+0000","last_active":"2024-05-17T21:57:32.200040+0000","last_peered":"2024-05-17T21:57:32.200040+0000","last_clean":"2024-05-17T21:57:32.200040+0000","last_became_active":"2024-05-17T21:57:30.324651+0000","last_became_peered":"2024-05-17T21:57:30.324651+0000","last_unstale":"2024-05-17T21:57:32.200040+0000","last_undegraded":"2024-05-17T21:57:32.200040+0000","last_fullsized":"2024-05-17T21:57:32.200040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:34:03.788428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199220+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:57:32.199220+0000","last_peered":"2024-05-17T21:57:32.199220+0000","last_clean":"2024-05-17T21:57:32.199220+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:57:32.199220+0000","last_undegraded":"2024-05-17T21:57:32.199220+0000","last_fullsized":"2024-05-17T21:57:32.199220+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261146+0000","last_change":"2024-05-17T21:57:29.277871+0000","last_active":"2024-05-17T21:57:31.261146+0000","last_peered":"2024-05-17T21:57:31.261146+0000","last_clean":"2024-05-17T21:57:31.261146+0000","last_became_active":"2024-05-17T21:57:29.276986+0000","last_became_peered":"2024-05-17T21:57:29.276986+0000","last_unstale":"2024-05-17T21:57:31.261146+0000","last_undegraded":"2024-05-17T21:57:31.261146+0000","last_fullsized":"2024-05-17T21:57:31.261146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201481+0000","last_change":"2024-05-17T21:57:30.281160+0000","last_active":"2024-05-17T21:57:32.201481+0000","last_peered":"2024-05-17T21:57:32.201481+0000","last_clean":"2024-05-17T21:57:32.201481+0000","last_became_active":"2024-05-17T21:57:30.280004+0000","last_became_peered":"2024-05-17T21:57:30.280004+0000","last_unstale":"2024-05-17T21:57:32.201481+0000","last_undegraded":"2024-05-17T21:57:32.201481+0000","last_fullsized":"2024-05-17T21:57:32.201481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198887+0000","last_change":"2024-05-17T21:57:29.308082+0000","last_active":"2024-05-17T21:57:32.198887+0000","last_peered":"2024-05-17T21:57:32.198887+0000","last_clean":"2024-05-17T21:57:32.198887+0000","last_became_active":"2024-05-17T21:57:29.306695+0000","last_became_peered":"2024-05-17T21:57:29.306695+0000","last_unstale":"2024-05-17T21:57:32.198887+0000","last_undegraded":"2024-05-17T21:57:32.198887+0000","last_fullsized":"2024-05-17T21:57:32.198887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:11.865092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200656+0000","last_change":"2024-05-17T21:57:30.281759+0000","last_active":"2024-05-17T21:57:32.200656+0000","last_peered":"2024-05-17T21:57:32.200656+0000","last_clean":"2024-05-17T21:57:32.200656+0000","last_became_active":"2024-05-17T21:57:30.281449+0000","last_became_peered":"2024-05-17T21:57:30.281449+0000","last_unstale":"2024-05-17T21:57:32.200656+0000","last_undegraded":"2024-05-17T21:57:32.200656+0000","last_fullsized":"2024-05-17T21:57:32.200656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:53:58.249441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262812+0000","last_change":"2024-05-17T21:57:29.280664+0000","last_active":"2024-05-17T21:57:31.262812+0000","last_peered":"2024-05-17T21:57:31.262812+0000","last_clean":"2024-05-17T21:57:31.262812+0000","last_became_active":"2024-05-17T21:57:29.280118+0000","last_became_peered":"2024-05-17T21:57:29.280118+0000","last_unstale":"2024-05-17T21:57:31.262812+0000","last_undegraded":"2024-05-17T21:57:31.262812+0000","last_fullsized":"2024-05-17T21:57:31.262812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268285+0000","last_change":"2024-05-17T21:57:30.287528+0000","last_active":"2024-05-17T21:57:31.268285+0000","last_peered":"2024-05-17T21:57:31.268285+0000","last_clean":"2024-05-17T21:57:31.268285+0000","last_became_active":"2024-05-17T21:57:30.285405+0000","last_became_peered":"2024-05-17T21:57:30.285405+0000","last_unstale":"2024-05-17T21:57:31.268285+0000","last_undegraded":"2024-05-17T21:57:31.268285+0000","last_fullsized":"2024-05-17T21:57:31.268285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262287+0000","last_change":"2024-05-17T21:57:29.281356+0000","last_active":"2024-05-17T21:57:31.262287+0000","last_peered":"2024-05-17T21:57:31.262287+0000","last_clean":"2024-05-17T21:57:31.262287+0000","last_became_active":"2024-05-17T21:57:29.279906+0000","last_became_peered":"2024-05-17T21:57:29.279906+0000","last_unstale":"2024-05-17T21:57:31.262287+0000","last_undegraded":"2024-05-17T21:57:31.262287+0000","last_fullsized":"2024-05-17T21:57:31.262287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:21.611486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261918+0000","last_change":"2024-05-17T21:57:30.277518+0000","last_active":"2024-05-17T21:57:31.261918+0000","last_peered":"2024-05-17T21:57:31.261918+0000","last_clean":"2024-05-17T21:57:31.261918+0000","last_became_active":"2024-05-17T21:57:30.275892+0000","last_became_peered":"2024-05-17T21:57:30.275892+0000","last_unstale":"2024-05-17T21:57:31.261918+0000","last_undegraded":"2024-05-17T21:57:31.261918+0000","last_fullsized":"2024-05-17T21:57:31.261918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:14:12.876388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268304+0000","last_change":"2024-05-17T21:57:29.270571+0000","last_active":"2024-05-17T21:57:31.268304+0000","last_peered":"2024-05-17T21:57:31.268304+0000","last_clean":"2024-05-17T21:57:31.268304+0000","last_became_active":"2024-05-17T21:57:29.270219+0000","last_became_peered":"2024-05-17T21:57:29.270219+0000","last_unstale":"2024-05-17T21:57:31.268304+0000","last_undegraded":"2024-05-17T21:57:31.268304+0000","last_fullsized":"2024-05-17T21:57:31.268304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262482+0000","last_change":"2024-05-17T21:57:30.281497+0000","last_active":"2024-05-17T21:57:31.262482+0000","last_peered":"2024-05-17T21:57:31.262482+0000","last_clean":"2024-05-17T21:57:31.262482+0000","last_became_active":"2024-05-17T21:57:30.280473+0000","last_became_peered":"2024-05-17T21:57:30.280473+0000","last_unstale":"2024-05-17T21:57:31.262482+0000","last_undegraded":"2024-05-17T21:57:31.262482+0000","last_fullsized":"2024-05-17T21:57:31.262482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:23.144954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.200226+0000","last_change":"2024-05-17T21:57:29.310998+0000","last_active":"2024-05-17T21:57:32.200226+0000","last_peered":"2024-05-17T21:57:32.200226+0000","last_clean":"2024-05-17T21:57:32.200226+0000","last_became_active":"2024-05-17T21:57:29.310445+0000","last_became_peered":"2024-05-17T21:57:29.310445+0000","last_unstale":"2024-05-17T21:57:32.200226+0000","last_undegraded":"2024-05-17T21:57:32.200226+0000","last_fullsized":"2024-05-17T21:57:32.200226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.270156+0000","last_change":"2024-05-17T21:57:30.286938+0000","last_active":"2024-05-17T21:57:31.270156+0000","last_peered":"2024-05-17T21:57:31.270156+0000","last_clean":"2024-05-17T21:57:31.270156+0000","last_became_active":"2024-05-17T21:57:30.286141+0000","last_became_peered":"2024-05-17T21:57:30.286141+0000","last_unstale":"2024-05-17T21:57:31.270156+0000","last_undegraded":"2024-05-17T21:57:31.270156+0000","last_fullsized":"2024-05-17T21:57:31.270156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268967+0000","last_change":"2024-05-17T21:57:29.291111+0000","last_active":"2024-05-17T21:57:31.268967+0000","last_peered":"2024-05-17T21:57:31.268967+0000","last_clean":"2024-05-17T21:57:31.268967+0000","last_became_active":"2024-05-17T21:57:29.290818+0000","last_became_peered":"2024-05-17T21:57:29.290818+0000","last_unstale":"2024-05-17T21:57:31.268967+0000","last_undegraded":"2024-05-17T21:57:31.268967+0000","last_fullsized":"2024-05-17T21:57:31.268967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:08:45.200725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202605+0000","last_change":"2024-05-17T21:57:30.321709+0000","last_active":"2024-05-17T21:57:32.202605+0000","last_peered":"2024-05-17T21:57:32.202605+0000","last_clean":"2024-05-17T21:57:32.202605+0000","last_became_active":"2024-05-17T21:57:30.321312+0000","last_became_peered":"2024-05-17T21:57:30.321312+0000","last_unstale":"2024-05-17T21:57:32.202605+0000","last_undegraded":"2024-05-17T21:57:32.202605+0000","last_fullsized":"2024-05-17T21:57:32.202605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:23:19.679659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272058+0000","last_change":"2024-05-17T21:57:29.310186+0000","last_active":"2024-05-17T21:57:31.272058+0000","last_peered":"2024-05-17T21:57:31.272058+0000","last_clean":"2024-05-17T21:57:31.272058+0000","last_became_active":"2024-05-17T21:57:29.309480+0000","last_became_peered":"2024-05-17T21:57:29.309480+0000","last_unstale":"2024-05-17T21:57:31.272058+0000","last_undegraded":"2024-05-17T21:57:31.272058+0000","last_fullsized":"2024-05-17T21:57:31.272058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202172+0000","last_change":"2024-05-17T21:57:30.276347+0000","last_active":"2024-05-17T21:57:32.202172+0000","last_peered":"2024-05-17T21:57:32.202172+0000","last_clean":"2024-05-17T21:57:32.202172+0000","last_became_active":"2024-05-17T21:57:30.275932+0000","last_became_peered":"2024-05-17T21:57:30.275932+0000","last_unstale":"2024-05-17T21:57:32.202172+0000","last_undegraded":"2024-05-17T21:57:32.202172+0000","last_fullsized":"2024-05-17T21:57:32.202172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:40.931275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.198846+0000","last_change":"2024-05-17T21:57:29.307483+0000","last_active":"2024-05-17T21:57:32.198846+0000","last_peered":"2024-05-17T21:57:32.198846+0000","last_clean":"2024-05-17T21:57:32.198846+0000","last_became_active":"2024-05-17T21:57:29.305855+0000","last_became_peered":"2024-05-17T21:57:29.305855+0000","last_unstale":"2024-05-17T21:57:32.198846+0000","last_undegraded":"2024-05-17T21:57:32.198846+0000","last_fullsized":"2024-05-17T21:57:32.198846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:05.086558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261305+0000","last_change":"2024-05-17T21:57:30.287920+0000","last_active":"2024-05-17T21:57:31.261305+0000","last_peered":"2024-05-17T21:57:31.261305+0000","last_clean":"2024-05-17T21:57:31.261305+0000","last_became_active":"2024-05-17T21:57:30.285382+0000","last_became_peered":"2024-05-17T21:57:30.285382+0000","last_unstale":"2024-05-17T21:57:31.261305+0000","last_undegraded":"2024-05-17T21:57:31.261305+0000","last_fullsized":"2024-05-17T21:57:31.261305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:06.097854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268342+0000","last_change":"2024-05-17T21:57:29.291983+0000","last_active":"2024-05-17T21:57:31.268342+0000","last_peered":"2024-05-17T21:57:31.268342+0000","last_clean":"2024-05-17T21:57:31.268342+0000","last_became_active":"2024-05-17T21:57:29.291695+0000","last_became_peered":"2024-05-17T21:57:29.291695+0000","last_unstale":"2024-05-17T21:57:31.268342+0000","last_undegraded":"2024-05-17T21:57:31.268342+0000","last_fullsized":"2024-05-17T21:57:31.268342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201964+0000","last_change":"2024-05-17T21:57:30.274417+0000","last_active":"2024-05-17T21:57:32.201964+0000","last_peered":"2024-05-17T21:57:32.201964+0000","last_clean":"2024-05-17T21:57:32.201964+0000","last_became_active":"2024-05-17T21:57:30.272167+0000","last_became_peered":"2024-05-17T21:57:30.272167+0000","last_unstale":"2024-05-17T21:57:32.201964+0000","last_undegraded":"2024-05-17T21:57:32.201964+0000","last_fullsized":"2024-05-17T21:57:32.201964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:52:49.036112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.268209+0000","last_change":"2024-05-17T21:57:29.281250+0000","last_active":"2024-05-17T21:57:31.268209+0000","last_peered":"2024-05-17T21:57:31.268209+0000","last_clean":"2024-05-17T21:57:31.268209+0000","last_became_active":"2024-05-17T21:57:29.280913+0000","last_became_peered":"2024-05-17T21:57:29.280913+0000","last_unstale":"2024-05-17T21:57:31.268209+0000","last_undegraded":"2024-05-17T21:57:31.268209+0000","last_fullsized":"2024-05-17T21:57:31.268209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:32.779855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.203516+0000","last_change":"2024-05-17T21:57:30.288741+0000","last_active":"2024-05-17T21:57:32.203516+0000","last_peered":"2024-05-17T21:57:32.203516+0000","last_clean":"2024-05-17T21:57:32.203516+0000","last_became_active":"2024-05-17T21:57:30.288383+0000","last_became_peered":"2024-05-17T21:57:30.288383+0000","last_unstale":"2024-05-17T21:57:32.203516+0000","last_undegraded":"2024-05-17T21:57:32.203516+0000","last_fullsized":"2024-05-17T21:57:32.203516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:14:42.407343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199760+0000","last_change":"2024-05-17T21:57:29.307532+0000","last_active":"2024-05-17T21:57:32.199760+0000","last_peered":"2024-05-17T21:57:32.199760+0000","last_clean":"2024-05-17T21:57:32.199760+0000","last_became_active":"2024-05-17T21:57:29.306783+0000","last_became_peered":"2024-05-17T21:57:29.306783+0000","last_unstale":"2024-05-17T21:57:32.199760+0000","last_undegraded":"2024-05-17T21:57:32.199760+0000","last_fullsized":"2024-05-17T21:57:32.199760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:58:32.946761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.272292+0000","last_change":"2024-05-17T21:57:30.316509+0000","last_active":"2024-05-17T21:57:31.272292+0000","last_peered":"2024-05-17T21:57:31.272292+0000","last_clean":"2024-05-17T21:57:31.272292+0000","last_became_active":"2024-05-17T21:57:30.315773+0000","last_became_peered":"2024-05-17T21:57:30.315773+0000","last_unstale":"2024-05-17T21:57:31.272292+0000","last_undegraded":"2024-05-17T21:57:31.272292+0000","last_fullsized":"2024-05-17T21:57:31.272292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:12:04.197796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.261905+0000","last_change":"2024-05-17T21:57:30.280585+0000","last_active":"2024-05-17T21:57:31.261905+0000","last_peered":"2024-05-17T21:57:31.261905+0000","last_clean":"2024-05-17T21:57:31.261905+0000","last_became_active":"2024-05-17T21:57:30.279214+0000","last_became_peered":"2024-05-17T21:57:30.279214+0000","last_unstale":"2024-05-17T21:57:31.261905+0000","last_undegraded":"2024-05-17T21:57:31.261905+0000","last_fullsized":"2024-05-17T21:57:31.261905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:07:21.850554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199406+0000","last_change":"2024-05-17T21:57:29.310082+0000","last_active":"2024-05-17T21:57:32.199406+0000","last_peered":"2024-05-17T21:57:32.199406+0000","last_clean":"2024-05-17T21:57:32.199406+0000","last_became_active":"2024-05-17T21:57:29.309288+0000","last_became_peered":"2024-05-17T21:57:29.309288+0000","last_unstale":"2024-05-17T21:57:32.199406+0000","last_undegraded":"2024-05-17T21:57:32.199406+0000","last_fullsized":"2024-05-17T21:57:32.199406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:10.117777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:31.262800+0000","last_change":"2024-05-17T21:57:30.302983+0000","last_active":"2024-05-17T21:57:31.262800+0000","last_peered":"2024-05-17T21:57:31.262800+0000","last_clean":"2024-05-17T21:57:31.262800+0000","last_became_active":"2024-05-17T21:57:30.302146+0000","last_became_peered":"2024-05-17T21:57:30.302146+0000","last_unstale":"2024-05-17T21:57:31.262800+0000","last_undegraded":"2024-05-17T21:57:31.262800+0000","last_fullsized":"2024-05-17T21:57:31.262800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:45:49.065759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.199774+0000","last_change":"2024-05-17T21:57:29.308164+0000","last_active":"2024-05-17T21:57:32.199774+0000","last_peered":"2024-05-17T21:57:32.199774+0000","last_clean":"2024-05-17T21:57:32.199774+0000","last_became_active":"2024-05-17T21:57:29.307552+0000","last_became_peered":"2024-05-17T21:57:29.307552+0000","last_unstale":"2024-05-17T21:57:32.199774+0000","last_undegraded":"2024-05-17T21:57:32.199774+0000","last_fullsized":"2024-05-17T21:57:32.199774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:52:46.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202699+0000","last_change":"2024-05-17T21:57:30.277781+0000","last_active":"2024-05-17T21:57:32.202699+0000","last_peered":"2024-05-17T21:57:32.202699+0000","last_clean":"2024-05-17T21:57:32.202699+0000","last_became_active":"2024-05-17T21:57:30.277206+0000","last_became_peered":"2024-05-17T21:57:30.277206+0000","last_unstale":"2024-05-17T21:57:32.202699+0000","last_undegraded":"2024-05-17T21:57:32.202699+0000","last_fullsized":"2024-05-17T21:57:32.202699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201872+0000","last_change":"2024-05-17T21:57:29.280172+0000","last_active":"2024-05-17T21:57:32.201872+0000","last_peered":"2024-05-17T21:57:32.201872+0000","last_clean":"2024-05-17T21:57:32.201872+0000","last_became_active":"2024-05-17T21:57:29.279827+0000","last_became_peered":"2024-05-17T21:57:29.279827+0000","last_unstale":"2024-05-17T21:57:32.201872+0000","last_undegraded":"2024-05-17T21:57:32.201872+0000","last_fullsized":"2024-05-17T21:57:32.201872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:56:15.004628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.201468+0000","last_change":"2024-05-17T21:57:30.316828+0000","last_active":"2024-05-17T21:57:32.201468+0000","last_peered":"2024-05-17T21:57:32.201468+0000","last_clean":"2024-05-17T21:57:32.201468+0000","last_became_active":"2024-05-17T21:57:30.316471+0000","last_became_peered":"2024-05-17T21:57:30.316471+0000","last_unstale":"2024-05-17T21:57:32.201468+0000","last_undegraded":"2024-05-17T21:57:32.201468+0000","last_fullsized":"2024-05-17T21:57:32.201468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:29.240170+0000","last_clean_scrub_stamp":"2024-05-17T21:57:29.240170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.957979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-17T21:57:32.202822+0000","last_change":"2024-05-17T21:57:29.305595+0000","last_active":"2024-05-17T21:57:32.202822+0000","last_peered":"2024-05-17T21:57:32.202822+0000","last_clean":"2024-05-17T21:57:32.202822+0000","last_became_active":"2024-05-17T21:57:29.305272+0000","last_became_peered":"2024-05-17T21:57:29.305272+0000","last_unstale":"2024-05-17T21:57:32.202822+0000","last_undegraded":"2024-05-17T21:57:32.202822+0000","last_fullsized":"2024-05-17T21:57:32.202822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:28.228873+0000","last_clean_scrub_stamp":"2024-05-17T21:57:28.228873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:33.665928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:57:42.192 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T21:57:42.192 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-17T21:57:42.192 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-17T21:57:42.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T21:57:42.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.373+0000 7fda915df640 1 Processor -- start 2024-05-17T21:57:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.373+0000 7fda915df640 1 -- start start 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda915df640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda8c1461c0 0x7fda8c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda915df640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda8c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda915df640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8c1465d0 con 0x7fda8c15e6b0 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda915df640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fda8c06ef30 con 0x7fda8c1461c0 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda915df640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c06f210 con 0x7fda8c0706e0 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8a7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fda8c15e6b0 0x7fda8c06ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33730/0 (socket says 172.21.15.83:33730) 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8a7fc640 1 -- 172.21.15.83:0/2555514961 learned_addr learned my addr 172.21.15.83:0/2555514961 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976783738 0 0) 0x7fda8c1465d0 con 0x7fda8c15e6b0 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8affd640 1 --2- 172.21.15.83:0/2555514961 >> v2:172.21.15.155:3300/0 conn(0x7fda8c1461c0 0x7fda8c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda78005740 con 0x7fda8c15e6b0 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda8c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1023773401 0 0) 0x7fda78005740 con 0x7fda8c15e6b0 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 msgr2=0x7fda8c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda8c1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 >> v2:172.21.15.155:3300/0 conn(0x7fda8c1461c0 msgr2=0x7fda8c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 --2- 172.21.15.83:0/2555514961 >> v2:172.21.15.155:3300/0 conn(0x7fda8c1461c0 0x7fda8c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c06f4f0 con 0x7fda8c15e6b0 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fda74001700 con 0x7fda8c15e6b0 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8affd640 1 --2- 172.21.15.83:0/2555514961 >> v2:172.21.15.155:3300/0 conn(0x7fda8c1461c0 0x7fda8c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 >> v1:172.21.15.83:6789/0 conn(0x7fda8c15e6b0 legacy=0x7fda8c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda78005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780062a0 0x7fda78006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda78006f60 0x7fda780073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c06ef30 con 0x7fda78006f60 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c06f210 con 0x7fda780062a0 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda7800de20 con 0x7fda8c0706e0 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda78006f60 0x7fda780073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8affd640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda78005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8a7fc640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780062a0 0x7fda78006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 -- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 msgr2=0x7fda78005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c0706e0 0x7fda78005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 -- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780062a0 msgr2=0x7fda78006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda780062a0 0x7fda78006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.377+0000 7fda8b7fe640 1 -- 172.21.15.83:0/2555514961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda78005740 con 0x7fda78006f60 2024-05-17T21:57:42.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda78006f60 0x7fda780073b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda84007360 tx=0x7fda8401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e60532bb267fff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda840b2070 con 0x7fda78006f60 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda840a5d60 con 0x7fda78006f60 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda89ffb640 1 -- 172.21.15.83:0/2555514961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84009be0 con 0x7fda78006f60 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- 172.21.15.83:0/2555514961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda78006f60 msgr2=0x7fda780073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 --2- 172.21.15.83:0/2555514961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda78006f60 0x7fda780073b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda84007360 tx=0x7fda8401ce60 comp rx=0 tx=0).stop 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda8b7fe640 1 -- 172.21.15.83:0/2555514961 reap_dead start 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- 172.21.15.83:0/2555514961 shutdown_connections 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- 172.21.15.83:0/2555514961 >> 172.21.15.83:0/2555514961 conn(0x7fda8c06cd90 msgr2=0x7fda8c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- 172.21.15.83:0/2555514961 shutdown_connections 2024-05-17T21:57:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- 172.21.15.83:0/2555514961 wait complete. 2024-05-17T21:57:42.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 Processor -- start 2024-05-17T21:57:42.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.381+0000 7fda915df640 1 -- start start 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c13eed0 0x7fda8c0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda8c0b1ba0 0x7fda8c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c1605a0 con 0x7fda8c13f2e0 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c06fab0 con 0x7fda8c0b1ba0 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c160720 con 0x7fda8c13eed0 2024-05-17T21:57:42.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39788/0 (socket says 172.21.15.83:39788) 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 -- 172.21.15.83:0/411018747 learned_addr learned my addr 172.21.15.83:0/411018747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8affd640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c13eed0 0x7fda8c0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8b7fe640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda8c0b1ba0 0x7fda8c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 -- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c13eed0 msgr2=0x7fda8c0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c13eed0 0x7fda8c0b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 -- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda8c0b1ba0 msgr2=0x7fda8c0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda8c0b1ba0 0x7fda8c0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 -- 172.21.15.83:0/411018747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c0b20b0 con 0x7fda8c13f2e0 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda8a7fc640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e8f0 tx=0x7fda7400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400c870 con 0x7fda8c13f2e0 2024-05-17T21:57:42.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74018070 con 0x7fda8c13f2e0 2024-05-17T21:57:42.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c0b2390 con 0x7fda8c13f2e0 2024-05-17T21:57:42.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7400cc30 con 0x7fda8c13f2e0 2024-05-17T21:57:42.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c0b2950 con 0x7fda8c13f2e0 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c15e170 con 0x7fda8c13f2e0 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda7401d460 con 0x7fda8c13f2e0 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.385+0000 7fda837fe640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.389+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fda7408f220 con 0x7fda8c13f2e0 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.389+0000 7fda8affd640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.389+0000 7fda8affd640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 0x7fda6c071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda8c143980 tx=0x7fda7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.393+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda7405c1a0 con 0x7fda8c13f2e0 2024-05-17T21:57:42.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.569+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda8c0b3a70 con 0x7fda8c13f2e0 2024-05-17T21:57:42.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda837fe640 1 -- 172.21.15.83:0/411018747 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fda7405baa0 con 0x7fda8c13f2e0 2024-05-17T21:57:42.563 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:42.563 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T21:57:42.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 msgr2=0x7fda6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 0x7fda6c071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda8c143980 tx=0x7fda7c008040 comp rx=0 tx=0).stop 2024-05-17T21:57:42.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 msgr2=0x7fda8c0b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda7400e8f0 tx=0x7fda7400edb0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 shutdown_connections 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda6c06f400 0x7fda6c071910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda8c0b1ba0 0x7fda8c0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda8c13f2e0 0x7fda8c0b5c40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 --2- 172.21.15.83:0/411018747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda8c13eed0 0x7fda8c0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 >> 172.21.15.83:0/411018747 conn(0x7fda8c06cd90 msgr2=0x7fda8c0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 shutdown_connections 2024-05-17T21:57:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.573+0000 7fda915df640 1 -- 172.21.15.83:0/411018747 wait complete. 2024-05-17T21:57:42.583 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-17T21:57:42.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 Processor -- start 2024-05-17T21:57:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 -- start start 2024-05-17T21:57:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 0x7f5328144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f532815e080 0x7f5328072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f532815ea40 con 0x7f532815e490 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f532815ed20 con 0x7f532815e080 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f533059f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f532815f000 con 0x7f53280d1040 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532eb3c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f532815e490 0x7f53281616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33746/0 (socket says 172.21.15.83:33746) 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532eb3c640 1 -- 172.21.15.83:0/3458229496 learned_addr learned my addr 172.21.15.83:0/3458229496 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2229152989 0 0) 0x7f532815ea40 con 0x7f532815e490 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532e33b640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 0x7f5328144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532db3a640 1 --2- 172.21.15.83:0/3458229496 >> v2:172.21.15.155:3300/0 conn(0x7f532815e080 0x7f5328072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5310003530 con 0x7f532815e490 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1522991475 0 0) 0x7f5310003530 con 0x7f532815e490 2024-05-17T21:57:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 msgr2=0x7f5328144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 0x7f5328144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 >> v2:172.21.15.155:3300/0 conn(0x7f532815e080 msgr2=0x7f5328072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 --2- 172.21.15.83:0/3458229496 >> v2:172.21.15.155:3300/0 conn(0x7f532815e080 0x7f5328072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532806ef30 con 0x7f532815e490 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f5318001a80 con 0x7f532815e490 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 >> v1:172.21.15.83:6789/0 conn(0x7f532815e490 legacy=0x7f53281616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5310003a10 0x7f5310003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f5310004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 0x7f53100051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f532815ed20 con 0x7f5310004810 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f532815f000 con 0x7f5310003a10 2024-05-17T21:57:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f531000bb30 con 0x7f53280d1040 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.733+0000 7f532db3a640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f5310004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532e33b640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5310003a10 0x7f5310003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 -- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 msgr2=0x7f53100051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53280d1040 0x7f53100051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 -- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5310003a10 msgr2=0x7f5310003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5310003a10 0x7f5310003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 -- 172.21.15.83:0/3458229496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5310003530 con 0x7f5310004810 2024-05-17T21:57:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532db3a640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f5310004c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f531c00ba00 tx=0x7f531c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896ecbeb6c62cd17 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0ae070 con 0x7f5310004810 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c0a9680 con 0x7f5310004810 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532d339640 1 -- 172.21.15.83:0/3458229496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c00b600 con 0x7f5310004810 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 -- 172.21.15.83:0/3458229496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 msgr2=0x7f5310004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 --2- 172.21.15.83:0/3458229496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f5310004c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f531c00ba00 tx=0x7f531c01c6a0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f532eb3c640 1 -- 172.21.15.83:0/3458229496 reap_dead start 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 -- 172.21.15.83:0/3458229496 shutdown_connections 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 -- 172.21.15.83:0/3458229496 >> 172.21.15.83:0/3458229496 conn(0x7f532806cd90 msgr2=0x7f53280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 -- 172.21.15.83:0/3458229496 shutdown_connections 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.737+0000 7f533059f640 1 -- 172.21.15.83:0/3458229496 wait complete. 2024-05-17T21:57:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 Processor -- start 2024-05-17T21:57:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- start start 2024-05-17T21:57:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532813ce30 0x7f5328146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532813d240 0x7f5328146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53280708c0 con 0x7f5310004810 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5328070740 con 0x7f532813d240 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5328070a40 con 0x7f532813ce30 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39810/0 (socket says 172.21.15.83:39810) 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 -- 172.21.15.83:0/2754429972 learned_addr learned my addr 172.21.15.83:0/2754429972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532db3a640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532813ce30 0x7f5328146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 -- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532813ce30 msgr2=0x7f5328146840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532813ce30 0x7f5328146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 -- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532813d240 msgr2=0x7f5328146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532813d240 0x7f5328146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 -- 172.21.15.83:0/2754429972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532813d9a0 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f532e33b640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53200115e0 tx=0x7f5320011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320015870 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320015a10 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532813dc80 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320015de0 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532813e240 con 0x7f5310004810 2024-05-17T21:57:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.741+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53281f3c90 con 0x7f5310004810 2024-05-17T21:57:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.745+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f532001d400 con 0x7f5310004810 2024-05-17T21:57:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.745+0000 7f5326ffd640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.745+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f532008fbb0 con 0x7f5310004810 2024-05-17T21:57:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.745+0000 7f532db3a640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.745+0000 7f532db3a640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f531c0b0000 tx=0x7f531c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.749+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5320019030 con 0x7f5310004810 2024-05-17T21:57:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5328142e10 con 0x7f5310004810 2024-05-17T21:57:42.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f5326ffd640 1 -- 172.21.15.83:0/2754429972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f532005caa0 con 0x7f5310004810 2024-05-17T21:57:42.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:42.921 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:57:42.921 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 msgr2=0x7f5314071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f531c0b0000 tx=0x7f531c0046d0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 msgr2=0x7f53281f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.929+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53200115e0 tx=0x7f5320011aa0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 shutdown_connections 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532813d240 0x7f5328146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532813ce30 0x7f5328146840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 --2- 172.21.15.83:0/2754429972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5310004810 0x7f53281f4360 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 >> 172.21.15.83:0/2754429972 conn(0x7f532806cd90 msgr2=0x7f532815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 shutdown_connections 2024-05-17T21:57:42.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:42.933+0000 7f533059f640 1 -- 172.21.15.83:0/2754429972 wait complete. 2024-05-17T21:57:42.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.081+0000 7fcce64a4640 1 Processor -- start 2024-05-17T21:57:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.081+0000 7fcce64a4640 1 -- start start 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fcce64a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcce0144080 0x7fcce0072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fcce64a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce0073780 con 0x7fcce0073370 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fcce64a4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce015e790 con 0x7fcce00d1040 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fcce64a4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcce015ea40 con 0x7fcce0144080 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcce0144080 0x7fcce0072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcce0144080 0x7fcce0072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39818/0 (socket says 172.21.15.83:39818) 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fcce4a41640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcce0073370 0x7fcce006f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33748/0 (socket says 172.21.15.83:33748) 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 -- 172.21.15.83:0/2767678777 learned_addr learned my addr 172.21.15.83:0/2767678777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 -- 172.21.15.83:0/2767678777 >> v1:172.21.15.155:6789/0 conn(0x7fcce00d1040 legacy=0x7fcce015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 -- 172.21.15.83:0/2767678777 >> v1:172.21.15.83:6789/0 conn(0x7fcce0073370 legacy=0x7fcce006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccd7fff640 1 -- 172.21.15.83:0/2767678777 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce015ed20 con 0x7fcce0144080 2024-05-17T21:57:43.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.085+0000 7fccdf7fe640 1 -- 172.21.15.83:0/2767678777 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509500157 0 0) 0x7fccd80014a0 con 0x7fcce0073370 2024-05-17T21:57:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fccd7fff640 1 --2- 172.21.15.83:0/2767678777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0144080 0x7fcce0072e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009870 tx=0x7fccc801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26deb89034d3b4cb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fccdf7fe640 1 -- 172.21.15.83:0/2767678777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80a95f0 con 0x7fcce0144080 2024-05-17T21:57:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fccdf7fe640 1 -- 172.21.15.83:0/2767678777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc80a9790 con 0x7fcce0144080 2024-05-17T21:57:43.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fccdf7fe640 1 -- 172.21.15.83:0/2767678777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80a9b60 con 0x7fcce0144080 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- 172.21.15.83:0/2767678777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0144080 msgr2=0x7fcce0072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 --2- 172.21.15.83:0/2767678777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0144080 0x7fcce0072e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009870 tx=0x7fccc801c530 comp rx=0 tx=0).stop 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- 172.21.15.83:0/2767678777 shutdown_connections 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 --2- 172.21.15.83:0/2767678777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0144080 0x7fcce0072e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- 172.21.15.83:0/2767678777 >> 172.21.15.83:0/2767678777 conn(0x7fcce006cd90 msgr2=0x7fcce00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- 172.21.15.83:0/2767678777 shutdown_connections 2024-05-17T21:57:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- 172.21.15.83:0/2767678777 wait complete. 2024-05-17T21:57:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 Processor -- start 2024-05-17T21:57:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.089+0000 7fcce64a4640 1 -- start start 2024-05-17T21:57:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce00d1040 0x7fcce013bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcce0144080 0x7fcce013d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0070650 con 0x7fcce0073370 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0144490 con 0x7fcce00d1040 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcce00707d0 con 0x7fcce0144080 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39832/0 (socket says 172.21.15.83:39832) 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 -- 172.21.15.83:0/3636464390 learned_addr learned my addr 172.21.15.83:0/3636464390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce4a41640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcce0144080 0x7fcce013d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 -- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcce0144080 msgr2=0x7fcce013d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcce0144080 0x7fcce013d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 -- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce00d1040 msgr2=0x7fcce013bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccd7fff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce00d1040 0x7fcce013bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce00d1040 0x7fcce013bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 -- 172.21.15.83:0/3636464390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce00afb30 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdffff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccd000b410 tx=0x7fccd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0015030 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd0004020 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce00afe10 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004450 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce00b03d0 con 0x7fcce0073370 2024-05-17T21:57:43.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce0072e30 con 0x7fcce0073370 2024-05-17T21:57:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fccd0018070 con 0x7fcce0073370 2024-05-17T21:57:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 0x7fccc0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fccd008ee30 con 0x7fcce0073370 2024-05-17T21:57:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccd7fff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 0x7fccc0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.093+0000 7fccd7fff640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 0x7fccc0071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccc80ae000 tx=0x7fccc80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.097+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fccd005bd20 con 0x7fcce0073370 2024-05-17T21:57:43.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.277+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcce00b1f50 con 0x7fcce0073370 2024-05-17T21:57:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fccdd7fa640 1 -- 172.21.15.83:0/3636464390 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fccd005b620 con 0x7fcce0073370 2024-05-17T21:57:43.272 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:43.272 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:57:43.272 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:57:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 msgr2=0x7fccc0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 0x7fccc0071940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccc80ae000 tx=0x7fccc80a7000 comp rx=0 tx=0).stop 2024-05-17T21:57:43.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 msgr2=0x7fcce015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccd000b410 tx=0x7fccd000b8d0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 shutdown_connections 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcce0144080 0x7fcce013d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fccc006f430 0x7fccc0071940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce00d1040 0x7fcce013bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 --2- 172.21.15.83:0/3636464390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcce0073370 0x7fcce015de80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 >> 172.21.15.83:0/3636464390 conn(0x7fcce006cd90 msgr2=0x7fcce0145550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 shutdown_connections 2024-05-17T21:57:43.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.281+0000 7fcce64a4640 1 -- 172.21.15.83:0/3636464390 wait complete. 2024-05-17T21:57:43.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.432+0000 7fe63b939640 1 Processor -- start 2024-05-17T21:57:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.432+0000 7fe63b939640 1 -- start start 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe63b939640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe63406ffd0 0x7fe6340703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe63b939640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6341605a0 0x7fe6341609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe63b939640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6341465d0 con 0x7fe6341461c0 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe63b939640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe634160ef0 con 0x7fe6341605a0 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe63b939640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe63406ef30 con 0x7fe63406ffd0 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6341605a0 0x7fe6341609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6341605a0 0x7fe6341609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39834/0 (socket says 172.21.15.83:39834) 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 -- 172.21.15.83:0/2437361421 learned_addr learned my addr 172.21.15.83:0/2437361421 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe638ed4640 1 --2- 172.21.15.83:0/2437361421 >> v2:172.21.15.155:3300/0 conn(0x7fe63406ffd0 0x7fe6340703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 -- 172.21.15.83:0/2437361421 >> v2:172.21.15.155:3300/0 conn(0x7fe63406ffd0 msgr2=0x7fe6340703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 --2- 172.21.15.83:0/2437361421 >> v2:172.21.15.155:3300/0 conn(0x7fe63406ffd0 0x7fe6340703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 -- 172.21.15.83:0/2437361421 >> v1:172.21.15.155:6789/0 conn(0x7fe6341461c0 legacy=0x7fe63415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.436+0000 7fe639ed6640 1 -- 172.21.15.83:0/2437361421 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe634070920 con 0x7fe6341605a0 2024-05-17T21:57:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe639ed6640 1 --2- 172.21.15.83:0/2437361421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6341605a0 0x7fe6341609b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe634071230 tx=0x7fe62c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3076a239374ec47c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe633fff640 1 -- 172.21.15.83:0/2437361421 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0aa610 con 0x7fe6341605a0 2024-05-17T21:57:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe633fff640 1 -- 172.21.15.83:0/2437361421 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c0af070 con 0x7fe6341605a0 2024-05-17T21:57:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe633fff640 1 -- 172.21.15.83:0/2437361421 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0aa9d0 con 0x7fe6341605a0 2024-05-17T21:57:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- 172.21.15.83:0/2437361421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6341605a0 msgr2=0x7fe6341609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 --2- 172.21.15.83:0/2437361421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6341605a0 0x7fe6341609b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe634071230 tx=0x7fe62c01c4f0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- 172.21.15.83:0/2437361421 shutdown_connections 2024-05-17T21:57:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 --2- 172.21.15.83:0/2437361421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6341605a0 0x7fe6341609b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 --2- 172.21.15.83:0/2437361421 >> v2:172.21.15.155:3300/0 conn(0x7fe63406ffd0 0x7fe6340703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- 172.21.15.83:0/2437361421 >> 172.21.15.83:0/2437361421 conn(0x7fe63406cd90 msgr2=0x7fe6340b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- 172.21.15.83:0/2437361421 shutdown_connections 2024-05-17T21:57:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- 172.21.15.83:0/2437361421 wait complete. 2024-05-17T21:57:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 Processor -- start 2024-05-17T21:57:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.440+0000 7fe63b939640 1 -- start start 2024-05-17T21:57:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6341461c0 0x7fe6340b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6341605a0 0x7fe6340b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe63415e880 con 0x7fe63406ffd0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe63406fab0 con 0x7fe6341605a0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6340d10a0 con 0x7fe6341461c0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39846/0 (socket says 172.21.15.83:39846) 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 -- 172.21.15.83:0/1188917947 learned_addr learned my addr 172.21.15.83:0/1188917947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 -- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6341461c0 msgr2=0x7fe6340b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6341461c0 0x7fe6340b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 -- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6341605a0 msgr2=0x7fe6340b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6341605a0 0x7fe6340b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 -- 172.21.15.83:0/1188917947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6341f6fa0 con 0x7fe63406ffd0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe6396d5640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b9f0 tx=0x7fe62400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62400cbd0 con 0x7fe63406ffd0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62400cd70 con 0x7fe63406ffd0 2024-05-17T21:57:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624003bc0 con 0x7fe63406ffd0 2024-05-17T21:57:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6340b1ff0 con 0x7fe63406ffd0 2024-05-17T21:57:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6340b4dd0 con 0x7fe63406ffd0 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6340703e0 con 0x7fe63406ffd0 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe624006d10 con 0x7fe63406ffd0 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 0x7fe620071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe62408f5d0 con 0x7fe63406ffd0 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe638ed4640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 0x7fe620071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.444+0000 7fe638ed4640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 0x7fe620071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe63413c5d0 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.448+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe6341f6fa0 con 0x7fe63406ffd0 2024-05-17T21:57:43.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.596+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6340b41a0 con 0x7fe63406ffd0 2024-05-17T21:57:43.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe631ffb640 1 -- 172.21.15.83:0/1188917947 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fe62405c630 con 0x7fe63406ffd0 2024-05-17T21:57:43.592 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:43.592 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 msgr2=0x7fe620071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 0x7fe620071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe63413c5d0 tx=0x7fe628008040 comp rx=0 tx=0).stop 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 msgr2=0x7fe6340b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b9f0 tx=0x7fe62400beb0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 shutdown_connections 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62006f400 0x7fe620071910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6341605a0 0x7fe6340b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6341461c0 0x7fe6340b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 --2- 172.21.15.83:0/1188917947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe63406ffd0 0x7fe6340b2660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 >> 172.21.15.83:0/1188917947 conn(0x7fe63406cd90 msgr2=0x7fe6340d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 shutdown_connections 2024-05-17T21:57:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.600+0000 7fe63b939640 1 -- 172.21.15.83:0/1188917947 wait complete. 2024-05-17T21:57:43.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:57:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.760+0000 7f4caa7af640 1 Processor -- start 2024-05-17T21:57:43.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.760+0000 7f4caa7af640 1 -- start start 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4caa7af640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ca4073540 0x7f4ca4168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4caa7af640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca415e790 con 0x7f4ca40d1040 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4caa7af640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca415ea40 con 0x7f4ca4144080 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4caa7af640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca415ed20 con 0x7f4ca4073540 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4ca40d1040 0x7f4ca415e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33764/0 (socket says 172.21.15.83:33764) 2024-05-17T21:57:43.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca3fff640 1 -- 172.21.15.83:0/1764788296 learned_addr learned my addr 172.21.15.83:0/1764788296 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684498765 0 0) 0x7f4ca415e790 con 0x7f4ca40d1040 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca8d4c640 1 --2- 172.21.15.83:0/1764788296 >> v2:172.21.15.155:3300/0 conn(0x7f4ca4073540 0x7f4ca4168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c90003530 con 0x7f4ca40d1040 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2510515918 0 0) 0x7f4c90003530 con 0x7f4ca40d1040 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 >> v2:172.21.15.155:3300/0 conn(0x7f4ca4073540 msgr2=0x7f4ca4168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 --2- 172.21.15.83:0/1764788296 >> v2:172.21.15.155:3300/0 conn(0x7f4ca4073540 0x7f4ca4168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 >> v1:172.21.15.155:6789/0 conn(0x7f4ca4144080 legacy=0x7f4ca4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca415f000 con 0x7f4ca40d1040 2024-05-17T21:57:43.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f4c94001700 con 0x7f4ca40d1040 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 >> v1:172.21.15.83:6789/0 conn(0x7f4ca40d1040 legacy=0x7f4ca415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c90003a60 0x7f4c90003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4c90004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c90005260 0x7f4c9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca415ed20 con 0x7f4c90004810 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9000c1f0 con 0x7f4c90005260 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9000c3f0 con 0x7f4c90003a60 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4c90004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca3fff640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c90003a60 0x7f4c90003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca8d4c640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c90005260 0x7f4c9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 -- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c90003a60 msgr2=0x7f4c90003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c90003a60 0x7f4c90003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 -- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c90005260 msgr2=0x7f4c9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c90005260 0x7f4c9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca37fe640 1 -- 172.21.15.83:0/1764788296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c90003530 con 0x7f4c90004810 2024-05-17T21:57:43.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca3fff640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c90003a60 0x7f4c90003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:43.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca8d4c640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c90005260 0x7f4c9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:43.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.764+0000 7f4ca8d4c640 1 -- 172.21.15.83:0/1764788296 reap_dead start 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4c90004c20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ceb70 tx=0x7f4c8c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef271e8125ee9bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0013e0 con 0x7f4c90004810 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c0052d0 con 0x7f4c90004810 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4ca2ffd640 1 -- 172.21.15.83:0/1764788296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003840 con 0x7f4c90004810 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 msgr2=0x7f4c90004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4c90004c20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ceb70 tx=0x7f4c8c01c330 comp rx=0 tx=0).stop 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- 172.21.15.83:0/1764788296 shutdown_connections 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 --2- 172.21.15.83:0/1764788296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4c90004c20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- 172.21.15.83:0/1764788296 >> 172.21.15.83:0/1764788296 conn(0x7f4ca406cd90 msgr2=0x7f4ca40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- 172.21.15.83:0/1764788296 shutdown_connections 2024-05-17T21:57:43.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- 172.21.15.83:0/1764788296 wait complete. 2024-05-17T21:57:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 Processor -- start 2024-05-17T21:57:43.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.768+0000 7f4caa7af640 1 -- start start 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca4160b70 0x7f4ca4160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 0x7f4ca413bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4144490 con 0x7f4c90004810 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca406f210 con 0x7f4ca4160b70 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca415e080 con 0x7f4ca413bb20 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39870/0 (socket says 172.21.15.83:39870) 2024-05-17T21:57:43.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 -- 172.21.15.83:0/3208110847 learned_addr learned my addr 172.21.15.83:0/3208110847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca8d4c640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 0x7f4ca413bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca4160b70 0x7f4ca4160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 -- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 msgr2=0x7f4ca413bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 0x7f4ca413bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 -- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca4160b70 msgr2=0x7f4ca4160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca4160b70 0x7f4ca4160f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 -- 172.21.15.83:0/3208110847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca415fdf0 con 0x7f4c90004810 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca8d4c640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 0x7f4ca413bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca3fff640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c94010b10 tx=0x7f4c94005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9401b070 con 0x7f4c90004810 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c94011040 con 0x7f4c90004810 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca413d5f0 con 0x7f4c90004810 2024-05-17T21:57:43.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400ed20 con 0x7f4c90004810 2024-05-17T21:57:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca4160370 con 0x7f4c90004810 2024-05-17T21:57:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca415e3b0 con 0x7f4c90004810 2024-05-17T21:57:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c9401e070 con 0x7f4c90004810 2024-05-17T21:57:43.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 0x7f4c84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4c94091bb0 con 0x7f4c90004810 2024-05-17T21:57:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.772+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 0x7f4c84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:43.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.776+0000 7f4ca37fe640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 0x7f4c84071940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ac000 tx=0x7f4c8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:43.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.780+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4c9405eb30 con 0x7f4c90004810 2024-05-17T21:57:43.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.960+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca41614c0 con 0x7f4c90004810 2024-05-17T21:57:43.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.960+0000 7f4ca0ff9640 1 -- 172.21.15.83:0/3208110847 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f4c9405e430 con 0x7f4c90004810 2024-05-17T21:57:43.954 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:43.954 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:57:43.954 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 msgr2=0x7f4c84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 0x7f4c84071940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0ac000 tx=0x7f4c8c009040 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 msgr2=0x7f4ca41f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c94010b10 tx=0x7f4c94005760 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 shutdown_connections 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca413bb20 0x7f4ca413bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4c8406f430 0x7f4c84071940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca4160b70 0x7f4ca4160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 --2- 172.21.15.83:0/3208110847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c90004810 0x7f4ca41f3700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:43.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 >> 172.21.15.83:0/3208110847 conn(0x7f4ca406cd90 msgr2=0x7f4ca40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 shutdown_connections 2024-05-17T21:57:43.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:43.964+0000 7f4caa7af640 1 -- 172.21.15.83:0/3208110847 wait complete. 2024-05-17T21:57:44.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T21:57:29.658174+0000', 'modified': '2024-05-17T21:57:32.679022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4269}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4269': {'gid': 4269, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/4099288168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 4099288168}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 4099288168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4269, 'qdb_cluster': [4269]} 2024-05-17T21:57:44.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T21:57:44.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T21:57:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.544+0000 7efe5065c640 1 Processor -- start 2024-05-17T21:57:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.544+0000 7efe5065c640 1 -- start start 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe5065c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe4806eec0 0x7efe480b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe5065c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe480b44c0 0x7efe480b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe5065c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4806f2d0 con 0x7efe4815be70 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe5065c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efe4806f5a0 con 0x7efe4806eec0 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe5065c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efe4806f880 con 0x7efe480b44c0 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efe4806eec0 0x7efe480b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4dbf7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efe4815be70 0x7efe481601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33774/0 (socket says 172.21.15.83:33774) 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4dbf7640 1 -- 172.21.15.83:0/1081384750 learned_addr learned my addr 172.21.15.83:0/1081384750 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4ebf9640 1 --2- 172.21.15.83:0/1081384750 >> v2:172.21.15.155:3300/0 conn(0x7efe480b44c0 0x7efe480b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 -- 172.21.15.83:0/1081384750 >> v2:172.21.15.155:3300/0 conn(0x7efe480b44c0 msgr2=0x7efe480b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/1081384750 >> v2:172.21.15.155:3300/0 conn(0x7efe480b44c0 0x7efe480b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 -- 172.21.15.83:0/1081384750 >> v1:172.21.15.83:6789/0 conn(0x7efe4815be70 legacy=0x7efe481601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 -- 172.21.15.83:0/1081384750 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4806fb60 con 0x7efe4806eec0 2024-05-17T21:57:44.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4d3f6640 1 -- 172.21.15.83:0/1081384750 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412663314 0 0) 0x7efe4806f2d0 con 0x7efe4815be70 2024-05-17T21:57:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/1081384750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4806eec0 0x7efe480b08a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe480ce2b0 tx=0x7efe3801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62469dba8b4a38f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4d3f6640 1 -- 172.21.15.83:0/1081384750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380b0070 con 0x7efe4806eec0 2024-05-17T21:57:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4d3f6640 1 -- 172.21.15.83:0/1081384750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe380ab400 con 0x7efe4806eec0 2024-05-17T21:57:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.548+0000 7efe4d3f6640 1 -- 172.21.15.83:0/1081384750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380ab7d0 con 0x7efe4806eec0 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- 172.21.15.83:0/1081384750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4806eec0 msgr2=0x7efe480b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- 172.21.15.83:0/1081384750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4806eec0 0x7efe480b08a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe480ce2b0 tx=0x7efe3801c3f0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- 172.21.15.83:0/1081384750 shutdown_connections 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- 172.21.15.83:0/1081384750 >> v2:172.21.15.155:3300/0 conn(0x7efe480b44c0 0x7efe480b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- 172.21.15.83:0/1081384750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4806eec0 0x7efe480b08a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- 172.21.15.83:0/1081384750 >> 172.21.15.83:0/1081384750 conn(0x7efe4806cd90 msgr2=0x7efe480b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- 172.21.15.83:0/1081384750 shutdown_connections 2024-05-17T21:57:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- 172.21.15.83:0/1081384750 wait complete. 2024-05-17T21:57:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 Processor -- start 2024-05-17T21:57:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- start start 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 0x7efe4813fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe480b44c0 0x7efe4815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe48073000 con 0x7efe4815be70 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe48072e80 con 0x7efe4806eec0 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.552+0000 7efe5065c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe48073180 con 0x7efe480b44c0 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39894/0 (socket says 172.21.15.83:39894) 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 -- 172.21.15.83:0/3618381397 learned_addr learned my addr 172.21.15.83:0/3618381397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 0x7efe4813fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe480b44c0 0x7efe4815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 -- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe480b44c0 msgr2=0x7efe4815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe480b44c0 0x7efe4815a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 -- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 msgr2=0x7efe4813fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 0x7efe4813fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 -- 172.21.15.83:0/3618381397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4815f920 con 0x7efe4815be70 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 0x7efe4813fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4ebf9640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe4000cc30 tx=0x7efe4000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 -- 172.21.15.83:0/3618381397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4000ce40 con 0x7efe4815be70 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 -- 172.21.15.83:0/3618381397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe400039a0 con 0x7efe4815be70 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48154760 con 0x7efe4815be70 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 -- 172.21.15.83:0/3618381397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40003d50 con 0x7efe4815be70 2024-05-17T21:57:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe48154d20 con 0x7efe4815be70 2024-05-17T21:57:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 -- 172.21.15.83:0/3618381397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe40003fb0 con 0x7efe4815be70 2024-05-17T21:57:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 0x7efe2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe3effd640 1 -- 172.21.15.83:0/3618381397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe4008f770 con 0x7efe4815be70 2024-05-17T21:57:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 0x7efe2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 0x7efe2c071910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe380ad000 tx=0x7efe38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.556+0000 7efe578d6000 1 Processor -- start 2024-05-17T21:57:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- start start 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a8bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4924d30 0x55bac4a8c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bac4929ca0 con 0x55bac4945fa0 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bac489c110 con 0x55bac493b600 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bac488bcb0 con 0x55bac4924d30 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39900/0 (socket says 172.21.15.83:39900) 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3945280781 learned_addr learned my addr 172.21.15.83:0/3945280781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4924d30 msgr2=0x55bac4a8c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4ebf9640 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4924d30 0x55bac4a8c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4924d30 0x55bac4a8c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a8bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 msgr2=0x55bac4a8bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a8bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3945280781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac4850370 con 0x55bac4945fa0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe38000c10 tx=0x7efe38004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9414de537f5797 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe2bfff640 1 -- 172.21.15.83:0/3945280781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380cc070 con 0x55bac4945fa0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe2bfff640 1 -- 172.21.15.83:0/3945280781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe380b1040 con 0x55bac4945fa0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe2bfff640 1 -- 172.21.15.83:0/3945280781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380c7470 con 0x55bac4945fa0 2024-05-17T21:57:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- 172.21.15.83:0/3945280781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 msgr2=0x55bac4a8b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efe38000c10 tx=0x7efe38004740 comp rx=0 tx=0).stop 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- 172.21.15.83:0/3945280781 shutdown_connections 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4924d30 0x55bac4a8c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a8bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 --2- 172.21.15.83:0/3945280781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4945fa0 0x55bac4a8b8b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- 172.21.15.83:0/3945280781 >> 172.21.15.83:0/3945280781 conn(0x55bac454d460 msgr2=0x55bac4a40e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- 172.21.15.83:0/3945280781 shutdown_connections 2024-05-17T21:57:44.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- 172.21.15.83:0/3945280781 wait complete. 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 -- start start 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.560+0000 7efe578d6000 1 Processor -- start 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a4d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bac4847d40 con 0x55bac4924d30 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bac488fcf0 con 0x55bac493b600 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bac4834e90 con 0x55bac4945fa0 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 10 client.0 _ll_register_callbacks cb 0x55bac4869fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39902/0 (socket says 172.21.15.83:39902) 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3505841679 learned_addr learned my addr 172.21.15.83:0/3505841679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 msgr2=0x55bac4a4d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a4d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 msgr2=0x55bac4a487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 -- 172.21.15.83:0/3505841679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac485a6e0 con 0x55bac4924d30 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe380c4040 tx=0x7efe3801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5534ef92b7e8617 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380cc040 con 0x55bac4924d30 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe380b1040 con 0x55bac4924d30 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe38003820 con 0x55bac4924d30 2024-05-17T21:57:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efe380dd050 con 0x55bac4924d30 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 10 client.4539 fetch_fsmap learned FSMap version 7 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bac484a690 con 0x55bac4924d30 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe380b1040 con 0x55bac4924d30 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7efe380e0020 con 0x55bac4924d30 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 10 client.4539 fetch_fsmap finished waiting for FSMap version 7 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 10 client.4539 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 20 client.4539 populate_metadata read hostname 'smithi083' 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac45327d0 0x55bac4a92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 4 client.4539 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 --> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac4827a00 con 0x55bac45327d0 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac45327d0 0x55bac4a92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 10 client.4539 ms_handle_connect on v2:172.21.15.83:6836/4099288168 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac45327d0 0x55bac4a92920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93717dbdb22830d server_cookie=45bc065bdf9bf713 in_seq=0 out_seq=0 2024-05-17T21:57:44.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 1 -- 172.21.15.83:0/3505841679 <== mds.0 v2:172.21.15.83:6836/4099288168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bac4827a00 con 0x55bac45327d0 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe297fa640 10 client.4539 handle_command_reply: tid=0 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.564+0000 7efe578d6000 1 client.4539 shutdown 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 msgr2=0x55bac4a48280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efe380c4040 tx=0x7efe3801ac30 comp rx=0 tx=0).stop 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 shutdown_connections 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac45327d0 0x55bac4a92920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a4d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac493b600 0x55bac4a487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 --2- 172.21.15.83:0/3505841679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4a48280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe297fa640 0 client.4539 ms_handle_reset on v2:172.21.15.83:6836/4099288168 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 >> 172.21.15.83:0/3505841679 conn(0x55bac454d460 msgr2=0x55bac4a40e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 shutdown_connections 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 1 -- 172.21.15.83:0/3505841679 wait complete. 2024-05-17T21:57:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.568+0000 7efe578d6000 20 client.4539 trim_cache size 0 max 16384 2024-05-17T21:57:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 Processor -- start 2024-05-17T21:57:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 -- start start 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4924d30 0x55bac4a40460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe4815c490 con 0x55bac493b600 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe4815cab0 con 0x55bac4924d30 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bac4836900 con 0x55bac4945fa0 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39916/0 (socket says 172.21.15.83:39916) 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 -- 172.21.15.83:0/4062908765 learned_addr learned my addr 172.21.15.83:0/4062908765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 -- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 msgr2=0x55bac4a40ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 -- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4924d30 msgr2=0x55bac4a40460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4924d30 0x55bac4a40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4924d30 0x55bac4a40460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 -- 172.21.15.83:0/4062908765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac4a44e90 con 0x55bac493b600 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efe44001540 tx=0x7efe44005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e888769f0c42cad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe2b7fe640 1 -- 172.21.15.83:0/4062908765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4401b070 con 0x55bac493b600 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe2b7fe640 1 -- 172.21.15.83:0/4062908765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440016d0 con 0x55bac493b600 2024-05-17T21:57:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.584+0000 7efe2b7fe640 1 -- 172.21.15.83:0/4062908765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44001b00 con 0x55bac493b600 2024-05-17T21:57:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/4062908765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 msgr2=0x55bac4a409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efe44001540 tx=0x7efe44005bd0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/4062908765 shutdown_connections 2024-05-17T21:57:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac4945fa0 0x55bac4a40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac493b600 0x55bac4a409a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- 172.21.15.83:0/4062908765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4924d30 0x55bac4a40460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/4062908765 >> 172.21.15.83:0/4062908765 conn(0x55bac454d460 msgr2=0x55bac4aa1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/4062908765 shutdown_connections 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/4062908765 wait complete. 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- start start 2024-05-17T21:57:44.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 Processor -- start 2024-05-17T21:57:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac493b600 0x55bac4afe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4945fa0 0x55bac4afe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bac488f000 con 0x55bac4924d30 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55bac4847d40 con 0x55bac4945fa0 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55bac489aee0 con 0x55bac493b600 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 10 client.0 _ll_register_callbacks cb 0x7efe380ca070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39918/0 (socket says 172.21.15.83:39918) 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 -- 172.21.15.83:0/2394854430 learned_addr learned my addr 172.21.15.83:0/2394854430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 -- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac493b600 msgr2=0x55bac4afe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac493b600 0x55bac4afe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 -- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4945fa0 msgr2=0x55bac4afe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4945fa0 0x55bac4afe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 -- 172.21.15.83:0/2394854430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac4912cb0 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe4e3f8640 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe38006f90 tx=0x7efe380a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff59ac2cb30aa412 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38003dc0 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe380bdb00 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe440016d0 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7efe380bdd00 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 10 client.4545 fetch_fsmap learned FSMap version 7 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bac484b4f0 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.588+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe440016d0 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7efe380e0040 con 0x55bac4924d30 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 10 client.4545 fetch_fsmap finished waiting for FSMap version 7 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 10 client.4545 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 20 client.4545 populate_metadata read hostname 'smithi083' 2024-05-17T21:57:44.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac4520830 0x55bac4ab9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 4 client.4545 mds_command: new command op to 4269 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 --> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] -- command(tid 0: {"prefix": "status"}) -- 0x55bac489e060 con 0x55bac4520830 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac4520830 0x55bac4ab9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe4dbf7640 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac4520830 0x55bac4ab9c80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae328958370a622 server_cookie=c22fccfd97ba5174 in_seq=0 out_seq=0 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe297fa640 10 client.4545 ms_handle_connect on v2:172.21.15.83:6836/4099288168 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe297fa640 1 -- 172.21.15.83:0/2394854430 <== mds.0 v2:172.21.15.83:6836/4099288168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55bac489e060 con 0x55bac4520830 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe297fa640 10 client.4545 handle_command_reply: tid=0 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 client.4545 shutdown 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 msgr2=0x55bac4afdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efe38006f90 tx=0x7efe380a6000 comp rx=0 tx=0).stop 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 shutdown_connections 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:6836/4099288168,v1:172.21.15.83:6837/4099288168] conn(0x55bac4520830 0x55bac4ab9c80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bac4945fa0 0x55bac4afe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55bac493b600 0x55bac4afe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 --2- 172.21.15.83:0/2394854430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bac4924d30 0x55bac4afdd90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe297fa640 0 client.4545 ms_handle_reset on v2:172.21.15.83:6836/4099288168 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 >> 172.21.15.83:0/2394854430 conn(0x55bac454d460 msgr2=0x55bac4a8b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 shutdown_connections 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 1 -- 172.21.15.83:0/2394854430 wait complete. 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe578d6000 20 client.4545 trim_cache size 0 max 16384 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T21:57:44.589 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4269, 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 14.340520906, 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 7, 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 20, 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 20, 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 19.844457548000001 2024-05-17T21:57:44.590 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 msgr2=0x7efe2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 0x7efe2c071910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe380ad000 tx=0x7efe38008040 comp rx=0 tx=0).stop 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 msgr2=0x7efe4815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe4000cc30 tx=0x7efe4000a5c0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 shutdown_connections 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe4815be70 0x7efe4815adc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe480b44c0 0x7efe4815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe2c06f400 0x7efe2c071910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 --2- 172.21.15.83:0/3618381397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe4806eec0 0x7efe4813fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 >> 172.21.15.83:0/3618381397 conn(0x7efe4806cd90 msgr2=0x7efe480b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 shutdown_connections 2024-05-17T21:57:44.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.592+0000 7efe5065c640 1 -- 172.21.15.83:0/3618381397 wait complete. 2024-05-17T21:57:44.602 INFO:teuthology.run_tasks:Running task sequential... 2024-05-17T21:57:44.613 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-17T21:57:44.613 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-17T21:57:44.614 INFO:teuthology.task.print:Enabling mgr modules 2024-05-17T21:57:44.614 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-17T21:57:44.670 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-17T21:57:44.670 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-17T21:57:44.670 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-17T21:57:44.670 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:44.671 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:44.671 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:44.671 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-17T21:57:44.671 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:44.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:44.672 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T21:57:44.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:44.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-05-17T21:57:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-05-17T21:57:44.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:57:44.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:44.740 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-05-17T21:57:44.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:44.747 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-05-17T21:57:44.807 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2024-05-17T21:57:44.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:57:44.808 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-17T21:57:44.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:57:44.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.968+0000 7f672cda5640 1 Processor -- start 2024-05-17T21:57:44.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.968+0000 7f672cda5640 1 -- start start 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672cda5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67280d1040 0x7f6728144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672cda5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6728073540 0x7f672806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672cda5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672815ea40 con 0x7f672815e080 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672cda5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f672815ed20 con 0x7f67280d1040 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672cda5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f672815f000 con 0x7f6728073540 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67280d1040 0x7f6728144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f6725d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f672815e080 0x7f6728072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33790/0 (socket says 172.21.15.83:33790) 2024-05-17T21:57:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f6725d9b640 1 -- 172.21.15.83:0/3062616919 learned_addr learned my addr 172.21.15.83:0/3062616919 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672659c640 1 -- 172.21.15.83:0/3062616919 >> v2:172.21.15.155:3300/0 conn(0x7f6728073540 msgr2=0x7f672806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f6726d9d640 1 --2- 172.21.15.83:0/3062616919 >> v2:172.21.15.155:3300/0 conn(0x7f6728073540 0x7f672806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672659c640 1 --2- 172.21.15.83:0/3062616919 >> v2:172.21.15.155:3300/0 conn(0x7f6728073540 0x7f672806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672659c640 1 -- 172.21.15.83:0/3062616919 >> v1:172.21.15.83:6789/0 conn(0x7f672815e080 legacy=0x7f6728072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672559a640 1 -- 172.21.15.83:0/3062616919 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239772277 0 0) 0x7f672815ea40 con 0x7f672815e080 2024-05-17T21:57:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.972+0000 7f672659c640 1 -- 172.21.15.83:0/3062616919 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672806ef30 con 0x7f67280d1040 2024-05-17T21:57:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672659c640 1 --2- 172.21.15.83:0/3062616919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67280d1040 0x7f6728144080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6728072c30 tx=0x7f671801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cca673945565d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672559a640 1 -- 172.21.15.83:0/3062616919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180b1070 con 0x7f67280d1040 2024-05-17T21:57:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672559a640 1 -- 172.21.15.83:0/3062616919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67180ac5f0 con 0x7f67280d1040 2024-05-17T21:57:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672559a640 1 -- 172.21.15.83:0/3062616919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180ac9c0 con 0x7f67280d1040 2024-05-17T21:57:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- 172.21.15.83:0/3062616919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67280d1040 msgr2=0x7f6728144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 --2- 172.21.15.83:0/3062616919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67280d1040 0x7f6728144080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6728072c30 tx=0x7f671801c480 comp rx=0 tx=0).stop 2024-05-17T21:57:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- 172.21.15.83:0/3062616919 shutdown_connections 2024-05-17T21:57:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 --2- 172.21.15.83:0/3062616919 >> v2:172.21.15.155:3300/0 conn(0x7f6728073540 0x7f672806f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 --2- 172.21.15.83:0/3062616919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67280d1040 0x7f6728144080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- 172.21.15.83:0/3062616919 >> 172.21.15.83:0/3062616919 conn(0x7f672806cd90 msgr2=0x7f67280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- 172.21.15.83:0/3062616919 shutdown_connections 2024-05-17T21:57:44.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- 172.21.15.83:0/3062616919 wait complete. 2024-05-17T21:57:44.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 Processor -- start 2024-05-17T21:57:44.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.976+0000 7f672cda5640 1 -- start start 2024-05-17T21:57:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67280d1040 0x7f67280aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 0x7f672813d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6728070810 con 0x7f67281408f0 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6728070690 con 0x7f672815e080 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6728070990 con 0x7f67280d1040 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39922/0 (socket says 172.21.15.83:39922) 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 -- 172.21.15.83:0/3852998805 learned_addr learned my addr 172.21.15.83:0/3852998805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6725d9b640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 0x7f672813d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 -- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67280d1040 msgr2=0x7f67280aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67280d1040 0x7f67280aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 -- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 msgr2=0x7f672813d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:44.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 0x7f672813d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 -- 172.21.15.83:0/3852998805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67280aaad0 con 0x7f67281408f0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6725d9b640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 0x7f672813d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6726d9d640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f672000cc00 tx=0x7f672000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000ce10 con 0x7f67281408f0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67200039a0 con 0x7f67281408f0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003d70 con 0x7f67281408f0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67280aadb0 con 0x7f67281408f0 2024-05-17T21:57:44.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67280ab370 con 0x7f67281408f0 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672806f730 con 0x7f67281408f0 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6720003f10 con 0x7f67281408f0 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.980+0000 7f6716ffd640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 0x7f6708071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.984+0000 7f672659c640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 0x7f6708071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.984+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f672008f980 con 0x7f67281408f0 2024-05-17T21:57:44.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.984+0000 7f672659c640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 0x7f6708071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f67180ae000 tx=0x7f6718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:44.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:44.988+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6720014030 con 0x7f67281408f0 2024-05-17T21:57:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.188+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67280aa860 con 0x7f67281408f0 2024-05-17T21:57:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.188+0000 7f6716ffd640 1 -- 172.21.15.83:0/3852998805 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f672005c900 con 0x7f67281408f0 2024-05-17T21:57:45.181 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:57:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 msgr2=0x7f6708071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 0x7f6708071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f67180ae000 tx=0x7f6718008040 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 msgr2=0x7f672813fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f672000cc00 tx=0x7f672000a5c0 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 shutdown_connections 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281408f0 0x7f672813fd10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f670806f400 0x7f6708071910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f672815e080 0x7f672813d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 --2- 172.21.15.83:0/3852998805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67280d1040 0x7f67280aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 >> 172.21.15.83:0/3852998805 conn(0x7f672806cd90 msgr2=0x7f67280b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 shutdown_connections 2024-05-17T21:57:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45.192+0000 7f672cda5640 1 -- 172.21.15.83:0/3852998805 wait complete. 2024-05-17T21:57:45.202 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:57:45.202 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:57:45.202 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T21:57:45.202 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T21:57:45.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:45.203 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:57:45.203 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:57:45.203 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:57:45.203 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:57:45.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:45.203 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:57:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42479sec preferred_lft 42479sec 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:57:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:45.211 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T21:57:45.211 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T21:57:45.212 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:45.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:45.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:45.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:45.336 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:57:45.346 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T21:57:45.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:45.347 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T21:57:45.420 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T21:57:45.420 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:57:45.420 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:57:45.420 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:57:45.420 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:45.421 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:57:45.422 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:45.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:45.533 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T21:57:45.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:45.580 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-05-17T21:57:45.627 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:45.627 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:45.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:45.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-17T21:57:45.627 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:45.743 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-17T21:57:45.743 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:45.743 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-17T21:57:45.744 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:45.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:45.846 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:45.846 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:45.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T21:57:45.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T21:57:45.847 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:45.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:45.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:45.946 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T21:57:45.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:45.946 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:45.991 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:57:45.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:45.991 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:46.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:46.096 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:46.144 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:46.145 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:46.146 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:46.147 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:46.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:46.148 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:46.195 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:57:46.196 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T21:57:46.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:46.255 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:46.303 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:46.304 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:46.305 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:46.306 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:46.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:46.306 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:46.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T21:57:46.356+0000 7fad8e9da580 -1 init, newargv = 0x55ab6286f080 newargc=15 2024-05-17T21:57:46.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[34974]: starting ceph client 2024-05-17T21:57:46.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[34974]: starting fuse 2024-05-17T21:57:47.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:47.470 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:47.517 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:47.519 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:47.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:47.520 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:47.571 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:57:47.572 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T21:57:47.572 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:57:47.573 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T21:57:47.574 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:57:47.697 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.34974.asok 2024-05-17T21:57:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:47.709 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34974.asok 2024-05-17T21:57:47.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:47.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34974.asok status 2024-05-17T21:57:47.885 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:57:47.885 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "34974", 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4560, 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T21:57:47.886 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4560 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3772506329 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3772506329 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4560 192.168.0.1:0/3772506329", 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/3772506329", 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 7, 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 20, 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T21:57:47.888 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:57:47.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:57:48.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 Processor -- start 2024-05-17T21:57:48.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 -- start start 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f001607d0 0x7f2f00070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0006f6d0 con 0x7f2f0006d970 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0006f9b0 con 0x7f2f0015d1b0 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2f05835640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0006fc90 con 0x7f2f001607d0 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efe7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2f0006d970 0x7f2f000b6310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47438/0 (socket says 172.21.15.83:47438) 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efe7fc640 1 -- 172.21.15.83:0/3243387271 learned_addr learned my addr 172.21.15.83:0/3243387271 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> v2:172.21.15.155:3300/0 conn(0x7f2f001607d0 0x7f2f00070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294587301 0 0) 0x7f2f0006f6d0 con 0x7f2f0006d970 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2eec004640 con 0x7f2f0006d970 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1563312475 0 0) 0x7f2eec004640 con 0x7f2f0006d970 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 >> v2:172.21.15.155:3300/0 conn(0x7f2f001607d0 msgr2=0x7f2f00070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 --2- 172.21.15.83:0/3243387271 >> v2:172.21.15.155:3300/0 conn(0x7f2f001607d0 0x7f2f00070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 >> v1:172.21.15.155:6789/0 conn(0x7f2f0015d1b0 legacy=0x7f2f000b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0006ff70 con 0x7f2f0006d970 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2ef0001700 con 0x7f2f0006d970 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> v2:172.21.15.155:3300/0 conn(0x7f2f001607d0 0x7f2f00070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 >> v1:172.21.15.83:6789/0 conn(0x7f2f0006d970 legacy=0x7f2f000b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 0x7f2eec004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eec005920 0x7f2eec005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec006420 0x7f2eec00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0006fc90 con 0x7f2eec004b70 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00d3c0 con 0x7f2eec005920 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec00d5c0 con 0x7f2eec006420 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 0x7f2eec004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2eff7fe640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec006420 0x7f2eec00cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 -- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec006420 msgr2=0x7f2eec00cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec006420 0x7f2eec00cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 -- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eec005920 msgr2=0x7f2eec005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2eec005920 0x7f2eec005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2efeffd640 1 -- 172.21.15.83:0/3243387271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec004640 con 0x7f2eec004b70 2024-05-17T21:57:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.060+0000 7f2eff7fe640 1 -- 172.21.15.83:0/3243387271 reap_dead start 2024-05-17T21:57:48.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2efeffd640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 0x7f2eec004fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098d0 tx=0x7f2ee801c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d624d9dac246498 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:48.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800f040 con 0x7f2eec004b70 2024-05-17T21:57:48.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee80af070 con 0x7f2eec004b70 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2efdffb640 1 -- 172.21.15.83:0/3243387271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80aa810 con 0x7f2eec004b70 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 -- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 msgr2=0x7f2eec004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 0x7f2eec004fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80098d0 tx=0x7f2ee801c5f0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 -- 172.21.15.83:0/3243387271 shutdown_connections 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 --2- 172.21.15.83:0/3243387271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2eec004b70 0x7f2eec004fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 -- 172.21.15.83:0/3243387271 >> 172.21.15.83:0/3243387271 conn(0x7f2f00073670 msgr2=0x7f2f00073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 -- 172.21.15.83:0/3243387271 shutdown_connections 2024-05-17T21:57:48.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.064+0000 7f2f05835640 1 -- 172.21.15.83:0/3243387271 wait complete. 2024-05-17T21:57:48.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 Processor -- start 2024-05-17T21:57:48.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 -- start start 2024-05-17T21:57:48.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec004b70 0x7f2f00159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 0x7f2f00156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b66a0 con 0x7f2f00159d50 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b6520 con 0x7f2f0014aca0 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000b6820 con 0x7f2eec004b70 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54314/0 (socket says 172.21.15.83:54314) 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 -- 172.21.15.83:0/453046136 learned_addr learned my addr 172.21.15.83:0/453046136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2eff7fe640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 0x7f2f00156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efeffd640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec004b70 0x7f2f00159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 -- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec004b70 msgr2=0x7f2f00159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec004b70 0x7f2f00159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 -- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 msgr2=0x7f2f00156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 0x7f2f00156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 -- 172.21.15.83:0/453046136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f00076f50 con 0x7f2f00159d50 2024-05-17T21:57:48.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2eff7fe640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 0x7f2f00156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:48.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2efe7fc640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c7d0 tx=0x7f2ef000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:48.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00074e0 con 0x7f2f00159d50 2024-05-17T21:57:48.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef00047f0 con 0x7f2f00159d50 2024-05-17T21:57:48.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0007680 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.068+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0015f900 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0014b0b0 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f000b4c10 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ef0011070 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2ef7fff640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 0x7f2ee0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef008f000 con 0x7f2f00159d50 2024-05-17T21:57:48.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2efeffd640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 0x7f2ee0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:48.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.072+0000 7f2efeffd640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 0x7f2ee0071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80ae000 tx=0x7f2ee80046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:48.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.076+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2ef005bef0 con 0x7f2f00159d50 2024-05-17T21:57:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f00157b10 con 0x7f2f00159d50 2024-05-17T21:57:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2ef7fff640 1 -- 172.21.15.83:0/453046136 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2ef005b7f0 con 0x7f2f00159d50 2024-05-17T21:57:48.257 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 msgr2=0x7f2ee0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 0x7f2ee0071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80ae000 tx=0x7f2ee80046d0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 msgr2=0x7f2f0014a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c7d0 tx=0x7f2ef000cc90 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 shutdown_connections 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ee006f400 0x7f2ee0071910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f0014aca0 0x7f2f00156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f00159d50 0x7f2f0014a760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 --2- 172.21.15.83:0/453046136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2eec004b70 0x7f2f00159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:48.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 >> 172.21.15.83:0/453046136 conn(0x7f2f00073670 msgr2=0x7f2f000b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 shutdown_connections 2024-05-17T21:57:48.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48.264+0000 7f2f05835640 1 -- 172.21.15.83:0/453046136 wait complete. 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:57:48.276 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-17T21:57:48.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.276 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.282 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42476sec preferred_lft 42476sec 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.283 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.285 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:57:48.333 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T21:57:48.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.335 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:48.392 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:48.393 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:48.393 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:57:48.393 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:48.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.512 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T21:57:48.559 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T21:57:48.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.560 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-05-17T21:57:48.607 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-17T21:57:48.608 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:48.608 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:48.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:48.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-17T21:57:48.608 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:48.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-17T21:57:48.737 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:48.737 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 56:c0:37:6c:07:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54c0:37ff:fe6c:750/64 scope link 2024-05-17T21:57:48.738 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:48.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-17T21:57:48.794 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:48.794 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:48.794 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 56:c0:37:6c:07:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54c0:37ff:fe6c:750/64 scope link 2024-05-17T21:57:48.795 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:48.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-17T21:57:48.796 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-17T21:57:48.797 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:48.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:48.940 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:48.940 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:57:48.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2024-05-17T21:57:48.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2024-05-17T21:57:48.941 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:57:48.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:49.003 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-17T21:57:49.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:49.003 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:49.051 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-17T21:57:49.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:49.052 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:49.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:49.156 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:49.205 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:49.205 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:49.205 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:49.206 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:49.207 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:49.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:49.208 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:49.251 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:57:49.251 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-05-17T21:57:49.251 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-17T21:57:49.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:49.309 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:49.356 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:49.356 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:49.356 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:49.356 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:49.356 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:49.357 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:49.358 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:49.359 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:49.359 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:49.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:49.359 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:49.406 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:57:49.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:2024-05-17T21:57:49.416+0000 7fd7082fc580 -1 init, newargv = 0x7fd6ec001f10 newargc=15 2024-05-17T21:57:49.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:ceph-fuse[35119]: starting ceph client 2024-05-17T21:57:49.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi083.stderr:ceph-fuse[35119]: starting fuse 2024-05-17T21:57:50.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:57:50.524 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:50.572 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:57:50.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:57:50.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:57:50.575 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:50.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:50.576 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:57:50.624 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:57:50.624 INFO:teuthology.orchestra.run.smithi083.stdout:46 2024-05-17T21:57:50.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T21:57:50.625 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.2")) 2024-05-17T21:57:50.626 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:57:50.746 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.2.35119.asok 2024-05-17T21:57:50.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:50.758 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35119.asok 2024-05-17T21:57:50.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:50.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35119.asok status 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "2", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "35119", 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T21:57:50.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4572, 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4572 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.2:0", 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 540537441 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:57:50.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.2:0", 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 540537441 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4572 192.168.0.2:0/540537441", 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.2:0/540537441", 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 7, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 20, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T21:57:50.937 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:57:50.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:57:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 Processor -- start 2024-05-17T21:57:51.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 -- start start 2024-05-17T21:57:51.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140717e0 0x7f7e141601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e141606f0 0x7f7e140b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e14160f50 con 0x7f7e1406f3c0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1406e690 con 0x7f7e141606f0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1c027640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1406e8e0 con 0x7f7e140717e0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e19dc3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7e1406f3c0 0x7f7e140710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47452/0 (socket says 172.21.15.83:47452) 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e19dc3640 1 -- 172.21.15.83:0/1830470819 learned_addr learned my addr 172.21.15.83:0/1830470819 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e1a5c4640 1 --2- 172.21.15.83:0/1830470819 >> v2:172.21.15.155:3300/0 conn(0x7f7e141606f0 0x7f7e140b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e195c2640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140717e0 0x7f7e141601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805063083 0 0) 0x7f7e14160f50 con 0x7f7e1406f3c0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.108+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dfc003530 con 0x7f7e1406f3c0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1314838444 0 0) 0x7f7dfc003530 con 0x7f7e1406f3c0 2024-05-17T21:57:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140717e0 msgr2=0x7f7e141601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140717e0 0x7f7e141601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 >> v2:172.21.15.155:3300/0 conn(0x7f7e141606f0 msgr2=0x7f7e140b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 --2- 172.21.15.83:0/1830470819 >> v2:172.21.15.155:3300/0 conn(0x7f7e141606f0 0x7f7e140b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1406ebc0 con 0x7f7e1406f3c0 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f7e04001700 con 0x7f7e1406f3c0 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 >> v1:172.21.15.83:6789/0 conn(0x7f7e1406f3c0 legacy=0x7f7e140710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc003a10 0x7f7dfc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dfc004810 0x7f7dfc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7dfc005160 0x7f7dfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1406e690 con 0x7f7dfc003a10 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1406e8e0 con 0x7f7dfc004810 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7dfc00bf10 con 0x7f7dfc005160 2024-05-17T21:57:51.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc003a10 0x7f7dfc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e195c2640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dfc004810 0x7f7dfc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e1a5c4640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7dfc005160 0x7f7dfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 -- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7dfc005160 msgr2=0x7f7dfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7dfc005160 0x7f7dfc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 -- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dfc004810 msgr2=0x7f7dfc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7dfc004810 0x7f7dfc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 -- 172.21.15.83:0/1830470819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc003530 con 0x7f7dfc003a10 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc003a10 0x7f7dfc003e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c240 tx=0x7f7e0401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebeaaf05499df94 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04007d60 con 0x7f7dfc003a10 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e040a9070 con 0x7f7dfc003a10 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e18dc1640 1 -- 172.21.15.83:0/1830470819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14160d10 con 0x7f7dfc003a10 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e1c027640 1 -- 172.21.15.83:0/1830470819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc003a10 msgr2=0x7f7dfc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e1c027640 1 --2- 172.21.15.83:0/1830470819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc003a10 0x7f7dfc003e80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c240 tx=0x7f7e0401cc10 comp rx=0 tx=0).stop 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.112+0000 7f7e1a5c4640 1 -- 172.21.15.83:0/1830470819 reap_dead start 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- 172.21.15.83:0/1830470819 shutdown_connections 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- 172.21.15.83:0/1830470819 >> 172.21.15.83:0/1830470819 conn(0x7f7e14068730 msgr2=0x7f7e14068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- 172.21.15.83:0/1830470819 shutdown_connections 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- 172.21.15.83:0/1830470819 wait complete. 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 Processor -- start 2024-05-17T21:57:51.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- start start 2024-05-17T21:57:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 0x7f7e140d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 0x7f7e140b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140717e0 con 0x7f7e14141ea0 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14157060 con 0x7f7e1413fe10 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1c027640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7e14071960 con 0x7f7e140b3130 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54338/0 (socket says 172.21.15.83:54338) 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 -- 172.21.15.83:0/292556351 learned_addr learned my addr 172.21.15.83:0/292556351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e195c2640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 0x7f7e140d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e1a5c4640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 0x7f7e140b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 -- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 msgr2=0x7f7e140b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 0x7f7e140b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 -- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 msgr2=0x7f7e140d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.116+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 0x7f7e140d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e19dc3640 1 -- 172.21.15.83:0/292556351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140b3a80 con 0x7f7e14141ea0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e195c2640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 0x7f7e140d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e1a5c4640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 0x7f7e140b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e19dc3640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c240 tx=0x7f7dfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04007d60 con 0x7f7e14141ea0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e040048b0 con 0x7f7e14141ea0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e141511f0 con 0x7f7e14141ea0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004ca0 con 0x7f7e14141ea0 2024-05-17T21:57:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141426f0 con 0x7f7e14141ea0 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e140b69d0 con 0x7f7e14141ea0 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e04007180 con 0x7f7e14141ea0 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 0x7f7e00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e195c2640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 0x7f7e00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e04128a50 con 0x7f7e14141ea0 2024-05-17T21:57:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.120+0000 7f7e195c2640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 0x7f7e00071910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007520 tx=0x7f7e0800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:51.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.124+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7e140b3a80 con 0x7f7e14141ea0 2024-05-17T21:57:51.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.324+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7e140d0180 con 0x7f7e14141ea0 2024-05-17T21:57:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.324+0000 7f7e127fc640 1 -- 172.21.15.83:0/292556351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f7e040169d0 con 0x7f7e14141ea0 2024-05-17T21:57:51.318 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:57:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 msgr2=0x7f7e00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 0x7f7e00071910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007520 tx=0x7f7e0800b040 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 msgr2=0x7f7e140d2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400c240 tx=0x7f7dfc005840 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 shutdown_connections 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e140b3130 0x7f7e140b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7e0006f400 0x7f7e00071910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e1413fe10 0x7f7e140d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 --2- 172.21.15.83:0/292556351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e14141ea0 0x7f7e140d2c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 >> 172.21.15.83:0/292556351 conn(0x7f7e14068730 msgr2=0x7f7e14165220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 shutdown_connections 2024-05-17T21:57:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:51.328+0000 7f7e1c027640 1 -- 172.21.15.83:0/292556351 wait complete. 2024-05-17T21:57:51.339 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:57:51.339 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:57:51.340 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-17T21:57:51.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:51.340 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:51.347 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42481sec preferred_lft 42481sec 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:fc:50:90:d6 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:57:51.348 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:51.349 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-17T21:57:51.349 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:51.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:51.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:51.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:51.480 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:57:51.489 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-05-17T21:57:51.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:51.490 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-05-17T21:57:51.553 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-05-17T21:57:51.553 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:51.554 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:57:51.554 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:51.554 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:51.555 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:57:51.555 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:57:51.555 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:57:51.555 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:51.639 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:51.667 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:51.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:51.675 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-05-17T21:57:51.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:51.680 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-05-17T21:57:51.727 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:51.727 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:51.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:51.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-17T21:57:51.727 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:51.816 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:51.840 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:51.847 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-17T21:57:51.848 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:51.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:51.956 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:51.963 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:51.963 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:51.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2024-05-17T21:57:51.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2024-05-17T21:57:51.963 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:52.045 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:52.061 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:52.070 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-17T21:57:52.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:52.070 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:52.123 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-17T21:57:52.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:52.123 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:52.170 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:52.229 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:52.280 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:52.281 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:52.282 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:52.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:52.283 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:52.331 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:57:52.331 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-17T21:57:52.376 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:52.390 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:52.439 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:52.440 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:52.441 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:52.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:52.442 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:52.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2024-05-17T21:57:52.486+0000 7fd8e6711580 -1 init, newargv = 0x7fd8d400a430 newargc=15 2024-05-17T21:57:52.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[33464]: starting ceph client 2024-05-17T21:57:52.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[33464]: starting fuse 2024-05-17T21:57:53.587 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:53.604 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:53.657 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:53.658 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:53.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:53.659 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:53.703 INFO:teuthology.orchestra.run.smithi155.stdout:45 2024-05-17T21:57:53.705 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T21:57:53.705 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> import re 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-17T21:57:53.706 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> return f 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:53.707 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2024-05-17T21:57:53.708 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-05-17T21:57:53.826 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.33464.asok 2024-05-17T21:57:53.830 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:53.837 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33464.asok 2024-05-17T21:57:53.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:53.838 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33464.asok status 2024-05-17T21:57:54.011 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-17T21:57:54.011 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "33464", 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4385, 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2024-05-17T21:57:54.012 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4385 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2306001541 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2306001541 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4385 192.168.0.1:0/2306001541", 2024-05-17T21:57:54.013 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/2306001541", 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 7, 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 20, 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2024-05-17T21:57:54.014 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-17T21:57:54.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:57:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 Processor -- start 2024-05-17T21:57:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 -- start start 2024-05-17T21:57:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8c81461c0 0x7fe8c815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8c815e6b0 0x7fe8c806ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c81465d0 con 0x7fe8c8070510 2024-05-17T21:57:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c806ef30 con 0x7fe8c81461c0 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cf1f0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c806f210 con 0x7fe8c815e6b0 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8c81461c0 0x7fe8c815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8c81461c0 0x7fe8c815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54352/0 (socket says 172.21.15.83:54352) 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2598884728 learned_addr learned my addr 172.21.15.83:0/2598884728 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8cd78d640 1 --1- 172.21.15.83:0/2598884728 >> v1:172.21.15.83:6789/0 conn(0x7fe8c8070510 0x7fe8c81605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47464/0 (socket says 172.21.15.83:47464) 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8c7fff640 1 --2- 172.21.15.83:0/2598884728 >> v2:172.21.15.155:3300/0 conn(0x7fe8c815e6b0 0x7fe8c806ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2598884728 >> v2:172.21.15.155:3300/0 conn(0x7fe8c815e6b0 msgr2=0x7fe8c806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 --2- 172.21.15.83:0/2598884728 >> v2:172.21.15.155:3300/0 conn(0x7fe8c815e6b0 0x7fe8c806ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2598884728 >> v1:172.21.15.83:6789/0 conn(0x7fe8c8070510 legacy=0x7fe8c81605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8c77fe640 1 -- 172.21.15.83:0/2598884728 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1053226108 0 0) 0x7fe8c81465d0 con 0x7fe8c8070510 2024-05-17T21:57:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.196+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2598884728 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8070920 con 0x7fe8c81461c0 2024-05-17T21:57:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8ccf8c640 1 --2- 172.21.15.83:0/2598884728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c81461c0 0x7fe8c815e170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8071230 tx=0x7fe8b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c41395d4b5e2fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:54.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8c77fe640 1 -- 172.21.15.83:0/2598884728 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80aa650 con 0x7fe8c81461c0 2024-05-17T21:57:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8c77fe640 1 -- 172.21.15.83:0/2598884728 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b80af070 con 0x7fe8c81461c0 2024-05-17T21:57:54.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8c77fe640 1 -- 172.21.15.83:0/2598884728 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80aaa10 con 0x7fe8c81461c0 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2598884728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c81461c0 msgr2=0x7fe8c815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2598884728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c81461c0 0x7fe8c815e170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8071230 tx=0x7fe8b801c4f0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2598884728 shutdown_connections 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2598884728 >> v2:172.21.15.155:3300/0 conn(0x7fe8c815e6b0 0x7fe8c806ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2598884728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c81461c0 0x7fe8c815e170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2598884728 >> 172.21.15.83:0/2598884728 conn(0x7fe8c806cd90 msgr2=0x7fe8c80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2598884728 shutdown_connections 2024-05-17T21:57:54.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2598884728 wait complete. 2024-05-17T21:57:54.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 Processor -- start 2024-05-17T21:57:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.200+0000 7fe8cf1f0640 1 -- start start 2024-05-17T21:57:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c81461c0 0x7fe8c813aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8c815e6b0 0x7fe8c8144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8160e30 con 0x7fe8c8070510 2024-05-17T21:57:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8160cb0 con 0x7fe8c81461c0 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c80d10a0 con 0x7fe8c815e6b0 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54360/0 (socket says 172.21.15.83:54360) 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2059338122 learned_addr learned my addr 172.21.15.83:0/2059338122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8c815e6b0 msgr2=0x7fe8c8144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8c815e6b0 0x7fe8c8144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c81461c0 msgr2=0x7fe8c813aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c81461c0 0x7fe8c813aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 -- 172.21.15.83:0/2059338122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c8143a80 con 0x7fe8c8070510 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8ccf8c640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8019100 tx=0x7fe8b801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80bd5f0 con 0x7fe8c8070510 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b80a6040 con 0x7fe8c8070510 2024-05-17T21:57:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80bd9f0 con 0x7fe8c8070510 2024-05-17T21:57:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c8143d60 con 0x7fe8c8070510 2024-05-17T21:57:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c8144320 con 0x7fe8c8070510 2024-05-17T21:57:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c815e170 con 0x7fe8c8070510 2024-05-17T21:57:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.204+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8b80bdb90 con 0x7fe8c8070510 2024-05-17T21:57:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.208+0000 7fe8c57fa640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 0x7fe8a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.208+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8b8129660 con 0x7fe8c8070510 2024-05-17T21:57:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.208+0000 7fe8c7fff640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 0x7fe8a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.208+0000 7fe8c7fff640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 0x7fe8a8071940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe8c815f1f0 tx=0x7fe8b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.208+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe8b80c3060 con 0x7fe8c8070510 2024-05-17T21:57:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.404+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8c80b41a0 con 0x7fe8c8070510 2024-05-17T21:57:54.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.404+0000 7fe8c57fa640 1 -- 172.21.15.83:0/2059338122 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fe8b8016110 con 0x7fe8c8070510 2024-05-17T21:57:54.399 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:57:54.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 msgr2=0x7fe8a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 0x7fe8a8071940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe8c815f1f0 tx=0x7fe8b0008040 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 msgr2=0x7fe8c80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8019100 tx=0x7fe8b801ad30 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 shutdown_connections 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8c815e6b0 0x7fe8c8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe8a806f430 0x7fe8a8071940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8c81461c0 0x7fe8c813aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 --2- 172.21.15.83:0/2059338122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c8070510 0x7fe8c80b2930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 >> 172.21.15.83:0/2059338122 conn(0x7fe8c806cd90 msgr2=0x7fe8c80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 shutdown_connections 2024-05-17T21:57:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:54.408+0000 7fe8cf1f0640 1 -- 172.21.15.83:0/2059338122 wait complete. 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:57:54.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-17T21:57:54.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.418 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2024-05-17T21:57:54.425 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:54.425 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:54.425 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:54.425 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:57:54.426 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42478sec preferred_lft 42478sec 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 02:42:fc:50:90:d6 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:54.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T21:57:54.428 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.428 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:57:54.479 INFO:teuthology.orchestra.run.smithi155.stdout:1 2024-05-17T21:57:54.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.481 DEBUG:teuthology.orchestra.run.smithi155:> route 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:54.530 INFO:teuthology.orchestra.run.smithi155.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:57:54.531 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:57:54.531 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:54.620 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:54.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:54.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.656 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-05-17T21:57:54.661 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-17T21:57:54.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.662 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2024-05-17T21:57:54.706 INFO:teuthology.orchestra.run.smithi155.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-17T21:57:54.706 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:54.707 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:54.707 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:54.707 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-17T21:57:54.707 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:54.796 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:54.816 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:54.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-17T21:57:54.836 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:54.836 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:54.836 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:54.836 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5a:cf:3f:3e:d0:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::58cf:3fff:fe3e:d0ec/64 scope link 2024-05-17T21:57:54.837 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:54.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2024-05-17T21:57:54.892 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 5a:cf:3f:3e:d0:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::58cf:3fff:fe3e:d0ec/64 scope link 2024-05-17T21:57:54.893 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:57:54.894 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-17T21:57:54.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-17T21:57:54.895 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:54.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:55.031 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:55.038 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:57:55.038 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2024-05-17T21:57:55.038 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.1 up 2024-05-17T21:57:55.038 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.1 master ceph-brx 2024-05-17T21:57:55.038 DEBUG:teuthology.orchestra.run.smithi155:> ') 2024-05-17T21:57:55.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:57:55.093 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:55.100 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-17T21:57:55.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:55.100 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:55.147 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-17T21:57:55.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:57:55.147 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:55.194 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:55.251 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:55.299 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:55.300 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:55.301 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:55.302 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:55.303 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:55.304 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:55.304 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:55.304 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:55.304 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:55.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:55.305 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:55.351 INFO:teuthology.orchestra.run.smithi155.stdout:45 2024-05-17T21:57:55.351 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-05-17T21:57:55.351 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-17T21:57:55.396 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:55.411 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:55.460 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:55.460 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:55.461 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:55.462 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:55.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:55.463 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:55.506 INFO:teuthology.orchestra.run.smithi155.stdout:45 2024-05-17T21:57:55.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:2024-05-17T21:57:55.514+0000 7ffb177ec580 -1 init, newargv = 0x7ffb08001f10 newargc=15 2024-05-17T21:57:55.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[33583]: starting ceph client 2024-05-17T21:57:55.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[33583]: starting fuse 2024-05-17T21:57:56.607 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2024-05-17T21:57:56.624 DEBUG:teuthology.orchestra.run.smithi155:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/sys 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/proc 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/dev 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/pts 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/run 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/ 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/security 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/shm 2024-05-17T21:57:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:/run/lock 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/cgroup 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/pstore 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/bpf 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/hugepages 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/dev/mqueue 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/debug 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/tracing 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/proc/fs/nfsd 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/fs/fuse/connections 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/sys/kernel/config 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/run/rpc_pipefs 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/run/user/1000 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph 2024-05-17T21:57:56.673 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-4 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-5 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-6 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/var/lib/ceph/osd/ceph-7 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T21:57:56.674 INFO:teuthology.orchestra.run.smithi155.stdout:/home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:56.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:56.675 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2024-05-17T21:57:56.723 INFO:teuthology.orchestra.run.smithi155.stdout:45 2024-05-17T21:57:56.723 INFO:teuthology.orchestra.run.smithi155.stdout:46 2024-05-17T21:57:56.723 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-05-17T21:57:56.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:57:56.723 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2024-05-17T21:57:56.723 DEBUG:teuthology.orchestra.run.smithi155:> import re 2024-05-17T21:57:56.723 DEBUG:teuthology.orchestra.run.smithi155:> import os 2024-05-17T21:57:56.723 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:57:56.724 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> return f 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.3")) 2024-05-17T21:57:56.725 DEBUG:teuthology.orchestra.run.smithi155:> ' 2024-05-17T21:57:56.848 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.3.33583.asok 2024-05-17T21:57:56.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-17T21:57:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:57:56.860 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33583.asok 2024-05-17T21:57:56.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:56.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33583.asok status 2024-05-17T21:57:57.038 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "3", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "33583", 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4522, 2024-05-17T21:57:57.039 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4522 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1368464841 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2024-05-17T21:57:57.040 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1368464841 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4522 192.168.0.2:0/1368464841", 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.2:0/1368464841", 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 7, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 20, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2024-05-17T21:57:57.041 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-17T21:57:57.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.055 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T21:57:57.055 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:57.062 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T21:57:57.062 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:57.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:57:57.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.118 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T21:57:57.118 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:57.165 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T21:57:57.165 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:57.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-17T21:57:57.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.226 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2024-05-17T21:57:57.226 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:57.234 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2024-05-17T21:57:57.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:57.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.234 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-17T21:57:57.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.292 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2024-05-17T21:57:57.292 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:57.340 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2024-05-17T21:57:57.340 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:57.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:57:57.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-17T21:57:57.396 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-17T21:57:57.438 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T21:57:57.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:57:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 Processor -- start 2024-05-17T21:57:57.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 -- start start 2024-05-17T21:57:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac301461c0 0x7fac3015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fac300706e0 0x7fac301605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac301465d0 con 0x7fac3015e6b0 2024-05-17T21:57:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fac3006ef30 con 0x7fac301461c0 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac34f29640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fac3006f210 con 0x7fac300706e0 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac301461c0 0x7fac3015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac301461c0 0x7fac3015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60434/0 (socket says 172.21.15.83:60434) 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 -- 172.21.15.83:0/4283764830 learned_addr learned my addr 172.21.15.83:0/4283764830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2ed9d640 1 --2- 172.21.15.83:0/4283764830 >> v2:172.21.15.155:3300/0 conn(0x7fac300706e0 0x7fac301605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 -- 172.21.15.83:0/4283764830 >> v2:172.21.15.155:3300/0 conn(0x7fac300706e0 msgr2=0x7fac301605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 --2- 172.21.15.83:0/4283764830 >> v2:172.21.15.155:3300/0 conn(0x7fac300706e0 0x7fac301605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 -- 172.21.15.83:0/4283764830 >> v1:172.21.15.83:6789/0 conn(0x7fac3015e6b0 legacy=0x7fac3006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.608+0000 7fac2e59c640 1 -- 172.21.15.83:0/4283764830 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3006f4f0 con 0x7fac301461c0 2024-05-17T21:57:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac2e59c640 1 --2- 172.21.15.83:0/4283764830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac301461c0 0x7fac3015e170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac30072c00 tx=0x7fac2001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e770bfcd9ca6e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac2d59a640 1 -- 172.21.15.83:0/4283764830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200ab600 con 0x7fac301461c0 2024-05-17T21:57:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac2d59a640 1 -- 172.21.15.83:0/4283764830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac200b0070 con 0x7fac301461c0 2024-05-17T21:57:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac2d59a640 1 -- 172.21.15.83:0/4283764830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200ab9c0 con 0x7fac301461c0 2024-05-17T21:57:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 -- 172.21.15.83:0/4283764830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac301461c0 msgr2=0x7fac3015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 --2- 172.21.15.83:0/4283764830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac301461c0 0x7fac3015e170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac30072c00 tx=0x7fac2001c4f0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 -- 172.21.15.83:0/4283764830 shutdown_connections 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 --2- 172.21.15.83:0/4283764830 >> v2:172.21.15.155:3300/0 conn(0x7fac300706e0 0x7fac301605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 --2- 172.21.15.83:0/4283764830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac301461c0 0x7fac3015e170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 -- 172.21.15.83:0/4283764830 >> 172.21.15.83:0/4283764830 conn(0x7fac3006cd90 msgr2=0x7fac300b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 -- 172.21.15.83:0/4283764830 shutdown_connections 2024-05-17T21:57:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.612+0000 7fac34f29640 1 -- 172.21.15.83:0/4283764830 wait complete. 2024-05-17T21:57:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 Processor -- start 2024-05-17T21:57:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- start start 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac300706e0 0x7fac3013f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 0x7fac301654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac30169940 con 0x7fac3015e6b0 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac300d12b0 con 0x7fac300706e0 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac30169ac0 con 0x7fac301461c0 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60440/0 (socket says 172.21.15.83:60440) 2024-05-17T21:57:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 -- 172.21.15.83:0/3344663806 learned_addr learned my addr 172.21.15.83:0/3344663806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2e59c640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac300706e0 0x7fac3013f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2dd9b640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 0x7fac301654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 -- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 msgr2=0x7fac301654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 0x7fac301654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 -- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac300706e0 msgr2=0x7fac3013f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac300706e0 0x7fac3013f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 -- 172.21.15.83:0/3344663806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3013f620 con 0x7fac3015e6b0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2dd9b640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 0x7fac301654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac2ed9d640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac28007e60 tx=0x7fac2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28014420 con 0x7fac3015e6b0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac28019070 con 0x7fac3015e6b0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28014820 con 0x7fac3015e6b0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac30167580 con 0x7fac3015e6b0 2024-05-17T21:57:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac30163ec0 con 0x7fac3015e6b0 2024-05-17T21:57:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.616+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac301605a0 con 0x7fac3015e6b0 2024-05-17T21:57:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.620+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac280149c0 con 0x7fac3015e6b0 2024-05-17T21:57:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.620+0000 7fac1effd640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 0x7fac10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.620+0000 7fac2e59c640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 0x7fac10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.620+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac2808fb50 con 0x7fac3015e6b0 2024-05-17T21:57:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.620+0000 7fac2e59c640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 0x7fac10071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac200af000 tx=0x7fac200a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.624+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fac2805ca40 con 0x7fac3015e6b0 2024-05-17T21:57:57.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.784+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac30163930 con 0x7fac3015e6b0 2024-05-17T21:57:57.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.788+0000 7fac1effd640 1 -- 172.21.15.83:0/3344663806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7fac2805c340 con 0x7fac3015e6b0 2024-05-17T21:57:57.782 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:57:57.783 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:57:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 msgr2=0x7fac10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 0x7fac10071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fac200af000 tx=0x7fac200a7000 comp rx=0 tx=0).stop 2024-05-17T21:57:57.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 msgr2=0x7fac3015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac28007e60 tx=0x7fac2800e4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 shutdown_connections 2024-05-17T21:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac3015e6b0 0x7fac3015f280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac301461c0 0x7fac301654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac1006f400 0x7fac10071910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 --2- 172.21.15.83:0/3344663806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac300706e0 0x7fac3013f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 >> 172.21.15.83:0/3344663806 conn(0x7fac3006cd90 msgr2=0x7fac300b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 shutdown_connections 2024-05-17T21:57:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.792+0000 7fac34f29640 1 -- 172.21.15.83:0/3344663806 wait complete. 2024-05-17T21:57:57.803 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-17T21:57:28.098831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-17T21:57:28.618308+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T21:57:57.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T21:57:57.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 Processor -- start 2024-05-17T21:57:57.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 -- start start 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1dd809ba00 0x7f1dd809be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd8071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd809c350 con 0x7f1dd80b88b0 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd80d1a40 con 0x7f1dd809ba00 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddf88e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd809d040 con 0x7f1dd8071400 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1dd809ba00 0x7f1dd809be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1dd809ba00 0x7f1dd809be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60446/0 (socket says 172.21.15.83:60446) 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddd62a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1dd80b88b0 0x7f1dd80d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49090/0 (socket says 172.21.15.83:49090) 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 -- 172.21.15.83:0/1990210987 learned_addr learned my addr 172.21.15.83:0/1990210987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddde2b640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd8071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 -- 172.21.15.83:0/1990210987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 msgr2=0x7f1dd8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd8071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 -- 172.21.15.83:0/1990210987 >> v1:172.21.15.83:6789/0 conn(0x7f1dd80b88b0 legacy=0x7f1dd80d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddce29640 1 -- 172.21.15.83:0/1990210987 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd809c710 con 0x7f1dd809ba00 2024-05-17T21:57:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1dcffff640 1 -- 172.21.15.83:0/1990210987 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425686963 0 0) 0x7f1dd4001120 con 0x7f1dd80b88b0 2024-05-17T21:57:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.964+0000 7f1ddde2b640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd8071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:57:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddce29640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd809be10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009870 tx=0x7f1dc801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b181cc20bb0e9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1dcffff640 1 -- 172.21.15.83:0/1990210987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80b0070 con 0x7f1dd809ba00 2024-05-17T21:57:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1dcffff640 1 -- 172.21.15.83:0/1990210987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8004230 con 0x7f1dd809ba00 2024-05-17T21:57:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1dcffff640 1 -- 172.21.15.83:0/1990210987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8003620 con 0x7f1dd809ba00 2024-05-17T21:57:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1990210987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 msgr2=0x7f1dd809be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd809be10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009870 tx=0x7f1dc801c440 comp rx=0 tx=0).stop 2024-05-17T21:57:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1990210987 shutdown_connections 2024-05-17T21:57:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd8071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1990210987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd809be10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1990210987 >> 172.21.15.83:0/1990210987 conn(0x7f1dd80d3620 msgr2=0x7f1dd80d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1990210987 shutdown_connections 2024-05-17T21:57:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.968+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1990210987 wait complete. 2024-05-17T21:57:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 Processor -- start 2024-05-17T21:57:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- start start 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd80b88b0 0x7f1dd80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd816a870 con 0x7f1dd809ba00 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd816a6f0 con 0x7f1dd80b88b0 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd816a9f0 con 0x7f1dd8071400 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60458/0 (socket says 172.21.15.83:60458) 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 -- 172.21.15.83:0/1148040800 learned_addr learned my addr 172.21.15.83:0/1148040800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:57.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddd62a640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 -- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 msgr2=0x7f1dd814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 -- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd80b88b0 msgr2=0x7f1dd80c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddde2b640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd80b88b0 0x7f1dd80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd80b88b0 0x7f1dd80c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 -- 172.21.15.83:0/1148040800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80b4680 con 0x7f1dd809ba00 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddce29640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801a070 tx=0x7f1dc80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80b9020 con 0x7f1dd809ba00 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc80b1040 con 0x7f1dd809ba00 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd80b4960 con 0x7f1dd809ba00 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80b0050 con 0x7f1dd809ba00 2024-05-17T21:57:57.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.972+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd8158890 con 0x7f1dd809ba00 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd8071810 con 0x7f1dd809ba00 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1dc80bc020 con 0x7f1dd809ba00 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1dcdffb640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 0x7f1dbc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1dc8128ef0 con 0x7f1dd809ba00 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1ddd62a640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 0x7f1dbc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:57.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.976+0000 7f1ddd62a640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 0x7f1dbc071910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000f70 tx=0x7f1dd4007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:57.980+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1dc80c3030 con 0x7f1dd809ba00 2024-05-17T21:57:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1dd80b1210 con 0x7f1dd809ba00 2024-05-17T21:57:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1dcdffb640 1 -- 172.21.15.83:0/1148040800 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1dc8016150 con 0x7f1dd809ba00 2024-05-17T21:57:58.137 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T21:57:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 msgr2=0x7f1dbc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 0x7f1dbc071910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000f70 tx=0x7f1dd4007c30 comp rx=0 tx=0).stop 2024-05-17T21:57:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 msgr2=0x7f1dd80c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.144+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1dc801a070 tx=0x7f1dc80191e0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 shutdown_connections 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1dbc06f400 0x7f1dbc071910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd80b88b0 0x7f1dd80c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd809ba00 0x7f1dd80c7940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 --2- 172.21.15.83:0/1148040800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1dd8071400 0x7f1dd814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 >> 172.21.15.83:0/1148040800 conn(0x7f1dd80d3620 msgr2=0x7f1dd80d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 shutdown_connections 2024-05-17T21:57:58.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.148+0000 7f1ddf88e640 1 -- 172.21.15.83:0/1148040800 wait complete. 2024-05-17T21:57:58.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-17T21:57:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 Processor -- start 2024-05-17T21:57:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 -- start start 2024-05-17T21:57:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c440cf830 0x7f9c440b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9c4415d700 0x7f9c441663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c440b5640 con 0x7f9c440cdf40 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9c44070870 con 0x7f9c440cf830 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c4a931640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9c440b58e0 con 0x7f9c4415d700 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c43fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c440cf830 0x7f9c440b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c43fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c440cf830 0x7f9c440b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60468/0 (socket says 172.21.15.83:60468) 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.320+0000 7f9c43fff640 1 -- 172.21.15.83:0/1131788824 learned_addr learned my addr 172.21.15.83:0/1131788824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c48ece640 1 --2- 172.21.15.83:0/1131788824 >> v2:172.21.15.155:3300/0 conn(0x7f9c4415d700 0x7f9c441663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c43fff640 1 -- 172.21.15.83:0/1131788824 >> v2:172.21.15.155:3300/0 conn(0x7f9c4415d700 msgr2=0x7f9c441663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c43fff640 1 --2- 172.21.15.83:0/1131788824 >> v2:172.21.15.155:3300/0 conn(0x7f9c4415d700 0x7f9c441663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c43fff640 1 -- 172.21.15.83:0/1131788824 >> v1:172.21.15.155:6789/0 conn(0x7f9c440cdf40 legacy=0x7f9c4415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c43fff640 1 -- 172.21.15.83:0/1131788824 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4406f060 con 0x7f9c440cf830 2024-05-17T21:57:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c43fff640 1 --2- 172.21.15.83:0/1131788824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c440cf830 0x7f9c440b3aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c4406d6e0 tx=0x7f9c3401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea98d62e05d4b31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c42ffd640 1 -- 172.21.15.83:0/1131788824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c340b1070 con 0x7f9c440cf830 2024-05-17T21:57:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c42ffd640 1 -- 172.21.15.83:0/1131788824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c340ac640 con 0x7f9c440cf830 2024-05-17T21:57:58.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c42ffd640 1 -- 172.21.15.83:0/1131788824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c340aca10 con 0x7f9c440cf830 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 -- 172.21.15.83:0/1131788824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c440cf830 msgr2=0x7f9c440b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 --2- 172.21.15.83:0/1131788824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c440cf830 0x7f9c440b3aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9c4406d6e0 tx=0x7f9c3401c500 comp rx=0 tx=0).stop 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 -- 172.21.15.83:0/1131788824 shutdown_connections 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 --2- 172.21.15.83:0/1131788824 >> v2:172.21.15.155:3300/0 conn(0x7f9c4415d700 0x7f9c441663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 --2- 172.21.15.83:0/1131788824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c440cf830 0x7f9c440b3aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 -- 172.21.15.83:0/1131788824 >> 172.21.15.83:0/1131788824 conn(0x7f9c4406cd90 msgr2=0x7f9c440b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 -- 172.21.15.83:0/1131788824 shutdown_connections 2024-05-17T21:57:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.324+0000 7f9c4a931640 1 -- 172.21.15.83:0/1131788824 wait complete. 2024-05-17T21:57:58.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 Processor -- start 2024-05-17T21:57:58.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- start start 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c440cdf40 0x7f9c441493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c440cf830 0x7f9c44159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c441692b0 con 0x7f9c4415d700 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44169130 con 0x7f9c440cdf40 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c44169430 con 0x7f9c440cf830 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60470/0 (socket says 172.21.15.83:60470) 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 -- 172.21.15.83:0/413151313 learned_addr learned my addr 172.21.15.83:0/413151313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c43fff640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c440cdf40 0x7f9c441493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c437fe640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c440cf830 0x7f9c44159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 -- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c440cf830 msgr2=0x7f9c44159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c440cf830 0x7f9c44159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 -- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c440cdf40 msgr2=0x7f9c441493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c440cdf40 0x7f9c441493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 -- 172.21.15.83:0/413151313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441532e0 con 0x7f9c4415d700 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c48ece640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007e60 tx=0x7f9c3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c014420 con 0x7f9c4415d700 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c019070 con 0x7f9c4415d700 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c014820 con 0x7f9c4415d700 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441535c0 con 0x7f9c4415d700 2024-05-17T21:57:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.328+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c44153b80 con 0x7f9c4415d700 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c440b3aa0 con 0x7f9c4415d700 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c3c0149c0 con 0x7f9c4415d700 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c40ff9640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 0x7f9c30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c43fff640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 0x7f9c30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9c3c08fac0 con 0x7f9c4415d700 2024-05-17T21:57:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.332+0000 7f9c43fff640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 0x7f9c30071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c340ae000 tx=0x7f9c34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.336+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9c3c05ca40 con 0x7f9c4415d700 2024-05-17T21:57:58.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c440b67f0 con 0x7f9c4415d700 2024-05-17T21:57:58.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c40ff9640 1 -- 172.21.15.83:0/413151313 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9c3c05c340 con 0x7f9c4415d700 2024-05-17T21:57:58.477 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-05-17T21:57:58.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 msgr2=0x7f9c30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 0x7f9c30071910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c340ae000 tx=0x7f9c34008040 comp rx=0 tx=0).stop 2024-05-17T21:57:58.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 msgr2=0x7f9c440d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.484+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007e60 tx=0x7f9c3c00e4b0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 shutdown_connections 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c4415d700 0x7f9c440d0040 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c440cf830 0x7f9c44159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c3006f400 0x7f9c30071910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 --2- 172.21.15.83:0/413151313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c440cdf40 0x7f9c441493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 >> 172.21.15.83:0/413151313 conn(0x7f9c4406cd90 msgr2=0x7f9c4415a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 shutdown_connections 2024-05-17T21:57:58.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.488+0000 7f9c4a931640 1 -- 172.21.15.83:0/413151313 wait complete. 2024-05-17T21:57:58.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-17T21:57:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 Processor -- start 2024-05-17T21:57:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 -- start start 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 0x7f8b880d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8b88143f20 0x7f8b88072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b880737d0 con 0x7f8b880733c0 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8b880b4110 con 0x7f8b88143f20 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8fe8f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b88070770 con 0x7f8b880b3d00 2024-05-17T21:57:58.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8e42c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8b880733c0 0x7f8b8806eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49100/0 (socket says 172.21.15.83:49100) 2024-05-17T21:57:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.656+0000 7f8b8e42c640 1 -- 172.21.15.83:0/3389870343 learned_addr learned my addr 172.21.15.83:0/3389870343 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 0x7f8b880d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8d42a640 1 --2- 172.21.15.83:0/3389870343 >> v2:172.21.15.155:3300/0 conn(0x7f8b88143f20 0x7f8b88072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013169473 0 0) 0x7f8b880737d0 con 0x7f8b880733c0 2024-05-17T21:57:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b74003530 con 0x7f8b880733c0 2024-05-17T21:57:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 506851783 0 0) 0x7f8b74003530 con 0x7f8b880733c0 2024-05-17T21:57:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 msgr2=0x7f8b880d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 0x7f8b880d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 >> v2:172.21.15.155:3300/0 conn(0x7f8b88143f20 msgr2=0x7f8b88072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 --2- 172.21.15.83:0/3389870343 >> v2:172.21.15.155:3300/0 conn(0x7f8b88143f20 0x7f8b88072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b880d10d0 con 0x7f8b880733c0 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f8b80001a80 con 0x7f8b880733c0 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 >> v1:172.21.15.83:6789/0 conn(0x7f8b880733c0 legacy=0x7f8b8806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b74003a60 0x7f8b74003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74004810 0x7f8b74004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 0x7f8b74005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b880b4110 con 0x7f8b74003a60 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88070770 con 0x7f8b74004810 2024-05-17T21:57:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7400bab0 con 0x7f8b880b3d00 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b74003a60 0x7f8b74003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8d42a640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74004810 0x7f8b74004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 -- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 msgr2=0x7f8b74005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b880b3d00 0x7f8b74005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 -- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74004810 msgr2=0x7f8b74004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b74004810 0x7f8b74004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 -- 172.21.15.83:0/3389870343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b74003530 con 0x7f8b74003a60 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b74003a60 0x7f8b74003ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009ae0 tx=0x7f8b7801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67be6d2eeeec4f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780b2070 con 0x7f8b74003a60 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b780a5590 con 0x7f8b74003a60 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.660+0000 7f8b8cc29640 1 -- 172.21.15.83:0/3389870343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780096b0 con 0x7f8b74003a60 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/3389870343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b74003a60 msgr2=0x7f8b74003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/3389870343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b74003a60 0x7f8b74003ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009ae0 tx=0x7f8b7801cab0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 -- 172.21.15.83:0/3389870343 reap_dead start 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/3389870343 shutdown_connections 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/3389870343 >> 172.21.15.83:0/3389870343 conn(0x7f8b8806cd90 msgr2=0x7f8b880b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/3389870343 shutdown_connections 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/3389870343 wait complete. 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 Processor -- start 2024-05-17T21:57:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- start start 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b74003a60 0x7f8b881fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b881fa7f0 0x7f8b881339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88144330 con 0x7f8b88133f10 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88147600 con 0x7f8b881fa7f0 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8fe8f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b88070c00 con 0x7f8b74003a60 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60494/0 (socket says 172.21.15.83:60494) 2024-05-17T21:57:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 -- 172.21.15.83:0/977291532 learned_addr learned my addr 172.21.15.83:0/977291532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b74003a60 0x7f8b881fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8d42a640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b881fa7f0 0x7f8b881339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 -- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b74003a60 msgr2=0x7f8b881fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b74003a60 0x7f8b881fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 -- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b881fa7f0 msgr2=0x7f8b881339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b881fa7f0 0x7f8b881339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.664+0000 7f8b8e42c640 1 -- 172.21.15.83:0/977291532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b881fac00 con 0x7f8b88133f10 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8e42c640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b88070720 tx=0x7f8b8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000a640 con 0x7f8b88133f10 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8000a7e0 con 0x7f8b88133f10 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b88134820 con 0x7f8b88133f10 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000abd0 con 0x7f8b88133f10 2024-05-17T21:57:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b88134de0 con 0x7f8b88133f10 2024-05-17T21:57:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b88072e80 con 0x7f8b88133f10 2024-05-17T21:57:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b80007070 con 0x7f8b88133f10 2024-05-17T21:57:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 0x7f8b6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8b8008f040 con 0x7f8b88133f10 2024-05-17T21:57:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 0x7f8b6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:58.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.668+0000 7f8b8dc2b640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 0x7f8b6c071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b780af000 tx=0x7f8b780046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.672+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8b80022080 con 0x7f8b88133f10 2024-05-17T21:57:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.832+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8b88145060 con 0x7f8b88133f10 2024-05-17T21:57:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b7e7fc640 1 -- 172.21.15.83:0/977291532 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8b881fac00 con 0x7f8b88133f10 2024-05-17T21:57:58.828 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 msgr2=0x7f8b6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 0x7f8b6c071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b780af000 tx=0x7f8b780046d0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 msgr2=0x7f8b881324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b88070720 tx=0x7f8b8000ccb0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 shutdown_connections 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b88133f10 0x7f8b881324c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b6c06f400 0x7f8b6c071910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b881fa7f0 0x7f8b881339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 --2- 172.21.15.83:0/977291532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b74003a60 0x7f8b881fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 >> 172.21.15.83:0/977291532 conn(0x7f8b8806cd90 msgr2=0x7f8b881446e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 shutdown_connections 2024-05-17T21:57:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:58.836+0000 7f8b8fe8f640 1 -- 172.21.15.83:0/977291532 wait complete. 2024-05-17T21:57:58.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-17T21:57:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 Processor -- start 2024-05-17T21:57:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 -- start start 2024-05-17T21:57:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f04ac0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04ac0af9c0 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04ac0afca0 con 0x7f04ac06c790 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04b1288640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac0aff80 con 0x7f04ac0af3e0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04ab7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04ac0b0fa0 0x7f04ac161460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49104/0 (socket says 172.21.15.83:49104) 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.008+0000 7f04ab7fe640 1 -- 172.21.15.83:0/228297742 learned_addr learned my addr 172.21.15.83:0/228297742 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f04ac0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709097044 0 0) 0x7f04ac0af9c0 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0498003530 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2477840205 0 0) 0x7f0498003530 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 msgr2=0x7f04ac0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f04ac0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 >> v1:172.21.15.155:6789/0 conn(0x7f04ac06c790 legacy=0x7f04ac156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac0b0260 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f04a4001700 con 0x7f04ac0b0fa0 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 >> v1:172.21.15.83:6789/0 conn(0x7f04ac0b0fa0 legacy=0x7f04ac161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f0498003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498004030 0x7f0498004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498004d50 0x7f0498005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac0aff80 con 0x7f0498004030 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f049800bc10 con 0x7f0498004d50 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f049800be10 con 0x7f04ac0af3e0 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498004030 0x7f0498004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aaffd640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f0498003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04ab7fe640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498004d50 0x7f0498005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 -- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 msgr2=0x7f0498003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04ac0af3e0 0x7f0498003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 -- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498004d50 msgr2=0x7f0498005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498004d50 0x7f0498005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 -- 172.21.15.83:0/228297742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498003530 con 0x7f0498004030 2024-05-17T21:57:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498004030 0x7f0498004810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04ac073a40 tx=0x7f049401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ded28ee4297760 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940b2070 con 0x7f0498004030 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04940a5510 con 0x7f0498004030 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.012+0000 7f04a9ffb640 1 -- 172.21.15.83:0/228297742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940096b0 con 0x7f0498004030 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- 172.21.15.83:0/228297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498004030 msgr2=0x7f0498004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 --2- 172.21.15.83:0/228297742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498004030 0x7f0498004810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04ac073a40 tx=0x7f049401ca30 comp rx=0 tx=0).stop 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04ab7fe640 1 -- 172.21.15.83:0/228297742 reap_dead start 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- 172.21.15.83:0/228297742 shutdown_connections 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- 172.21.15.83:0/228297742 >> 172.21.15.83:0/228297742 conn(0x7f04ac09b8b0 msgr2=0x7f04ac0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- 172.21.15.83:0/228297742 shutdown_connections 2024-05-17T21:57:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- 172.21.15.83:0/228297742 wait complete. 2024-05-17T21:57:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 Processor -- start 2024-05-17T21:57:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- start start 2024-05-17T21:57:59.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 0x7f04ac142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04ac155d70 0x7f04ac157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac161b70 con 0x7f04ac145c00 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac09be90 con 0x7f04ac155d70 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04b1288640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac161cf0 con 0x7f0498004030 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60508/0 (socket says 172.21.15.83:60508) 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 -- 172.21.15.83:0/3259603196 learned_addr learned my addr 172.21.15.83:0/3259603196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aaffd640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 0x7f04ac142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04ab7fe640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04ac155d70 0x7f04ac157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 -- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 msgr2=0x7f04ac142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 0x7f04ac142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 -- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04ac155d70 msgr2=0x7f04ac157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:57:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04ac155d70 0x7f04ac157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 -- 172.21.15.83:0/3259603196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac070240 con 0x7f04ac145c00 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aaffd640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 0x7f04ac142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.016+0000 7f04aa7fc640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f049401c860 tx=0x7f0498005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940b2040 con 0x7f04ac145c00 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049400f000 con 0x7f04ac145c00 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac144d90 con 0x7f04ac145c00 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940096b0 con 0x7f04ac145c00 2024-05-17T21:57:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac143a90 con 0x7f04ac145c00 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac142c80 con 0x7f04ac145c00 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04940c0030 con 0x7f04ac145c00 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 0x7f048c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04aaffd640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 0x7f048c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04941290a0 con 0x7f04ac145c00 2024-05-17T21:57:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.020+0000 7f04aaffd640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 0x7f048c071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f049c001e10 tx=0x7f049c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:57:59.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.024+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f04ac142c80 con 0x7f04ac145c00 2024-05-17T21:57:59.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:57:59.232+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f04ac143540 con 0x7f04ac145c00 2024-05-17T21:57:59.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:57:59.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:58:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.036+0000 7f04a37fe640 1 -- 172.21.15.83:0/3259603196 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v48) ==== 145+0+0 (secure 0 0 0) 0x7f04940c7060 con 0x7f04ac145c00 2024-05-17T21:58:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 msgr2=0x7f048c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 0x7f048c071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f049c001e10 tx=0x7f049c009040 comp rx=0 tx=0).stop 2024-05-17T21:58:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 msgr2=0x7f04ac144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f049401c860 tx=0x7f0498005840 comp rx=0 tx=0).stop 2024-05-17T21:58:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 shutdown_connections 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f048c06f400 0x7f048c071910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04ac155d70 0x7f04ac157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04ac145c00 0x7f04ac144850 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 --2- 172.21.15.83:0/3259603196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0498004030 0x7f04ac142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 >> 172.21.15.83:0/3259603196 conn(0x7f04ac09b8b0 msgr2=0x7f04ac071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 shutdown_connections 2024-05-17T21:58:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.040+0000 7f04b1288640 1 -- 172.21.15.83:0/3259603196 wait complete. 2024-05-17T21:58:00.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T21:58:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 Processor -- start 2024-05-17T21:58:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 -- start start 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549015ea50 0x7f54900b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 0x7f549006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f549006ccd0 0x7f549015f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f549015bae0 con 0x7f549006ccd0 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f549006d0e0 con 0x7f54900b4e50 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5498147640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f54900af470 con 0x7f549015ea50 2024-05-17T21:58:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f549006ccd0 0x7f549015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f549006ccd0 0x7f549015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60522/0 (socket says 172.21.15.83:60522) 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 -- 172.21.15.83:0/2660908819 learned_addr learned my addr 172.21.15.83:0/2660908819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f5495ee3640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549015ea50 0x7f54900b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54956e2640 1 --2- 172.21.15.83:0/2660908819 >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 0x7f549006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 -- 172.21.15.83:0/2660908819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549015ea50 msgr2=0x7f54900b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549015ea50 0x7f54900b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 -- 172.21.15.83:0/2660908819 >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 msgr2=0x7f549006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 --2- 172.21.15.83:0/2660908819 >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 0x7f549006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54966e4640 1 -- 172.21.15.83:0/2660908819 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549015e290 con 0x7f549006ccd0 2024-05-17T21:58:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.232+0000 7f54956e2640 1 --2- 172.21.15.83:0/2660908819 >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 0x7f549006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f54966e4640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549006ccd0 0x7f549015f240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f54900696c0 tx=0x7f548c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86e22190b4c66c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5494ee1640 1 -- 172.21.15.83:0/2660908819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0b0070 con 0x7f549006ccd0 2024-05-17T21:58:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5494ee1640 1 -- 172.21.15.83:0/2660908819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c0ab600 con 0x7f549006ccd0 2024-05-17T21:58:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5494ee1640 1 -- 172.21.15.83:0/2660908819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0ab9d0 con 0x7f549006ccd0 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 -- 172.21.15.83:0/2660908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549006ccd0 msgr2=0x7f549015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549006ccd0 0x7f549015f240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f54900696c0 tx=0x7f548c01c490 comp rx=0 tx=0).stop 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 -- 172.21.15.83:0/2660908819 shutdown_connections 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549006ccd0 0x7f549015f240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 --2- 172.21.15.83:0/2660908819 >> v2:172.21.15.155:3300/0 conn(0x7f54900b4e50 0x7f549006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 --2- 172.21.15.83:0/2660908819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549015ea50 0x7f54900b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 -- 172.21.15.83:0/2660908819 >> 172.21.15.83:0/2660908819 conn(0x7f549009b8b0 msgr2=0x7f54900b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 -- 172.21.15.83:0/2660908819 shutdown_connections 2024-05-17T21:58:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.236+0000 7f5498147640 1 -- 172.21.15.83:0/2660908819 wait complete. 2024-05-17T21:58:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 Processor -- start 2024-05-17T21:58:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- start start 2024-05-17T21:58:00.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549006ccd0 0x7f54900b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549015ea50 0x7f549014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f549015ee60 con 0x7f54900b4e50 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f54900b3f10 con 0x7f549015ea50 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f549015efe0 con 0x7f549006ccd0 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60524/0 (socket says 172.21.15.83:60524) 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 -- 172.21.15.83:0/2101244940 learned_addr learned my addr 172.21.15.83:0/2101244940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54966e4640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549015ea50 0x7f549014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5495ee3640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549006ccd0 0x7f54900b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 -- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549006ccd0 msgr2=0x7f54900b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549006ccd0 0x7f54900b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 -- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549015ea50 msgr2=0x7f549014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549015ea50 0x7f549014e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 -- 172.21.15.83:0/2101244940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549014f5e0 con 0x7f54900b4e50 2024-05-17T21:58:00.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54956e2640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f548000d560 tx=0x7f548000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480018020 con 0x7f54900b4e50 2024-05-17T21:58:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5480014620 con 0x7f54900b4e50 2024-05-17T21:58:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54900b74e0 con 0x7f54900b4e50 2024-05-17T21:58:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800149f0 con 0x7f54900b4e50 2024-05-17T21:58:00.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.240+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54900b58e0 con 0x7f54900b4e50 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5490160d10 con 0x7f54900b4e50 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f548001b020 con 0x7f54900b4e50 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f54867fc640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 0x7f5474071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f5495ee3640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 0x7f5474071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f548008f690 con 0x7f54900b4e50 2024-05-17T21:58:00.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.244+0000 7f5495ee3640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 0x7f5474071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54880073f0 tx=0x7f548800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.248+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5480022020 con 0x7f54900b4e50 2024-05-17T21:58:00.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.412+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f54901685b0 con 0x7f54900b4e50 2024-05-17T21:58:00.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.416+0000 7f54867fc640 1 -- 172.21.15.83:0/2101244940 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f549014f5e0 con 0x7f54900b4e50 2024-05-17T21:58:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 msgr2=0x7f5474071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 0x7f5474071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f54880073f0 tx=0x7f548800b040 comp rx=0 tx=0).stop 2024-05-17T21:58:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 msgr2=0x7f5490166d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f548000d560 tx=0x7f548000da20 comp rx=0 tx=0).stop 2024-05-17T21:58:00.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 shutdown_connections 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f547406f400 0x7f5474071910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549015ea50 0x7f549014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54900b4e50 0x7f5490166d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 --2- 172.21.15.83:0/2101244940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f549006ccd0 0x7f54900b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 >> 172.21.15.83:0/2101244940 conn(0x7f549009b8b0 msgr2=0x7f549015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 shutdown_connections 2024-05-17T21:58:00.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.420+0000 7f5498147640 1 -- 172.21.15.83:0/2101244940 wait complete. 2024-05-17T21:58:00.431 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T21:58:00.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:00.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:00.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T21:58:00.508 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:00.508 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:00.515 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:00.516 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:00.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:00.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 Processor -- start 2024-05-17T21:58:00.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 -- start start 2024-05-17T21:58:00.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7effc40b8a80 0x7effc409cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effc40b69b0 0x7effc40b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc40cf530 con 0x7effc409b990 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7effc40b6520 con 0x7effc40b69b0 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc9298640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7effc409d150 con 0x7effc40b8a80 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effc40b69b0 0x7effc40b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effc40b69b0 0x7effc40b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60538/0 (socket says 172.21.15.83:60538) 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 -- 172.21.15.83:0/1935730910 learned_addr learned my addr 172.21.15.83:0/1935730910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc2ffd640 1 --2- 172.21.15.83:0/1935730910 >> v2:172.21.15.155:3300/0 conn(0x7effc40b8a80 0x7effc409cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 -- 172.21.15.83:0/1935730910 >> v2:172.21.15.155:3300/0 conn(0x7effc40b8a80 msgr2=0x7effc409cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 --2- 172.21.15.83:0/1935730910 >> v2:172.21.15.155:3300/0 conn(0x7effc40b8a80 0x7effc409cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 -- 172.21.15.83:0/1935730910 >> v1:172.21.15.155:6789/0 conn(0x7effc409b990 legacy=0x7effc409bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:00.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.760+0000 7effc37fe640 1 -- 172.21.15.83:0/1935730910 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc40cf810 con 0x7effc40b69b0 2024-05-17T21:58:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc37fe640 1 --2- 172.21.15.83:0/1935730910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc40b69b0 0x7effc40b6dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7effc4074070 tx=0x7effbc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3804f07ab87f3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc1ffb640 1 -- 172.21.15.83:0/1935730910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0a5760 con 0x7effc40b69b0 2024-05-17T21:58:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc1ffb640 1 -- 172.21.15.83:0/1935730910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc0a5900 con 0x7effc40b69b0 2024-05-17T21:58:00.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc1ffb640 1 -- 172.21.15.83:0/1935730910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0a5cd0 con 0x7effc40b69b0 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 -- 172.21.15.83:0/1935730910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc40b69b0 msgr2=0x7effc40b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 --2- 172.21.15.83:0/1935730910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc40b69b0 0x7effc40b6dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7effc4074070 tx=0x7effbc01c9f0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 -- 172.21.15.83:0/1935730910 shutdown_connections 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 --2- 172.21.15.83:0/1935730910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc40b69b0 0x7effc40b6dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 --2- 172.21.15.83:0/1935730910 >> v2:172.21.15.155:3300/0 conn(0x7effc40b8a80 0x7effc409cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 -- 172.21.15.83:0/1935730910 >> 172.21.15.83:0/1935730910 conn(0x7effc40d36d0 msgr2=0x7effc40d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 -- 172.21.15.83:0/1935730910 shutdown_connections 2024-05-17T21:58:00.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.764+0000 7effc9298640 1 -- 172.21.15.83:0/1935730910 wait complete. 2024-05-17T21:58:00.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 Processor -- start 2024-05-17T21:58:00.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- start start 2024-05-17T21:58:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 0x7effc415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 0x7effc415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7effc416a7a0 con 0x7effc409b990 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effc40cf2d0 con 0x7effc40b69b0 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effc416a920 con 0x7effc40b8a80 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60540/0 (socket says 172.21.15.83:60540) 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 -- 172.21.15.83:0/3482738143 learned_addr learned my addr 172.21.15.83:0/3482738143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc37fe640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 0x7effc415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc27fc640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 0x7effc415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 -- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 msgr2=0x7effc415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 0x7effc415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 -- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 msgr2=0x7effc415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 0x7effc415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 -- 172.21.15.83:0/3482738143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc414bfe0 con 0x7effc409b990 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc37fe640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 0x7effc415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc27fc640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 0x7effc415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc2ffd640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7effb400d620 tx=0x7effb400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb400b620 con 0x7effc409b990 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb4019070 con 0x7effc409b990 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc415adc0 con 0x7effc409b990 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb400b9e0 con 0x7effc409b990 2024-05-17T21:58:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc415b380 con 0x7effc409b990 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.768+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc40b6dc0 con 0x7effc409b990 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.772+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7effb400bb80 con 0x7effc409b990 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.772+0000 7effab7fe640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 0x7effb0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.772+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7effb408f7b0 con 0x7effc409b990 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.772+0000 7effc27fc640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 0x7effb0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.772+0000 7effc27fc640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 0x7effb0071910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7effc415f4d0 tx=0x7effb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.776+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7effb405c620 con 0x7effc409b990 2024-05-17T21:58:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.952+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7effc40d1ed0 con 0x7effc409b990 2024-05-17T21:58:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effab7fe640 1 -- 172.21.15.83:0/3482738143 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7effb405bf20 con 0x7effc409b990 2024-05-17T21:58:00.947 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:00.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 msgr2=0x7effb0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 0x7effb0071910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7effc415f4d0 tx=0x7effb8008040 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 msgr2=0x7effc414baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7effb400d620 tx=0x7effb400dae0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 shutdown_connections 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effc40b8a80 0x7effc415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effb006f400 0x7effb0071910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effc40b69b0 0x7effc415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 --2- 172.21.15.83:0/3482738143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effc409b990 0x7effc414baa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 >> 172.21.15.83:0/3482738143 conn(0x7effc40d36d0 msgr2=0x7effc40711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 shutdown_connections 2024-05-17T21:58:00.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:00.956+0000 7effc9298640 1 -- 172.21.15.83:0/3482738143 wait complete. 2024-05-17T21:58:00.969 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:00.969 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:00.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:01.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 Processor -- start 2024-05-17T21:58:01.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 -- start start 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4874073590 0x7f48740d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 0x7f48741447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48740d10d0 con 0x7f48740b3d00 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f48740b4110 con 0x7f4874073590 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4879859640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4874070770 con 0x7f4874143f20 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4872ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f48740b3d00 0x7f4874072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49106/0 (socket says 172.21.15.83:49106) 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4872ffd640 1 -- 172.21.15.83:0/1695588345 learned_addr learned my addr 172.21.15.83:0/1695588345 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48727fc640 1 --2- 172.21.15.83:0/1695588345 >> v2:172.21.15.83:3300/0 conn(0x7f4874073590 0x7f48740d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48737fe640 1 --2- 172.21.15.83:0/1695588345 >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 0x7f48741447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48727fc640 1 -- 172.21.15.83:0/1695588345 >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 msgr2=0x7f48741447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:01.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f4871ffb640 1 -- 172.21.15.83:0/1695588345 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776622827 0 0) 0x7f48740d10d0 con 0x7f48740b3d00 2024-05-17T21:58:01.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48727fc640 1 --2- 172.21.15.83:0/1695588345 >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 0x7f48741447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48727fc640 1 -- 172.21.15.83:0/1695588345 >> v1:172.21.15.83:6789/0 conn(0x7f48740b3d00 legacy=0x7f4874072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48727fc640 1 -- 172.21.15.83:0/1695588345 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4874070120 con 0x7f4874073590 2024-05-17T21:58:01.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.180+0000 7f48737fe640 1 --2- 172.21.15.83:0/1695588345 >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 0x7f48741447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:01.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f48727fc640 1 --2- 172.21.15.83:0/1695588345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874073590 0x7f48740d0a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48740706e0 tx=0x7f486801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef85076e4f57f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4871ffb640 1 -- 172.21.15.83:0/1695588345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680b1070 con 0x7f4874073590 2024-05-17T21:58:01.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4871ffb640 1 -- 172.21.15.83:0/1695588345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48680ac640 con 0x7f4874073590 2024-05-17T21:58:01.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4871ffb640 1 -- 172.21.15.83:0/1695588345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680aca10 con 0x7f4874073590 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 -- 172.21.15.83:0/1695588345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874073590 msgr2=0x7f48740d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 --2- 172.21.15.83:0/1695588345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874073590 0x7f48740d0a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48740706e0 tx=0x7f486801c480 comp rx=0 tx=0).stop 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 -- 172.21.15.83:0/1695588345 shutdown_connections 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 --2- 172.21.15.83:0/1695588345 >> v2:172.21.15.155:3300/0 conn(0x7f4874143f20 0x7f48741447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 --2- 172.21.15.83:0/1695588345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874073590 0x7f48740d0a10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 -- 172.21.15.83:0/1695588345 >> 172.21.15.83:0/1695588345 conn(0x7f487406cd90 msgr2=0x7f48740b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 -- 172.21.15.83:0/1695588345 shutdown_connections 2024-05-17T21:58:01.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.184+0000 7f4879859640 1 -- 172.21.15.83:0/1695588345 wait complete. 2024-05-17T21:58:01.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 Processor -- start 2024-05-17T21:58:01.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- start start 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4874073590 0x7f487415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48740b3d00 0x7f4874142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4874070c00 con 0x7f4874143f20 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4874145540 con 0x7f4874073590 2024-05-17T21:58:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f48740af3b0 con 0x7f48740b3d00 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60554/0 (socket says 172.21.15.83:60554) 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 -- 172.21.15.83:0/3006022550 learned_addr learned my addr 172.21.15.83:0/3006022550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4872ffd640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4874073590 0x7f487415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48727fc640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48740b3d00 0x7f4874142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 -- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48740b3d00 msgr2=0x7f4874142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48740b3d00 0x7f4874142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 -- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4874073590 msgr2=0x7f487415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4874073590 0x7f487415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 -- 172.21.15.83:0/3006022550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487415db30 con 0x7f4874143f20 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f48737fe640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7f0 tx=0x7f486c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c015020 con 0x7f4874143f20 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c004250 con 0x7f4874143f20 2024-05-17T21:58:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48740aa440 con 0x7f4874143f20 2024-05-17T21:58:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c003620 con 0x7f4874143f20 2024-05-17T21:58:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.188+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48740aaa00 con 0x7f4874143f20 2024-05-17T21:58:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48741447f0 con 0x7f4874143f20 2024-05-17T21:58:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f486c018020 con 0x7f4874143f20 2024-05-17T21:58:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f485bfff640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 0x7f4860071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f4872ffd640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 0x7f4860071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f486c08f030 con 0x7f4874143f20 2024-05-17T21:58:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.192+0000 7f4872ffd640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 0x7f4860071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48640012f0 tx=0x7f4864005ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.196+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f486c01f030 con 0x7f4874143f20 2024-05-17T21:58:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.368+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4874168420 con 0x7f4874143f20 2024-05-17T21:58:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f485bfff640 1 -- 172.21.15.83:0/3006022550 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f486c05bfb0 con 0x7f4874143f20 2024-05-17T21:58:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 msgr2=0x7f4860071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 0x7f4860071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48640012f0 tx=0x7f4864005ec0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 msgr2=0x7f487413bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7f0 tx=0x7f486c00ccb0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 shutdown_connections 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4874143f20 0x7f487413bef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48740b3d00 0x7f4874142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f486006f400 0x7f4860071910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 --2- 172.21.15.83:0/3006022550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4874073590 0x7f487415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 >> 172.21.15.83:0/3006022550 conn(0x7f487406cd90 msgr2=0x7f48740d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 shutdown_connections 2024-05-17T21:58:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.372+0000 7f4879859640 1 -- 172.21.15.83:0/3006022550 wait complete. 2024-05-17T21:58:01.386 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T21:58:01.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:01.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:01.635 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:01.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:58:01.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:01.635 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T21:58:01.646 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:01.646 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:01.653 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:01.653 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:01.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.864+0000 7f4e2b220640 1 Processor -- start 2024-05-17T21:58:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.864+0000 7f4e2b220640 1 -- start start 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e2b220640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 0x7f4e240d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e2b220640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e240d10d0 con 0x7f4e240b3d00 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e2b220640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e24070120 con 0x7f4e24143f20 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e2b220640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e240b4110 con 0x7f4e24073590 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e28fbc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4e240b3d00 0x7f4e24072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49108/0 (socket says 172.21.15.83:49108) 2024-05-17T21:58:01.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e28fbc640 1 -- 172.21.15.83:0/4271764071 learned_addr learned my addr 172.21.15.83:0/4271764071 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 0x7f4e240d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677899651 0 0) 0x7f4e240d10d0 con 0x7f4e240b3d00 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e0c003530 con 0x7f4e240b3d00 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844425409 0 0) 0x7f4e0c003530 con 0x7f4e240b3d00 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 msgr2=0x7f4e240d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 0x7f4e240d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 >> v1:172.21.15.155:6789/0 conn(0x7f4e24143f20 legacy=0x7f4e241447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e24070400 con 0x7f4e240b3d00 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f4e20001700 con 0x7f4e240b3d00 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 >> v1:172.21.15.83:6789/0 conn(0x7f4e240b3d00 legacy=0x7f4e24072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c003a40 0x7f4e0c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e0c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 0x7f4e0c0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e240b4110 con 0x7f4e0c004810 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00bbc0 con 0x7f4e0c003a40 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c00bdc0 con 0x7f4e24073590 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e0c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e28fbc640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c003a40 0x7f4e0c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 -- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 msgr2=0x7f4e0c0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e24073590 0x7f4e0c0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 -- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c003a40 msgr2=0x7f4e0c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c003a40 0x7f4e0c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 -- 172.21.15.83:0/4271764071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c003530 con 0x7f4e0c004810 2024-05-17T21:58:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e28fbc640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c003a40 0x7f4e0c003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e0c004c20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009ab0 tx=0x7f4e1401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a37fc939a886e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e140b2070 con 0x7f4e0c004810 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.868+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e140a5680 con 0x7f4e0c004810 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e1b7fe640 1 -- 172.21.15.83:0/4271764071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e140096b0 con 0x7f4e0c004810 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- 172.21.15.83:0/4271764071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 msgr2=0x7f4e0c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 --2- 172.21.15.83:0/4271764071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e0c004c20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e14009ab0 tx=0x7f4e1401ca90 comp rx=0 tx=0).stop 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e297bd640 1 -- 172.21.15.83:0/4271764071 reap_dead start 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- 172.21.15.83:0/4271764071 shutdown_connections 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- 172.21.15.83:0/4271764071 >> 172.21.15.83:0/4271764071 conn(0x7f4e2406cd90 msgr2=0x7f4e240b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- 172.21.15.83:0/4271764071 shutdown_connections 2024-05-17T21:58:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- 172.21.15.83:0/4271764071 wait complete. 2024-05-17T21:58:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 Processor -- start 2024-05-17T21:58:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.872+0000 7f4e2b220640 1 -- start start 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e240abe00 0x7f4e240aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 0x7f4e240d1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406fed0 con 0x7f4e0c004810 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24070770 con 0x7f4e240abe00 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4e24144330 con 0x7f4e240d1470 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60576/0 (socket says 172.21.15.83:60576) 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 -- 172.21.15.83:0/400936289 learned_addr learned my addr 172.21.15.83:0/400936289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e240abe00 0x7f4e240aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e297bd640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 0x7f4e240d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 -- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 msgr2=0x7f4e240d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 0x7f4e240d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 -- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e240abe00 msgr2=0x7f4e240aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e240abe00 0x7f4e240aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 -- 172.21.15.83:0/400936289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e240d1dc0 con 0x7f4e0c004810 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e297bd640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 0x7f4e240d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e28fbc640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e20010b10 tx=0x7f4e2000e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2001a070 con 0x7f4e0c004810 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e20011040 con 0x7f4e0c004810 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e240aa850 con 0x7f4e0c004810 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20015400 con 0x7f4e0c004810 2024-05-17T21:58:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e240ab040 con 0x7f4e0c004810 2024-05-17T21:58:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24070c90 con 0x7f4e0c004810 2024-05-17T21:58:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e2001d070 con 0x7f4e0c004810 2024-05-17T21:58:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 0x7f4e10071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 0x7f4e10071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e200924e0 con 0x7f4e0c004810 2024-05-17T21:58:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.876+0000 7f4e1bfff640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 0x7f4e10071940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e140af000 tx=0x7f4e140046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:01.880+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4e240d1dc0 con 0x7f4e0c004810 2024-05-17T21:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.040+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e241469f0 con 0x7f4e0c004810 2024-05-17T21:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.040+0000 7f4e197fa640 1 -- 172.21.15.83:0/400936289 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f4e2005ee60 con 0x7f4e0c004810 2024-05-17T21:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 msgr2=0x7f4e10071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 0x7f4e10071940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e140af000 tx=0x7f4e140046d0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 msgr2=0x7f4e2413e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4e20010b10 tx=0x7f4e2000e100 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 shutdown_connections 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e240d1470 0x7f4e240d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4e1006f430 0x7f4e10071940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e240abe00 0x7f4e240aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 --2- 172.21.15.83:0/400936289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e0c004810 0x7f4e2413e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 >> 172.21.15.83:0/400936289 conn(0x7f4e2406cd90 msgr2=0x7f4e240b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 shutdown_connections 2024-05-17T21:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.044+0000 7f4e2b220640 1 -- 172.21.15.83:0/400936289 wait complete. 2024-05-17T21:58:02.055 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:02.055 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:02.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.212+0000 7f30f4eeb640 1 Processor -- start 2024-05-17T21:58:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.212+0000 7f30f4eeb640 1 -- start start 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30f4eeb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f30f00b88b0 0x7f30f00d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30f4eeb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30f4eeb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f009c350 con 0x7f30f00ce130 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30f4eeb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f30f00d1a40 con 0x7f30f00b88b0 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30f4eeb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30f009d040 con 0x7f30f009ba00 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30ee59c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f30f00b88b0 0x7f30f00d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:42078/0 (socket says 172.21.15.83:42078) 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30eed9d640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f30f00ce130 0x7f30f00ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:48840/0 (socket says 172.21.15.83:48840) 2024-05-17T21:58:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 -- 172.21.15.83:0/4208761586 learned_addr learned my addr 172.21.15.83:0/4208761586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 -- 172.21.15.83:0/4208761586 >> v2:172.21.15.155:3300/0 conn(0x7f30f00b88b0 msgr2=0x7f30f00d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 --2- 172.21.15.83:0/4208761586 >> v2:172.21.15.155:3300/0 conn(0x7f30f00b88b0 0x7f30f00d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 -- 172.21.15.83:0/4208761586 >> v1:172.21.15.155:6789/0 conn(0x7f30f00ce130 legacy=0x7f30f00ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.216+0000 7f30edd9b640 1 -- 172.21.15.83:0/4208761586 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f009c710 con 0x7f30f009ba00 2024-05-17T21:58:02.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30edd9b640 1 --2- 172.21.15.83:0/4208761586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30e40098a0 tx=0x7f30e401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8183a7894ecbd8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30ed59a640 1 -- 172.21.15.83:0/4208761586 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40b0070 con 0x7f30f009ba00 2024-05-17T21:58:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30ed59a640 1 -- 172.21.15.83:0/4208761586 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e40ab600 con 0x7f30f009ba00 2024-05-17T21:58:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30ed59a640 1 -- 172.21.15.83:0/4208761586 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40ab9d0 con 0x7f30f009ba00 2024-05-17T21:58:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 -- 172.21.15.83:0/4208761586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 msgr2=0x7f30f009be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/4208761586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30e40098a0 tx=0x7f30e401c420 comp rx=0 tx=0).stop 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 -- 172.21.15.83:0/4208761586 shutdown_connections 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/4208761586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f009ba00 0x7f30f009be10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/4208761586 >> v2:172.21.15.155:3300/0 conn(0x7f30f00b88b0 0x7f30f00d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 -- 172.21.15.83:0/4208761586 >> 172.21.15.83:0/4208761586 conn(0x7f30f00d3620 msgr2=0x7f30f00d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 -- 172.21.15.83:0/4208761586 shutdown_connections 2024-05-17T21:58:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 -- 172.21.15.83:0/4208761586 wait complete. 2024-05-17T21:58:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.220+0000 7f30f4eeb640 1 Processor -- start 2024-05-17T21:58:02.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- start start 2024-05-17T21:58:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30f00b88b0 0x7f30f0158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f00ce130 0x7f30f0158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b2dc0 con 0x7f30f009ba00 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00cf060 con 0x7f30f00b88b0 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00b2f40 con 0x7f30f00ce130 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60584/0 (socket says 172.21.15.83:60584) 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 -- 172.21.15.83:0/2500412245 learned_addr learned my addr 172.21.15.83:0/2500412245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30edd9b640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30f00b88b0 0x7f30f0158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30eed9d640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f00ce130 0x7f30f0158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 -- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f00ce130 msgr2=0x7f30f0158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f00ce130 0x7f30f0158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 -- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30f00b88b0 msgr2=0x7f30f0158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30f00b88b0 0x7f30f0158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 -- 172.21.15.83:0/2500412245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f0159720 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30ee59c640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e870 tx=0x7f30e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0017070 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e000c940 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f015c930 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000ccf0 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f00b17e0 con 0x7f30f009ba00 2024-05-17T21:58:02.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.224+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f00d3c80 con 0x7f30f009ba00 2024-05-17T21:58:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.228+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f30e001a070 con 0x7f30f009ba00 2024-05-17T21:58:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.228+0000 7f30deffd640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 0x7f30d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.228+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30e008f950 con 0x7f30f009ba00 2024-05-17T21:58:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.228+0000 7f30edd9b640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 0x7f30d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.228+0000 7f30edd9b640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 0x7f30d0071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30e40ad000 tx=0x7f30e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.232+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f30e005c940 con 0x7f30f009ba00 2024-05-17T21:58:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f30f00d22b0 con 0x7f30f009ba00 2024-05-17T21:58:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30deffd640 1 -- 172.21.15.83:0/2500412245 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f30e005c240 con 0x7f30f009ba00 2024-05-17T21:58:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 msgr2=0x7f30d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 0x7f30d0071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30e40ad000 tx=0x7f30e4008040 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 msgr2=0x7f30f01fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e870 tx=0x7f30e000ec70 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 shutdown_connections 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30f00ce130 0x7f30f0158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30d006f400 0x7f30d0071910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30f00b88b0 0x7f30f0158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 --2- 172.21.15.83:0/2500412245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30f009ba00 0x7f30f01fa360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 >> 172.21.15.83:0/2500412245 conn(0x7f30f00d3620 msgr2=0x7f30f00b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 shutdown_connections 2024-05-17T21:58:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.408+0000 7f30f4eeb640 1 -- 172.21.15.83:0/2500412245 wait complete. 2024-05-17T21:58:02.422 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2024-05-17T21:58:02.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:02.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-17T21:58:02.509 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:02.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:58:02.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:02.509 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-17T21:58:02.522 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:02.523 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:02.528 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:02.529 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:02.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.780+0000 7fc15c3f2640 1 Processor -- start 2024-05-17T21:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 -- start start 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1540b7570 0x7fc1540cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540ce440 0x7fc1540ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1540ced90 0x7fc1541655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc1540cda70 con 0x7fc1540ced90 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc1540cf1a0 con 0x7fc1540b7570 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15c3f2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc1540d0c20 con 0x7fc1540ce440 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1540ced90 0x7fc1541655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1540ced90 0x7fc1541655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60588/0 (socket says 172.21.15.83:60588) 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 -- 172.21.15.83:0/4111918273 learned_addr learned my addr 172.21.15.83:0/4111918273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15998d640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540ce440 0x7fc1540ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a18e640 1 --2- 172.21.15.83:0/4111918273 >> v2:172.21.15.155:3300/0 conn(0x7fc1540b7570 0x7fc1540cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 -- 172.21.15.83:0/4111918273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540ce440 msgr2=0x7fc1540ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540ce440 0x7fc1540ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 -- 172.21.15.83:0/4111918273 >> v2:172.21.15.155:3300/0 conn(0x7fc1540b7570 msgr2=0x7fc1540cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 --2- 172.21.15.83:0/4111918273 >> v2:172.21.15.155:3300/0 conn(0x7fc1540b7570 0x7fc1540cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.784+0000 7fc15a98f640 1 -- 172.21.15.83:0/4111918273 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540d0f00 con 0x7fc1540ced90 2024-05-17T21:58:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15a98f640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ced90 0x7fc1541655e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc15406e7a0 tx=0x7fc14c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71639c9cbff158f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15918c640 1 -- 172.21.15.83:0/4111918273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0b1070 con 0x7fc1540ced90 2024-05-17T21:58:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15918c640 1 -- 172.21.15.83:0/4111918273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c0ac620 con 0x7fc1540ced90 2024-05-17T21:58:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15918c640 1 -- 172.21.15.83:0/4111918273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0ac9f0 con 0x7fc1540ced90 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- 172.21.15.83:0/4111918273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ced90 msgr2=0x7fc1541655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ced90 0x7fc1541655e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc15406e7a0 tx=0x7fc14c01c4f0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- 172.21.15.83:0/4111918273 shutdown_connections 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ced90 0x7fc1541655e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/4111918273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540ce440 0x7fc1540ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/4111918273 >> v2:172.21.15.155:3300/0 conn(0x7fc1540b7570 0x7fc1540cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- 172.21.15.83:0/4111918273 >> 172.21.15.83:0/4111918273 conn(0x7fc1540b4170 msgr2=0x7fc1540b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- 172.21.15.83:0/4111918273 shutdown_connections 2024-05-17T21:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- 172.21.15.83:0/4111918273 wait complete. 2024-05-17T21:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 Processor -- start 2024-05-17T21:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.788+0000 7fc15c3f2640 1 -- start start 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540b7570 0x7fc1540ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1540ced90 0x7fc15406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540b7b00 con 0x7fc1540ce440 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1540b7980 con 0x7fc1540ced90 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc154168750 con 0x7fc1540b7570 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60600/0 (socket says 172.21.15.83:60600) 2024-05-17T21:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 -- 172.21.15.83:0/998509072 learned_addr learned my addr 172.21.15.83:0/998509072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 -- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540b7570 msgr2=0x7fc1540ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540b7570 0x7fc1540ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 -- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1540ced90 msgr2=0x7fc15406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15a98f640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1540ced90 0x7fc15406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1540ced90 0x7fc15406c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 -- 172.21.15.83:0/998509072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1540c9680 con 0x7fc1540ce440 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15998d640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc14400d5d0 tx=0x7fc14400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144004240 con 0x7fc1540ce440 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1440043e0 con 0x7fc1540ce440 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1540c9960 con 0x7fc1540ce440 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144003840 con 0x7fc1540ce440 2024-05-17T21:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1540c9f20 con 0x7fc1540ce440 2024-05-17T21:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1540ca360 con 0x7fc1540ce440 2024-05-17T21:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1440039e0 con 0x7fc1540ce440 2024-05-17T21:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 0x7fc138071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc15a18e640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 0x7fc138071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:02.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.792+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc14408f4e0 con 0x7fc1540ce440 2024-05-17T21:58:02.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.796+0000 7fc15a18e640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 0x7fc138071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1500070b0 tx=0x7fc150007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:02.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.800+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc14405c3d0 con 0x7fc1540ce440 2024-05-17T21:58:02.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.956+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1540cdf00 con 0x7fc1540ce440 2024-05-17T21:58:02.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.956+0000 7fc14affd640 1 -- 172.21.15.83:0/998509072 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fc14405bcd0 con 0x7fc1540ce440 2024-05-17T21:58:02.951 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 msgr2=0x7fc138071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 0x7fc138071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1500070b0 tx=0x7fc150007040 comp rx=0 tx=0).stop 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 msgr2=0x7fc1541556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc14400d5d0 tx=0x7fc14400da90 comp rx=0 tx=0).stop 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 shutdown_connections 2024-05-17T21:58:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc13806f400 0x7fc138071910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1540ced90 0x7fc15406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1540ce440 0x7fc1541556a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 --2- 172.21.15.83:0/998509072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1540b7570 0x7fc1540ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 >> 172.21.15.83:0/998509072 conn(0x7fc1540b4170 msgr2=0x7fc1540b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 shutdown_connections 2024-05-17T21:58:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:02.960+0000 7fc15c3f2640 1 -- 172.21.15.83:0/998509072 wait complete. 2024-05-17T21:58:02.971 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:02.971 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:02.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:03.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 Processor -- start 2024-05-17T21:58:03.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 -- start start 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79380cdf70 0x7f79380ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 0x7f79380cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793806c930 con 0x7f79380b2f70 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f793806d540 con 0x7f79380cdf70 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f793dfea640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f793806cc10 con 0x7f79380d1f50 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f79377fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f79380b2f70 0x7f79380d1840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49112/0 (socket says 172.21.15.83:49112) 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7936ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79380cdf70 0x7f79380ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f79377fe640 1 -- 172.21.15.83:0/2294601793 learned_addr learned my addr 172.21.15.83:0/2294601793 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7937fff640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 0x7f79380cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7936ffd640 1 -- 172.21.15.83:0/2294601793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 msgr2=0x7f79380cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:03.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7936ffd640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 0x7f79380cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7936ffd640 1 -- 172.21.15.83:0/2294601793 >> v1:172.21.15.83:6789/0 conn(0x7f79380b2f70 legacy=0x7f79380d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7936ffd640 1 -- 172.21.15.83:0/2294601793 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793806cef0 con 0x7f79380cdf70 2024-05-17T21:58:03.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.176+0000 7f7937fff640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 0x7f79380cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f7936ffd640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380cdf70 0x7f79380ce6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f792c0097c0 tx=0x7f792c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daee261da65bcfbb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f79367fc640 1 -- 172.21.15.83:0/2294601793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0b0070 con 0x7f79380cdf70 2024-05-17T21:58:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f79367fc640 1 -- 172.21.15.83:0/2294601793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792c0ab600 con 0x7f79380cdf70 2024-05-17T21:58:03.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f79367fc640 1 -- 172.21.15.83:0/2294601793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0ab9d0 con 0x7f79380cdf70 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 -- 172.21.15.83:0/2294601793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380cdf70 msgr2=0x7f79380ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380cdf70 0x7f79380ce6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f792c0097c0 tx=0x7f792c01c410 comp rx=0 tx=0).stop 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 -- 172.21.15.83:0/2294601793 shutdown_connections 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380d1f50 0x7f79380cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 --2- 172.21.15.83:0/2294601793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380cdf70 0x7f79380ce6f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 -- 172.21.15.83:0/2294601793 >> 172.21.15.83:0/2294601793 conn(0x7f79380b4170 msgr2=0x7f79380b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:03.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 -- 172.21.15.83:0/2294601793 shutdown_connections 2024-05-17T21:58:03.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.180+0000 7f793dfea640 1 -- 172.21.15.83:0/2294601793 wait complete. 2024-05-17T21:58:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 Processor -- start 2024-05-17T21:58:03.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- start start 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380b2f70 0x7f79380b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79380cdf70 0x7f7938150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cfd50 con 0x7f79380d1f50 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cfbd0 con 0x7f79380cdf70 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f79380b8060 con 0x7f79380b2f70 2024-05-17T21:58:03.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60614/0 (socket says 172.21.15.83:60614) 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 -- 172.21.15.83:0/3484651938 learned_addr learned my addr 172.21.15.83:0/3484651938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 -- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380b2f70 msgr2=0x7f79380b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380b2f70 0x7f79380b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7936ffd640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79380cdf70 0x7f7938150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 -- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79380cdf70 msgr2=0x7f7938150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79380cdf70 0x7f7938150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 -- 172.21.15.83:0/3484651938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7938158300 con 0x7f79380d1f50 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f7937fff640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7930007e60 tx=0x7f793000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793001a070 con 0x7f79380d1f50 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7930015600 con 0x7f79380d1f50 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79380b0810 con 0x7f79380d1f50 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300159f0 con 0x7f79380d1f50 2024-05-17T21:58:03.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7938151da0 con 0x7f79380d1f50 2024-05-17T21:58:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.184+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7938158050 con 0x7f79380d1f50 2024-05-17T21:58:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.188+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f793001d070 con 0x7f79380d1f50 2024-05-17T21:58:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.188+0000 7f791ffff640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 0x7f7924071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.188+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f793008f520 con 0x7f79380d1f50 2024-05-17T21:58:03.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.188+0000 7f79377fe640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 0x7f7924071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.188+0000 7f79377fe640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 0x7f7924071940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7928001550 tx=0x7f7928007a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.192+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7930010070 con 0x7f79380d1f50 2024-05-17T21:58:03.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.364+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f79380d1840 con 0x7f79380d1f50 2024-05-17T21:58:03.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.364+0000 7f791ffff640 1 -- 172.21.15.83:0/3484651938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f793005c410 con 0x7f79380d1f50 2024-05-17T21:58:03.358 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:03.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 msgr2=0x7f7924071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 0x7f7924071940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7928001550 tx=0x7f7928007a50 comp rx=0 tx=0).stop 2024-05-17T21:58:03.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 msgr2=0x7f79381575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7930007e60 tx=0x7f793000d4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 shutdown_connections 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79380d1f50 0x7f79381575d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f792406f430 0x7f7924071940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79380cdf70 0x7f7938150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 --2- 172.21.15.83:0/3484651938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79380b2f70 0x7f79380b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 >> 172.21.15.83:0/3484651938 conn(0x7f79380b4170 msgr2=0x7f79380d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 shutdown_connections 2024-05-17T21:58:03.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.368+0000 7f793dfea640 1 -- 172.21.15.83:0/3484651938 wait complete. 2024-05-17T21:58:03.380 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T21:58:03.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:03.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-17T21:58:03.392 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:03.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:58:03.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:03.392 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-17T21:58:03.444 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:03.444 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:03.450 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:03.450 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:03.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:03.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 Processor -- start 2024-05-17T21:58:03.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 -- start start 2024-05-17T21:58:03.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50dc09c9a0 0x7f50dc0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 0x7f50dc071700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc071c40 0x7f50dc0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f50dc072050 con 0x7f50dc09c9a0 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f50dc0b2ee0 con 0x7f50dc09cdb0 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50e1e0a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f50dc0cfad0 con 0x7f50dc071c40 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50db7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50dc09c9a0 0x7f50dc0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50db7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50dc09c9a0 0x7f50dc0711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60620/0 (socket says 172.21.15.83:60620) 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.660+0000 7f50db7fe640 1 -- 172.21.15.83:0/2136717098 learned_addr learned my addr 172.21.15.83:0/2136717098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 -- 172.21.15.83:0/2136717098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc071c40 msgr2=0x7f50dc0ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 --2- 172.21.15.83:0/2136717098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc071c40 0x7f50dc0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 -- 172.21.15.83:0/2136717098 >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 msgr2=0x7f50dc071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50daffd640 1 --2- 172.21.15.83:0/2136717098 >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 0x7f50dc071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 --2- 172.21.15.83:0/2136717098 >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 0x7f50dc071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 -- 172.21.15.83:0/2136717098 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc0d1630 con 0x7f50dc09c9a0 2024-05-17T21:58:03.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50daffd640 1 --2- 172.21.15.83:0/2136717098 >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 0x7f50dc071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50db7fe640 1 --2- 172.21.15.83:0/2136717098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc09c9a0 0x7f50dc0711c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0cdc00 tx=0x7f50d401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c52d340720d6da0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50da7fc640 1 -- 172.21.15.83:0/2136717098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40a56f0 con 0x7f50dc09c9a0 2024-05-17T21:58:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50da7fc640 1 -- 172.21.15.83:0/2136717098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d40a5890 con 0x7f50dc09c9a0 2024-05-17T21:58:03.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50da7fc640 1 -- 172.21.15.83:0/2136717098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40a5c60 con 0x7f50dc09c9a0 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 -- 172.21.15.83:0/2136717098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc09c9a0 msgr2=0x7f50dc0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/2136717098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc09c9a0 0x7f50dc0711c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0cdc00 tx=0x7f50d401c960 comp rx=0 tx=0).stop 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 -- 172.21.15.83:0/2136717098 shutdown_connections 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/2136717098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc071c40 0x7f50dc0ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/2136717098 >> v2:172.21.15.155:3300/0 conn(0x7f50dc09cdb0 0x7f50dc071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/2136717098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc09c9a0 0x7f50dc0711c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 -- 172.21.15.83:0/2136717098 >> 172.21.15.83:0/2136717098 conn(0x7f50dc0d00b0 msgr2=0x7f50dc0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 -- 172.21.15.83:0/2136717098 shutdown_connections 2024-05-17T21:58:03.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.664+0000 7f50e1e0a640 1 -- 172.21.15.83:0/2136717098 wait complete. 2024-05-17T21:58:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 Processor -- start 2024-05-17T21:58:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 -- start start 2024-05-17T21:58:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 0x7f50dc15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 0x7f50dc15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc0d1200 con 0x7f50dc071c40 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc0b8840 con 0x7f50dc09cdb0 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50e1e0a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f50dc0d1380 con 0x7f50dc09c9a0 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60636/0 (socket says 172.21.15.83:60636) 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.668+0000 7f50db7fe640 1 -- 172.21.15.83:0/3689104545 learned_addr learned my addr 172.21.15.83:0/3689104545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50dbfff640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 0x7f50dc15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 -- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 msgr2=0x7f50dc15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50daffd640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 0x7f50dc15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 0x7f50dc15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 -- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 msgr2=0x7f50dc15b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 0x7f50dc15b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 -- 172.21.15.83:0/3689104545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc15ee80 con 0x7f50dc071c40 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50dbfff640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 0x7f50dc15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50daffd640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 0x7f50dc15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50db7fe640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f50d40af040 tx=0x7f50d4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40bc020 con 0x7f50dc071c40 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d40b8670 con 0x7f50dc071c40 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc15e8c0 con 0x7f50dc071c40 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40b8a60 con 0x7f50dc071c40 2024-05-17T21:58:03.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc15f9b0 con 0x7f50dc071c40 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50dc0d19e0 con 0x7f50dc071c40 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50d40bf020 con 0x7f50dc071c40 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 0x7f50bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50daffd640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 0x7f50bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f50d412a2d0 con 0x7f50dc071c40 2024-05-17T21:58:03.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.672+0000 7f50daffd640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 0x7f50bc071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50dc161c30 tx=0x7f50c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:03.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.676+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f50d40c6030 con 0x7f50dc071c40 2024-05-17T21:58:03.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.848+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f50dc0717d0 con 0x7f50dc071c40 2024-05-17T21:58:03.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50d3fff640 1 -- 172.21.15.83:0/3689104545 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f50dc15ee80 con 0x7f50dc071c40 2024-05-17T21:58:03.844 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:03.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 msgr2=0x7f50bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 0x7f50bc071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50dc161c30 tx=0x7f50c4008040 comp rx=0 tx=0).stop 2024-05-17T21:58:03.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 msgr2=0x7f50dc15dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:03.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f50d40af040 tx=0x7f50d4019700 comp rx=0 tx=0).stop 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 shutdown_connections 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50bc06f400 0x7f50bc071910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50dc09cdb0 0x7f50dc15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50dc09c9a0 0x7f50dc15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 --2- 172.21.15.83:0/3689104545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50dc071c40 0x7f50dc15dfc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 >> 172.21.15.83:0/3689104545 conn(0x7f50dc0d00b0 msgr2=0x7f50dc0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 shutdown_connections 2024-05-17T21:58:03.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:03.852+0000 7f50e1e0a640 1 -- 172.21.15.83:0/3689104545 wait complete. 2024-05-17T21:58:03.866 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:03.866 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:03.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:04.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 Processor -- start 2024-05-17T21:58:04.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 -- start start 2024-05-17T21:58:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058806cea0 0x7f058815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058809bf60 con 0x7f058815e930 2024-05-17T21:58:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058815e170 con 0x7f05880b3bc0 2024-05-17T21:58:04.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f058daaa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05880b5090 con 0x7f058806cea0 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f058815e930 0x7f058806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49126/0 (socket says 172.21.15.83:49126) 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 -- 172.21.15.83:0/1308205207 learned_addr learned my addr 172.21.15.83:0/1308205207 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0587fff640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058806cea0 0x7f058815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106782233 0 0) 0x7f058809bf60 con 0x7f058815e930 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0570003530 con 0x7f058815e930 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136601249 0 0) 0x7f0570003530 con 0x7f058815e930 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058806cea0 msgr2=0x7f058815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058806cea0 0x7f058815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 >> v1:172.21.15.155:6789/0 conn(0x7f05880b3bc0 legacy=0x7f05880af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05880b4ad0 con 0x7f058815e930 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f057c001700 con 0x7f058815e930 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 >> v1:172.21.15.83:6789/0 conn(0x7f058815e930 legacy=0x7f058806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0570003a40 0x7f0570003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f0570004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0570005160 0x7f057000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05880b5090 con 0x7f0570004810 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f057000c020 con 0x7f0570005160 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f057000c220 con 0x7f0570003a40 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f0570004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f05877fe640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0570003a40 0x7f0570003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0587fff640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0570005160 0x7f057000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 -- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0570003a40 msgr2=0x7f0570003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0570003a40 0x7f0570003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 -- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0570005160 msgr2=0x7f057000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0570005160 0x7f057000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0586ffd640 1 -- 172.21.15.83:0/1308205207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0570003530 con 0x7f0570004810 2024-05-17T21:58:04.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.036+0000 7f0587fff640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0570005160 0x7f057000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f0586ffd640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f0570004c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f057c001060 tx=0x7f057c01f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c6aeb4a4d58298 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c007d20 con 0x7f0570004810 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c005ae0 con 0x7f0570004810 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f05867fc640 1 -- 172.21.15.83:0/1308205207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c007070 con 0x7f0570004810 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 -- 172.21.15.83:0/1308205207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 msgr2=0x7f0570004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 --2- 172.21.15.83:0/1308205207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f0570004c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f057c001060 tx=0x7f057c01f2d0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f0587fff640 1 -- 172.21.15.83:0/1308205207 reap_dead start 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 -- 172.21.15.83:0/1308205207 shutdown_connections 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 -- 172.21.15.83:0/1308205207 >> 172.21.15.83:0/1308205207 conn(0x7f058809b8b0 msgr2=0x7f05880b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 -- 172.21.15.83:0/1308205207 shutdown_connections 2024-05-17T21:58:04.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.040+0000 7f058daaa640 1 -- 172.21.15.83:0/1308205207 wait complete. 2024-05-17T21:58:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 Processor -- start 2024-05-17T21:58:04.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- start start 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058815cf80 0x7f058814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0588155d80 0x7f0588156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05880b4880 con 0x7f0570004810 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05880b3fd0 con 0x7f0588155d80 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f058815e410 con 0x7f058815cf80 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60656/0 (socket says 172.21.15.83:60656) 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 -- 172.21.15.83:0/3432484844 learned_addr learned my addr 172.21.15.83:0/3432484844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f0587fff640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0588155d80 0x7f0588156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f0586ffd640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058815cf80 0x7f058814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 -- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058815cf80 msgr2=0x7f058814b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058815cf80 0x7f058814b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 -- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0588155d80 msgr2=0x7f0588156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0588155d80 0x7f0588156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 -- 172.21.15.83:0/3432484844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05881566d0 con 0x7f0570004810 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f05877fe640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f057800e4c0 tx=0x7f057800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05780052e0 con 0x7f0570004810 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0578018070 con 0x7f0570004810 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05881e8250 con 0x7f0570004810 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578003840 con 0x7f0570004810 2024-05-17T21:58:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.044+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05881e8810 con 0x7f0570004810 2024-05-17T21:58:04.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05880af450 con 0x7f0570004810 2024-05-17T21:58:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f05780039e0 con 0x7f0570004810 2024-05-17T21:58:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f056ffff640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 0x7f0574071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f057808ee90 con 0x7f0570004810 2024-05-17T21:58:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f0586ffd640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 0x7f0574071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.048+0000 7f0586ffd640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 0x7f0574071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ae000 tx=0x7f057c000ff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.052+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f05881566d0 con 0x7f0570004810 2024-05-17T21:58:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f058806d680 con 0x7f0570004810 2024-05-17T21:58:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f056ffff640 1 -- 172.21.15.83:0/3432484844 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f057805be40 con 0x7f0570004810 2024-05-17T21:58:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 msgr2=0x7f0574071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 0x7f0574071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f057c0ae000 tx=0x7f057c000ff0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 msgr2=0x7f05881e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.220+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f057800e4c0 tx=0x7f057800e980 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 shutdown_connections 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f057406f430 0x7f0574071940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0588155d80 0x7f0588156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f058815cf80 0x7f058814b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 --2- 172.21.15.83:0/3432484844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0570004810 0x7f05881e97b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 >> 172.21.15.83:0/3432484844 conn(0x7f058809b8b0 msgr2=0x7f05880717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:04.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 shutdown_connections 2024-05-17T21:58:04.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.224+0000 7f058daaa640 1 -- 172.21.15.83:0/3432484844 wait complete. 2024-05-17T21:58:04.233 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2024-05-17T21:58:04.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:04.234 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-17T21:58:04.248 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:04.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T21:58:04.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:04.248 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-17T21:58:04.296 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T21:58:04.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:04.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 Processor -- start 2024-05-17T21:58:04.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 -- start start 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc500d1040 0x7fdc50144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 0x7fdc5006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5015ea40 con 0x7fdc5015e080 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5015ed20 con 0x7fdc500d1040 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc5759a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5015f000 con 0x7fdc50073540 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc500d1040 0x7fdc50144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc54b35640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc5015e080 0x7fdc50072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49140/0 (socket says 172.21.15.83:49140) 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc500d1040 0x7fdc50144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60672/0 (socket says 172.21.15.83:60672) 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc54b35640 1 -- 172.21.15.83:0/3571734174 learned_addr learned my addr 172.21.15.83:0/3571734174 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55b37640 1 --2- 172.21.15.83:0/3571734174 >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 0x7fdc5006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 -- 172.21.15.83:0/3571734174 >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 msgr2=0x7fdc5006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 --2- 172.21.15.83:0/3571734174 >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 0x7fdc5006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 -- 172.21.15.83:0/3571734174 >> v1:172.21.15.83:6789/0 conn(0x7fdc5015e080 legacy=0x7fdc50072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55336640 1 -- 172.21.15.83:0/3571734174 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5006ef30 con 0x7fdc500d1040 2024-05-17T21:58:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.456+0000 7fdc55b37640 1 --2- 172.21.15.83:0/3571734174 >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 0x7fdc5006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc55336640 1 --2- 172.21.15.83:0/3571734174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc500d1040 0x7fdc50144080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc50072c30 tx=0x7fdc4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e890761e26373f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc4ffff640 1 -- 172.21.15.83:0/3571734174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480b1070 con 0x7fdc500d1040 2024-05-17T21:58:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc4ffff640 1 -- 172.21.15.83:0/3571734174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc480ac5f0 con 0x7fdc500d1040 2024-05-17T21:58:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc4ffff640 1 -- 172.21.15.83:0/3571734174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480ac9c0 con 0x7fdc500d1040 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 -- 172.21.15.83:0/3571734174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc500d1040 msgr2=0x7fdc50144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 --2- 172.21.15.83:0/3571734174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc500d1040 0x7fdc50144080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdc50072c30 tx=0x7fdc4801c480 comp rx=0 tx=0).stop 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 -- 172.21.15.83:0/3571734174 shutdown_connections 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 --2- 172.21.15.83:0/3571734174 >> v2:172.21.15.155:3300/0 conn(0x7fdc50073540 0x7fdc5006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 --2- 172.21.15.83:0/3571734174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc500d1040 0x7fdc50144080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 -- 172.21.15.83:0/3571734174 >> 172.21.15.83:0/3571734174 conn(0x7fdc5006cd90 msgr2=0x7fdc500b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 -- 172.21.15.83:0/3571734174 shutdown_connections 2024-05-17T21:58:04.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.460+0000 7fdc5759a640 1 -- 172.21.15.83:0/3571734174 wait complete. 2024-05-17T21:58:04.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 Processor -- start 2024-05-17T21:58:04.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- start start 2024-05-17T21:58:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 0x7fdc50158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc5015e080 0x7fdc50136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50070810 con 0x7fdc5013ee00 2024-05-17T21:58:04.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50070690 con 0x7fdc5015e080 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdc50070990 con 0x7fdc500d1040 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60678/0 (socket says 172.21.15.83:60678) 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 -- 172.21.15.83:0/2305248330 learned_addr learned my addr 172.21.15.83:0/2305248330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc54b35640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc5015e080 0x7fdc50136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55336640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 0x7fdc50158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 -- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 msgr2=0x7fdc50158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 0x7fdc50158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 -- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc5015e080 msgr2=0x7fdc50136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc5015e080 0x7fdc50136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 -- 172.21.15.83:0/2305248330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc50158e90 con 0x7fdc5013ee00 2024-05-17T21:58:04.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55336640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 0x7fdc50158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc55b37640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000cdb0 tx=0x7fdc4000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000f680 con 0x7fdc5013ee00 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4000f840 con 0x7fdc5013ee00 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc50134a80 con 0x7fdc5013ee00 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4000fc30 con 0x7fdc5013ee00 2024-05-17T21:58:04.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.464+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc50135040 con 0x7fdc5013ee00 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5006f730 con 0x7fdc5013ee00 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdc400213f0 con 0x7fdc5013ee00 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc4dffb640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 0x7fdc3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc4008feb0 con 0x7fdc5013ee00 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc55336640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 0x7fdc3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.468+0000 7fdc55336640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 0x7fdc3c071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc480ae000 tx=0x7fdc48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.472+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdc40014030 con 0x7fdc5013ee00 2024-05-17T21:58:04.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.668+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc5013f750 con 0x7fdc5013ee00 2024-05-17T21:58:04.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.668+0000 7fdc4dffb640 1 -- 172.21.15.83:0/2305248330 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fdc4005cec0 con 0x7fdc5013ee00 2024-05-17T21:58:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:58:04.663 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:58:04.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 msgr2=0x7fdc3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 0x7fdc3c071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc480ae000 tx=0x7fdc48008040 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 msgr2=0x7fdc5013f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000cdb0 tx=0x7fdc4000a650 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 shutdown_connections 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5013ee00 0x7fdc5013f210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdc3c06f400 0x7fdc3c071910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc5015e080 0x7fdc50136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 --2- 172.21.15.83:0/2305248330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc500d1040 0x7fdc50158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 >> 172.21.15.83:0/2305248330 conn(0x7fdc5006cd90 msgr2=0x7fdc5013d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 shutdown_connections 2024-05-17T21:58:04.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.672+0000 7fdc5759a640 1 -- 172.21.15.83:0/2305248330 wait complete. 2024-05-17T21:58:04.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 Processor -- start 2024-05-17T21:58:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 -- start start 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f169c06cea0 0x7f169c168800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f169c15e0d0 con 0x7f169c15dcc0 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f169c0b5570 con 0x7f169c0b4e20 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f16a0b84640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f169c0b6180 con 0x7f169c06cea0 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f1699d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f169c15dcc0 0x7f169c06c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49156/0 (socket says 172.21.15.83:49156) 2024-05-17T21:58:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f1699d9b640 1 -- 172.21.15.83:0/3630554092 learned_addr learned my addr 172.21.15.83:0/3630554092 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169ad9d640 1 --2- 172.21.15.83:0/3630554092 >> v2:172.21.15.155:3300/0 conn(0x7f169c06cea0 0x7f169c168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684291092 0 0) 0x7f169c15e0d0 con 0x7f169c15dcc0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1684003530 con 0x7f169c15dcc0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2158602525 0 0) 0x7f1684003530 con 0x7f169c15dcc0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 >> v2:172.21.15.155:3300/0 conn(0x7f169c06cea0 msgr2=0x7f169c168800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 --2- 172.21.15.83:0/3630554092 >> v2:172.21.15.155:3300/0 conn(0x7f169c06cea0 0x7f169c168800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 >> v1:172.21.15.155:6789/0 conn(0x7f169c0b4e20 legacy=0x7f169c0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c0b5850 con 0x7f169c15dcc0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1690001700 con 0x7f169c15dcc0 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169ad9d640 1 --2- 172.21.15.83:0/3630554092 >> v2:172.21.15.155:3300/0 conn(0x7f169c06cea0 0x7f169c168800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 >> v1:172.21.15.83:6789/0 conn(0x7f169c15dcc0 legacy=0x7f169c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 0x7f1684003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1684004810 0x7f1684004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684005310 0x7f168400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f169c0b6180 con 0x7f1684003a60 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f168400c2b0 con 0x7f1684005310 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f168400c4b0 con 0x7f1684004810 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.828+0000 7f169a59c640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 0x7f1684003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f1699d9b640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1684004810 0x7f1684004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 -- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1684004810 msgr2=0x7f1684004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1684004810 0x7f1684004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 -- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684005310 msgr2=0x7f168400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169ad9d640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684005310 0x7f168400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684005310 0x7f168400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169ad9d640 1 -- 172.21.15.83:0/3630554092 reap_dead start 2024-05-17T21:58:04.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 -- 172.21.15.83:0/3630554092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1684003530 con 0x7f1684003a60 2024-05-17T21:58:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169a59c640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 0x7f1684003ed0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f168c001790 tx=0x7f168c01c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c633459787853bc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0013e0 con 0x7f1684003a60 2024-05-17T21:58:04.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c0052d0 con 0x7f1684003a60 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f169959a640 1 -- 172.21.15.83:0/3630554092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c003840 con 0x7f1684003a60 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 -- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 msgr2=0x7f1684003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 0x7f1684003ed0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f168c001790 tx=0x7f168c01c2e0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 -- 172.21.15.83:0/3630554092 shutdown_connections 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 --2- 172.21.15.83:0/3630554092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1684003a60 0x7f1684003ed0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 -- 172.21.15.83:0/3630554092 >> 172.21.15.83:0/3630554092 conn(0x7f169c0b4270 msgr2=0x7f169c0b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 -- 172.21.15.83:0/3630554092 shutdown_connections 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.832+0000 7f16a0b84640 1 -- 172.21.15.83:0/3630554092 wait complete. 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 Processor -- start 2024-05-17T21:58:04.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- start start 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 0x7f169c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f169c0b7bb0 0x7f169c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f169c06cb20 con 0x7f169c0cafc0 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f169c06c9a0 con 0x7f1684003a60 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f169c06cca0 con 0x7f169c0b7bb0 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60708/0 (socket says 172.21.15.83:60708) 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 -- 172.21.15.83:0/1305989090 learned_addr learned my addr 172.21.15.83:0/1305989090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f169ad9d640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f169c0b7bb0 0x7f169c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f169a59c640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 0x7f169c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 -- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f169c0b7bb0 msgr2=0x7f169c0c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f169c0b7bb0 0x7f169c0c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 -- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 msgr2=0x7f169c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 0x7f169c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 -- 172.21.15.83:0/1305989090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c152e00 con 0x7f169c0cafc0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f169a59c640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 0x7f169c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f1699d9b640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f169000d910 tx=0x7f169000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16900074e0 con 0x7f169c0cafc0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16900047f0 con 0x7f169c0cafc0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c0b7fc0 con 0x7f169c0cafc0 2024-05-17T21:58:04.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690007680 con 0x7f169c0cafc0 2024-05-17T21:58:04.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c0c72a0 con 0x7f169c0cafc0 2024-05-17T21:58:04.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.836+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c0cb7d0 con 0x7f169c0cafc0 2024-05-17T21:58:04.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.840+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f169001b020 con 0x7f169c0cafc0 2024-05-17T21:58:04.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.840+0000 7f168affd640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 0x7f167c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.840+0000 7f169a59c640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 0x7f167c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.840+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f169008f840 con 0x7f169c0cafc0 2024-05-17T21:58:04.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.840+0000 7f169a59c640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 0x7f167c071940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f168c0ac000 tx=0x7f168c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:04.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:04.844+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1690022070 con 0x7f169c0cafc0 2024-05-17T21:58:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.036+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169c06cfa0 con 0x7f169c0cafc0 2024-05-17T21:58:05.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.036+0000 7f168affd640 1 -- 172.21.15.83:0/1305989090 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f169c0b7fc0 con 0x7f169c0cafc0 2024-05-17T21:58:05.031 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:05.031 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:58:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 msgr2=0x7f167c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 0x7f167c071940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f168c0ac000 tx=0x7f168c009040 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 msgr2=0x7f169c0b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f169000d910 tx=0x7f169000ddd0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 shutdown_connections 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f169c0b7bb0 0x7f169c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c0cafc0 0x7f169c0b7670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f167c06f430 0x7f167c071940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 --2- 172.21.15.83:0/1305989090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1684003a60 0x7f169c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 >> 172.21.15.83:0/1305989090 conn(0x7f169c0b4270 msgr2=0x7f169c0708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 shutdown_connections 2024-05-17T21:58:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.040+0000 7f16a0b84640 1 -- 172.21.15.83:0/1305989090 wait complete. 2024-05-17T21:58:05.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.200+0000 7f5735572640 1 Processor -- start 2024-05-17T21:58:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.200+0000 7f5735572640 1 -- start start 2024-05-17T21:58:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f5735572640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5730073540 0x7f57300b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f5735572640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f573006ff60 con 0x7f573015fef0 2024-05-17T21:58:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f5735572640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57300b49b0 con 0x7f57300b5a50 2024-05-17T21:58:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f5735572640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f57300b4c90 con 0x7f5730073540 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f573015fef0 0x7f573006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49158/0 (socket says 172.21.15.83:49158) 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 -- 172.21.15.83:0/581163659 learned_addr learned my addr 172.21.15.83:0/581163659 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572f7fe640 1 --2- 172.21.15.83:0/581163659 >> v2:172.21.15.155:3300/0 conn(0x7f5730073540 0x7f57300b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660697568 0 0) 0x7f573006ff60 con 0x7f573015fef0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5718003530 con 0x7f573015fef0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1337435372 0 0) 0x7f5718003530 con 0x7f573015fef0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 >> v2:172.21.15.155:3300/0 conn(0x7f5730073540 msgr2=0x7f57300b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 --2- 172.21.15.83:0/581163659 >> v2:172.21.15.155:3300/0 conn(0x7f5730073540 0x7f57300b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 >> v1:172.21.15.155:6789/0 conn(0x7f57300b5a50 legacy=0x7f5730072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57300b4f70 con 0x7f573015fef0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f5720001700 con 0x7f573015fef0 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 >> v1:172.21.15.83:6789/0 conn(0x7f573015fef0 legacy=0x7f573006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 0x7f5718003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5718004810 0x7f5718004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718005360 0x7f571800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57300b4c90 con 0x7f5718003a60 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f571800c290 con 0x7f5718004810 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f571800c490 con 0x7f5718005360 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 0x7f5718003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572e7fc640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5718004810 0x7f5718004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572f7fe640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718005360 0x7f571800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 -- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718005360 msgr2=0x7f571800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718005360 0x7f571800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 -- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5718004810 msgr2=0x7f5718004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5718004810 0x7f5718004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572f7fe640 1 -- 172.21.15.83:0/581163659 reap_dead start 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572effd640 1 -- 172.21.15.83:0/581163659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5718003530 con 0x7f5718003a60 2024-05-17T21:58:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.204+0000 7f572e7fc640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5718004810 0x7f5718004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f572effd640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 0x7f5718003ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f572000c240 tx=0x7f572001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b849cea90ba36a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720007cf0 con 0x7f5718003a60 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57200a9020 con 0x7f5718003a60 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f572dffb640 1 -- 172.21.15.83:0/581163659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720005b20 con 0x7f5718003a60 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 -- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 msgr2=0x7f5718003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 0x7f5718003ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f572000c240 tx=0x7f572001cb80 comp rx=0 tx=0).stop 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 -- 172.21.15.83:0/581163659 shutdown_connections 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 --2- 172.21.15.83:0/581163659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5718003a60 0x7f5718003ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 -- 172.21.15.83:0/581163659 >> 172.21.15.83:0/581163659 conn(0x7f573006cd90 msgr2=0x7f57300b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 -- 172.21.15.83:0/581163659 shutdown_connections 2024-05-17T21:58:05.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.208+0000 7f5735572640 1 -- 172.21.15.83:0/581163659 wait complete. 2024-05-17T21:58:05.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 Processor -- start 2024-05-17T21:58:05.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- start start 2024-05-17T21:58:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718003a60 0x7f573014ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5730162b10 0x7f5730162f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57300b4750 con 0x7f57301f9420 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57300b45d0 con 0x7f5730162b10 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f57300af8e0 con 0x7f5718003a60 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60732/0 (socket says 172.21.15.83:60732) 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 -- 172.21.15.83:0/28333447 learned_addr learned my addr 172.21.15.83:0/28333447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572effd640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718003a60 0x7f573014ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572f7fe640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5730162b10 0x7f5730162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 -- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718003a60 msgr2=0x7f573014ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718003a60 0x7f573014ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 -- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5730162b10 msgr2=0x7f5730162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5730162b10 0x7f5730162f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 -- 172.21.15.83:0/28333447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730163460 con 0x7f57301f9420 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f572e7fc640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f572400e4e0 tx=0x7f572400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240053a0 con 0x7f57301f9420 2024-05-17T21:58:05.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724007020 con 0x7f57301f9420 2024-05-17T21:58:05.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57301fbf20 con 0x7f57301f9420 2024-05-17T21:58:05.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240071d0 con 0x7f57301f9420 2024-05-17T21:58:05.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57301fc4a0 con 0x7f57301f9420 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.212+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573014f190 con 0x7f57301f9420 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.216+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f572401a030 con 0x7f57301f9420 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.216+0000 7f57177fe640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 0x7f571c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.216+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f572408e7c0 con 0x7f57301f9420 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.216+0000 7f572effd640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 0x7f571c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.216+0000 7f572effd640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 0x7f571c071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57200aa000 tx=0x7f57300b66e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.220+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5724021020 con 0x7f57301f9420 2024-05-17T21:58:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.380+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f573006f850 con 0x7f57301f9420 2024-05-17T21:58:05.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.380+0000 7f57177fe640 1 -- 172.21.15.83:0/28333447 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f5730163460 con 0x7f57301f9420 2024-05-17T21:58:05.373 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:05.373 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 msgr2=0x7f571c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 0x7f571c071940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57200aa000 tx=0x7f57300b66e0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 msgr2=0x7f573015f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f572400e4e0 tx=0x7f572400e9a0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 shutdown_connections 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f571c06f430 0x7f571c071940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5730162b10 0x7f5730162f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57301f9420 0x7f573015f330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 --2- 172.21.15.83:0/28333447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5718003a60 0x7f573014ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 >> 172.21.15.83:0/28333447 conn(0x7f573006cd90 msgr2=0x7f57300b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 shutdown_connections 2024-05-17T21:58:05.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.384+0000 7f5735572640 1 -- 172.21.15.83:0/28333447 wait complete. 2024-05-17T21:58:05.394 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T21:58:05.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:05.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 Processor -- start 2024-05-17T21:58:05.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 -- start start 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62c8073540 0x7f62c8160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c815c830 con 0x7f62c815d2f0 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c81608a0 con 0x7f62c815fe20 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ce5cc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f62c8160b80 con 0x7f62c8073540 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62c7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62c815d2f0 0x7f62c8159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49168/0 (socket says 172.21.15.83:49168) 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62c7fff640 1 -- 172.21.15.83:0/1146558791 learned_addr learned my addr 172.21.15.83:0/1146558791 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ccb69640 1 --2- 172.21.15.83:0/1146558791 >> v2:172.21.15.83:3300/0 conn(0x7f62c8073540 0x7f62c8160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62c6ffd640 1 -- 172.21.15.83:0/1146558791 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404139985 0 0) 0x7f62c815c830 con 0x7f62c815d2f0 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62c6ffd640 1 -- 172.21.15.83:0/1146558791 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62b0003530 con 0x7f62c815d2f0 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ccb69640 1 -- 172.21.15.83:0/1146558791 >> v1:172.21.15.155:6789/0 conn(0x7f62c815fe20 legacy=0x7f62c8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ccb69640 1 -- 172.21.15.83:0/1146558791 >> v1:172.21.15.83:6789/0 conn(0x7f62c815d2f0 legacy=0x7f62c8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.536+0000 7f62ccb69640 1 -- 172.21.15.83:0/1146558791 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c815caf0 con 0x7f62c8073540 2024-05-17T21:58:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ccb69640 1 --2- 172.21.15.83:0/1146558791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c8073540 0x7f62c8160230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009d90 tx=0x7f62c001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a28f0eaa31f586c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62c6ffd640 1 -- 172.21.15.83:0/1146558791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00b2070 con 0x7f62c8073540 2024-05-17T21:58:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62c6ffd640 1 -- 172.21.15.83:0/1146558791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c00a5830 con 0x7f62c8073540 2024-05-17T21:58:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62c6ffd640 1 -- 172.21.15.83:0/1146558791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a5c00 con 0x7f62c8073540 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- 172.21.15.83:0/1146558791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c8073540 msgr2=0x7f62c8160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/1146558791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c8073540 0x7f62c8160230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009d90 tx=0x7f62c001c8e0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- 172.21.15.83:0/1146558791 shutdown_connections 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/1146558791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c8073540 0x7f62c8160230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- 172.21.15.83:0/1146558791 >> 172.21.15.83:0/1146558791 conn(0x7f62c806cd90 msgr2=0x7f62c80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- 172.21.15.83:0/1146558791 shutdown_connections 2024-05-17T21:58:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- 172.21.15.83:0/1146558791 wait complete. 2024-05-17T21:58:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 Processor -- start 2024-05-17T21:58:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.540+0000 7f62ce5cc640 1 -- start start 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c815fe20 0x7f62c81f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c70 0x7f62c81f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62c80b0700 con 0x7f62c815d2f0 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62c80b0580 con 0x7f62c80d1c70 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62c80b0880 con 0x7f62c815fe20 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60748/0 (socket says 172.21.15.83:60748) 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 -- 172.21.15.83:0/285721715 learned_addr learned my addr 172.21.15.83:0/285721715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ccb69640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c70 0x7f62c81f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 -- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c815fe20 msgr2=0x7f62c81f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c77fe640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c815fe20 0x7f62c81f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c815fe20 0x7f62c81f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 -- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c70 msgr2=0x7f62c81f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c70 0x7f62c81f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 -- 172.21.15.83:0/285721715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c8144990 con 0x7f62c815d2f0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c7fff640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c460 tx=0x7f62b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8017070 con 0x7f62c815d2f0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b8005cf0 con 0x7f62c815d2f0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c80b2e40 con 0x7f62c815d2f0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b80072d0 con 0x7f62c815d2f0 2024-05-17T21:58:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c8144ff0 con 0x7f62c815d2f0 2024-05-17T21:58:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c8160230 con 0x7f62c815d2f0 2024-05-17T21:58:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f62b801a070 con 0x7f62c815d2f0 2024-05-17T21:58:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 0x7f62b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62b808e540 con 0x7f62c815d2f0 2024-05-17T21:58:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.544+0000 7f62c77fe640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 0x7f62b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.548+0000 7f62c77fe640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 0x7f62b4071940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62c8154700 tx=0x7f62bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.548+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f62b805ae70 con 0x7f62c815d2f0 2024-05-17T21:58:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.744+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c8154450 con 0x7f62c815d2f0 2024-05-17T21:58:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.744+0000 7f62c4ff9640 1 -- 172.21.15.83:0/285721715 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f62b805f170 con 0x7f62c815d2f0 2024-05-17T21:58:05.738 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:05.738 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:58:05.739 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:58:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 msgr2=0x7f62b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 0x7f62b4071940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f62c8154700 tx=0x7f62bc008040 comp rx=0 tx=0).stop 2024-05-17T21:58:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 msgr2=0x7f62c80d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c460 tx=0x7f62b800c920 comp rx=0 tx=0).stop 2024-05-17T21:58:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 shutdown_connections 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f62b406f430 0x7f62b4071940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62c80d1c70 0x7f62c81f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62c815fe20 0x7f62c81f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 --2- 172.21.15.83:0/285721715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62c815d2f0 0x7f62c80d1730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 >> 172.21.15.83:0/285721715 conn(0x7f62c806cd90 msgr2=0x7f62c80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 shutdown_connections 2024-05-17T21:58:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.748+0000 7f62ce5cc640 1 -- 172.21.15.83:0/285721715 wait complete. 2024-05-17T21:58:05.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:05.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.900+0000 7ff670bdb640 1 Processor -- start 2024-05-17T21:58:05.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.900+0000 7ff670bdb640 1 -- start start 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff670bdb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff66c073540 0x7ff66c168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff670bdb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66c15e790 con 0x7ff66c0d1040 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff670bdb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66c15ea40 con 0x7ff66c144080 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff670bdb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c15ed20 con 0x7ff66c073540 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66ad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff66c073540 0x7ff66c168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66a59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff66c0d1040 0x7ff66c15e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49178/0 (socket says 172.21.15.83:49178) 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66a59c640 1 -- 172.21.15.83:0/842408951 learned_addr learned my addr 172.21.15.83:0/842408951 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66ad9d640 1 -- 172.21.15.83:0/842408951 >> v1:172.21.15.155:6789/0 conn(0x7ff66c144080 legacy=0x7ff66c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66ad9d640 1 -- 172.21.15.83:0/842408951 >> v1:172.21.15.83:6789/0 conn(0x7ff66c0d1040 legacy=0x7ff66c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66ad9d640 1 -- 172.21.15.83:0/842408951 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c15f000 con 0x7ff66c073540 2024-05-17T21:58:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/842408951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c073540 0x7ff66c168840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6640097c0 tx=0x7ff66401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc35658a3a9f6be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66959a640 1 -- 172.21.15.83:0/842408951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640a7070 con 0x7ff66c073540 2024-05-17T21:58:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66959a640 1 -- 172.21.15.83:0/842408951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6640b0070 con 0x7ff66c073540 2024-05-17T21:58:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.904+0000 7ff66959a640 1 -- 172.21.15.83:0/842408951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640ab890 con 0x7ff66c073540 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- 172.21.15.83:0/842408951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c073540 msgr2=0x7ff66c168840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 --2- 172.21.15.83:0/842408951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c073540 0x7ff66c168840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff6640097c0 tx=0x7ff66401c470 comp rx=0 tx=0).stop 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- 172.21.15.83:0/842408951 shutdown_connections 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 --2- 172.21.15.83:0/842408951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c073540 0x7ff66c168840 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- 172.21.15.83:0/842408951 >> 172.21.15.83:0/842408951 conn(0x7ff66c06cd90 msgr2=0x7ff66c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- 172.21.15.83:0/842408951 shutdown_connections 2024-05-17T21:58:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- 172.21.15.83:0/842408951 wait complete. 2024-05-17T21:58:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 Processor -- start 2024-05-17T21:58:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- start start 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 0x7ff66c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 0x7ff66c142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c16b720 con 0x7ff66c144080 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c16b5a0 con 0x7ff66c0d1040 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff670bdb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c16b8a0 con 0x7ff66c0ac2b0 2024-05-17T21:58:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff669d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff669d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60762/0 (socket says 172.21.15.83:60762) 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.908+0000 7ff669d9b640 1 -- 172.21.15.83:0/3330956959 learned_addr learned my addr 172.21.15.83:0/3330956959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 0x7ff66c142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66a59c640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 0x7ff66c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 -- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 msgr2=0x7ff66c142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 0x7ff66c142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 -- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 msgr2=0x7ff66c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 0x7ff66c15b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 -- 172.21.15.83:0/3330956959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c15b8e0 con 0x7ff66c144080 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66ad9d640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 0x7ff66c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66a59c640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 0x7ff66c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff669d9b640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff65400b9b0 tx=0x7ff65400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654019070 con 0x7ff66c144080 2024-05-17T21:58:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65400cba0 con 0x7ff66c144080 2024-05-17T21:58:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c141fd0 con 0x7ff66c144080 2024-05-17T21:58:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654014440 con 0x7ff66c144080 2024-05-17T21:58:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c142590 con 0x7ff66c144080 2024-05-17T21:58:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c141d60 con 0x7ff66c144080 2024-05-17T21:58:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff65401c070 con 0x7ff66c144080 2024-05-17T21:58:05.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 0x7ff64c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:05.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66a59c640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 0x7ff64c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:05.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff65408ee70 con 0x7ff66c144080 2024-05-17T21:58:05.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.912+0000 7ff66a59c640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 0x7ff64c071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f70 tx=0x7ff65c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:05.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:05.916+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff65405b7a0 con 0x7ff66c144080 2024-05-17T21:58:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.104+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff66c15e080 con 0x7ff66c144080 2024-05-17T21:58:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.104+0000 7ff662ffd640 1 -- 172.21.15.83:0/3330956959 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7ff65405faa0 con 0x7ff66c144080 2024-05-17T21:58:06.098 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:06.098 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/565292890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":565292890},{"type":"v1","addr":"172.21.15.155:6833","nonce":565292890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3568210423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3568210423},{"type":"v1","addr":"172.21.15.155:6835","nonce":3568210423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3211945611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3211945611},{"type":"v1","addr":"172.21.15.83:6835","nonce":3211945611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:57:29.658174+0000","modified":"2024-05-17T21:57:32.679022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4269":{"gid":4269,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/4099288168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4099288168},{"type":"v1","addr":"172.21.15.83:6837","nonce":4099288168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4269,"qdb_cluster":[4269]},"id":1}]} 2024-05-17T21:58:06.098 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 msgr2=0x7ff64c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 0x7ff64c071910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff65c000f70 tx=0x7ff65c009040 comp rx=0 tx=0).stop 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 msgr2=0x7ff66c0abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff65400b9b0 tx=0x7ff65400be70 comp rx=0 tx=0).stop 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 shutdown_connections 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff66c0ac2b0 0x7ff66c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff66c144080 0x7ff66c0abd70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff64c06f400 0x7ff64c071910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 --2- 172.21.15.83:0/3330956959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff66c0d1040 0x7ff66c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 >> 172.21.15.83:0/3330956959 conn(0x7ff66c06cd90 msgr2=0x7ff66c0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 shutdown_connections 2024-05-17T21:58:06.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.108+0000 7ff670bdb640 1 -- 172.21.15.83:0/3330956959 wait complete. 2024-05-17T21:58:06.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.264+0000 7f1139f93640 1 Processor -- start 2024-05-17T21:58:06.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.264+0000 7f1139f93640 1 -- start start 2024-05-17T21:58:06.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1139f93640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11341461c0 0x7f113415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1139f93640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11340706e0 0x7f11341605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1139f93640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11341465d0 con 0x7f113415e6b0 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1139f93640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f113406ef30 con 0x7f11341461c0 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1139f93640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f113406f210 con 0x7f11340706e0 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11341461c0 0x7f113415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11341461c0 0x7f113415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60778/0 (socket says 172.21.15.83:60778) 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 -- 172.21.15.83:0/1931397798 learned_addr learned my addr 172.21.15.83:0/1931397798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f1133fff640 1 --2- 172.21.15.83:0/1931397798 >> v2:172.21.15.155:3300/0 conn(0x7f11340706e0 0x7f11341605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 -- 172.21.15.83:0/1931397798 >> v2:172.21.15.155:3300/0 conn(0x7f11340706e0 msgr2=0x7f11341605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 --2- 172.21.15.83:0/1931397798 >> v2:172.21.15.155:3300/0 conn(0x7f11340706e0 0x7f11341605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 -- 172.21.15.83:0/1931397798 >> v1:172.21.15.155:6789/0 conn(0x7f113415e6b0 legacy=0x7f113406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.268+0000 7f11337fe640 1 -- 172.21.15.83:0/1931397798 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113406f4f0 con 0x7f11341461c0 2024-05-17T21:58:06.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f11337fe640 1 --2- 172.21.15.83:0/1931397798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11341461c0 0x7f113415e170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1134072c00 tx=0x7f112401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392823958fd087a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1132ffd640 1 -- 172.21.15.83:0/1931397798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240b4070 con 0x7f11341461c0 2024-05-17T21:58:06.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1132ffd640 1 -- 172.21.15.83:0/1931397798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11240a7b60 con 0x7f11341461c0 2024-05-17T21:58:06.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1132ffd640 1 -- 172.21.15.83:0/1931397798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240af440 con 0x7f11341461c0 2024-05-17T21:58:06.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- 172.21.15.83:0/1931397798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11341461c0 msgr2=0x7f113415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 --2- 172.21.15.83:0/1931397798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11341461c0 0x7f113415e170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1134072c00 tx=0x7f112401ebc0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- 172.21.15.83:0/1931397798 shutdown_connections 2024-05-17T21:58:06.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 --2- 172.21.15.83:0/1931397798 >> v2:172.21.15.155:3300/0 conn(0x7f11340706e0 0x7f11341605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 --2- 172.21.15.83:0/1931397798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11341461c0 0x7f113415e170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- 172.21.15.83:0/1931397798 >> 172.21.15.83:0/1931397798 conn(0x7f113406cd90 msgr2=0x7f11340b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- 172.21.15.83:0/1931397798 shutdown_connections 2024-05-17T21:58:06.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- 172.21.15.83:0/1931397798 wait complete. 2024-05-17T21:58:06.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 Processor -- start 2024-05-17T21:58:06.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.272+0000 7f1139f93640 1 -- start start 2024-05-17T21:58:06.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11341461c0 0x7f11340b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f113415e6b0 0x7f11340b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1134072e30 con 0x7f11340706e0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11340d12b0 con 0x7f11341461c0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1134072fb0 con 0x7f113415e6b0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60788/0 (socket says 172.21.15.83:60788) 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 -- 172.21.15.83:0/2384754545 learned_addr learned my addr 172.21.15.83:0/2384754545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f112affd640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11341461c0 0x7f11340b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 -- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f113415e6b0 msgr2=0x7f11340b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f113415e6b0 0x7f11340b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 -- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11341461c0 msgr2=0x7f11340b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11341461c0 0x7f11340b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 -- 172.21.15.83:0/2384754545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11340b6ae0 con 0x7f11340706e0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f11337fe640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f112401f460 tx=0x7f112401b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240b4040 con 0x7f11340706e0 2024-05-17T21:58:06.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11240bf070 con 0x7f11340706e0 2024-05-17T21:58:06.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11340b4670 con 0x7f11340706e0 2024-05-17T21:58:06.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240ba680 con 0x7f11340706e0 2024-05-17T21:58:06.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11340b4bf0 con 0x7f11340706e0 2024-05-17T21:58:06.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11341605a0 con 0x7f11340706e0 2024-05-17T21:58:06.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11240a7d10 con 0x7f11340706e0 2024-05-17T21:58:06.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 0x7f1114071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f112affd640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 0x7f1114071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f112412c500 con 0x7f11340706e0 2024-05-17T21:58:06.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.276+0000 7f112affd640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 0x7f1114071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1134139820 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.280+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f11340b4670 con 0x7f11340706e0 2024-05-17T21:58:06.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.432+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11340b41a0 con 0x7f11340706e0 2024-05-17T21:58:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.432+0000 7f1130ff9640 1 -- 172.21.15.83:0/2384754545 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16581 (secure 0 0 0) 0x7f11240187e0 con 0x7f11340706e0 2024-05-17T21:58:06.426 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:06.427 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:57:31.253252+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:57:28.098831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-17T21:57:28.618308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 msgr2=0x7f1114071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 0x7f1114071910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1134139820 tx=0x7f111c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 msgr2=0x7f1134156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f112401f460 tx=0x7f112401b960 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 shutdown_connections 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f113415e6b0 0x7f11340b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f111406f400 0x7f1114071910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11341461c0 0x7f11340b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 --2- 172.21.15.83:0/2384754545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11340706e0 0x7f1134156ea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 >> 172.21.15.83:0/2384754545 conn(0x7f113406cd90 msgr2=0x7f113413bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 shutdown_connections 2024-05-17T21:58:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.436+0000 7f1139f93640 1 -- 172.21.15.83:0/2384754545 wait complete. 2024-05-17T21:58:06.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T21:58:06.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 Processor -- start 2024-05-17T21:58:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 -- start start 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c073590 0x7ffb5c06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb5c06ff60 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb5c070240 con 0x7ffb5c0b0910 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb63bd0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5c0cf430 con 0x7ffb5c073590 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffb5c15c1a0 0x7ffb5c072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49192/0 (socket says 172.21.15.83:49192) 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 -- 172.21.15.83:0/3286802775 learned_addr learned my addr 172.21.15.83:0/3286802775 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207204004 0 0) 0x7ffb5c06ff60 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6116b640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c073590 0x7ffb5c06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb48004640 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473193506 0 0) 0x7ffb48004640 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c073590 msgr2=0x7ffb5c06f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c073590 0x7ffb5c06f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 >> v1:172.21.15.155:6789/0 conn(0x7ffb5c0b0910 legacy=0x7ffb5c0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c0cf710 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ffb58001700 con 0x7ffb5c15c1a0 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 >> v1:172.21.15.83:6789/0 conn(0x7ffb5c15c1a0 legacy=0x7ffb5c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb48004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb48005920 0x7ffb48005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb48006390 0x7ffb4800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c0cf430 con 0x7ffb48004b50 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4800d0d0 con 0x7ffb48005920 2024-05-17T21:58:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4800d2d0 con 0x7ffb48006390 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb48004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 -- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb48006390 msgr2=0x7ffb4800cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb48006390 0x7ffb4800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 -- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb48005920 msgr2=0x7ffb48005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb48005920 0x7ffb48005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.592+0000 7ffb6196c640 1 -- 172.21.15.83:0/3286802775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb48004640 con 0x7ffb48004b50 2024-05-17T21:58:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb6196c640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb48004fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c240 tx=0x7ffb5801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c609124462fb6f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58007d70 con 0x7ffb48004b50 2024-05-17T21:58:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb580a9020 con 0x7ffb48004b50 2024-05-17T21:58:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb6096a640 1 -- 172.21.15.83:0/3286802775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb580a5ba0 con 0x7ffb48004b50 2024-05-17T21:58:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 -- 172.21.15.83:0/3286802775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 msgr2=0x7ffb48004fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/3286802775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb48004fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c240 tx=0x7ffb5801cbc0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb6216d640 1 -- 172.21.15.83:0/3286802775 reap_dead start 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 -- 172.21.15.83:0/3286802775 shutdown_connections 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 -- 172.21.15.83:0/3286802775 >> 172.21.15.83:0/3286802775 conn(0x7ffb5c06cd90 msgr2=0x7ffb5c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 -- 172.21.15.83:0/3286802775 shutdown_connections 2024-05-17T21:58:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.596+0000 7ffb63bd0640 1 -- 172.21.15.83:0/3286802775 wait complete. 2024-05-17T21:58:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 Processor -- start 2024-05-17T21:58:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- start start 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c15a3f0 0x7ffb5c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 0x7ffb5c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c06f8d0 con 0x7ffb48004b50 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c0737a0 con 0x7ffb5c1585a0 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5c06fa50 con 0x7ffb5c15a3f0 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60800/0 (socket says 172.21.15.83:60800) 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 -- 172.21.15.83:0/781454392 learned_addr learned my addr 172.21.15.83:0/781454392 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6116b640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c15a3f0 0x7ffb5c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6216d640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 0x7ffb5c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 -- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c15a3f0 msgr2=0x7ffb5c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c15a3f0 0x7ffb5c15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 -- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 msgr2=0x7ffb5c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 0x7ffb5c1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 -- 172.21.15.83:0/781454392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c1526c0 con 0x7ffb48004b50 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6216d640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 0x7ffb5c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb6196c640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ab040 tx=0x7ffb48006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58001500 con 0x7ffb48004b50 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb580016a0 con 0x7ffb48004b50 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c15b0b0 con 0x7ffb48004b50 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004a10 con 0x7ffb48004b50 2024-05-17T21:58:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.600+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c15b670 con 0x7ffb48004b50 2024-05-17T21:58:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb5c072e80 con 0x7ffb48004b50 2024-05-17T21:58:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffb58004bb0 con 0x7ffb48004b50 2024-05-17T21:58:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb527fc640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 0x7ffb40071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb58129990 con 0x7ffb48004b50 2024-05-17T21:58:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb6116b640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 0x7ffb40071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.604+0000 7ffb6116b640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 0x7ffb40071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0090e0 tx=0x7ffb4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.608+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffb580c1030 con 0x7ffb48004b50 2024-05-17T21:58:06.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.788+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffb5c152110 con 0x7ffb48004b50 2024-05-17T21:58:06.783 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T21:58:06.788+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:58:06.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.792+0000 7ffb527fc640 1 -- 172.21.15.83:0/781454392 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7ffb5c1526c0 con 0x7ffb48004b50 2024-05-17T21:58:06.785 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T21:58:06.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 msgr2=0x7ffb40071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 0x7ffb40071940 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0090e0 tx=0x7ffb4c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:06.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 msgr2=0x7ffb5c151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb580ab040 tx=0x7ffb48006950 comp rx=0 tx=0).stop 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 shutdown_connections 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb4006f430 0x7ffb40071940 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb5c1585a0 0x7ffb5c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5c15a3f0 0x7ffb5c15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 --2- 172.21.15.83:0/781454392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb48004b50 0x7ffb5c151bd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 >> 172.21.15.83:0/781454392 conn(0x7ffb5c06cd90 msgr2=0x7ffb5c0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 shutdown_connections 2024-05-17T21:58:06.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.796+0000 7ffb63bd0640 1 -- 172.21.15.83:0/781454392 wait complete. 2024-05-17T21:58:06.792 INFO:tasks.ceph.mds.a.smithi083.stderr: -2369> 2024-05-17T21:57:35.773+0000 7f5dfd612640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:58:06.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T21:58:06.865 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:58:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.956+0000 7f742f513640 1 Processor -- start 2024-05-17T21:58:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.956+0000 7f742f513640 1 -- start start 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742f513640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f742815e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742f513640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742809bf60 con 0x7f742806cea0 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742f513640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742815e170 con 0x7f74280af450 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742f513640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74280b5090 con 0x7f74280b3bc0 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742dab0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f742806cea0 0x7f742815bba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49214/0 (socket says 172.21.15.83:49214) 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742dab0640 1 -- 172.21.15.83:0/3243080342 learned_addr learned my addr 172.21.15.83:0/3243080342 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:06.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742d2af640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f742815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045638534 0 0) 0x7f742809bf60 con 0x7f742806cea0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7414003530 con 0x7f742806cea0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3349985185 0 0) 0x7f7414003530 con 0x7f742806cea0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 msgr2=0x7f742815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f742815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 >> v1:172.21.15.155:6789/0 conn(0x7f74280af450 legacy=0x7f742806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74280b4ad0 con 0x7f742806cea0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f7420001a80 con 0x7f742806cea0 2024-05-17T21:58:06.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 >> v1:172.21.15.83:6789/0 conn(0x7f742806cea0 legacy=0x7f742815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7414003a40 0x7f7414003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f7414004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f74140052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b5090 con 0x7f7414004810 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f741400bb90 con 0x7f7414003a40 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74140101a0 con 0x7f74280b3bc0 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f7414004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742d2af640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7414003a40 0x7f7414003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742dab0640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f74140052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 -- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 msgr2=0x7f74140052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74280b3bc0 0x7f74140052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 -- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7414003a40 msgr2=0x7f7414003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7414003a40 0x7f7414003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742caae640 1 -- 172.21.15.83:0/3243080342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7414003530 con 0x7f7414004810 2024-05-17T21:58:06.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.960+0000 7f742d2af640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7414003a40 0x7f7414003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742caae640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f7414004c20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7418001730 tx=0x7f741801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a30fd15ea3df server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418001380 con 0x7f7414004810 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74180052d0 con 0x7f7414004810 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f741ffff640 1 -- 172.21.15.83:0/3243080342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418003840 con 0x7f7414004810 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- 172.21.15.83:0/3243080342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 msgr2=0x7f7414004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 --2- 172.21.15.83:0/3243080342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f7414004c20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7418001730 tx=0x7f741801c3e0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742dab0640 1 -- 172.21.15.83:0/3243080342 reap_dead start 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- 172.21.15.83:0/3243080342 shutdown_connections 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- 172.21.15.83:0/3243080342 >> 172.21.15.83:0/3243080342 conn(0x7f742809b8b0 msgr2=0x7f74280b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- 172.21.15.83:0/3243080342 shutdown_connections 2024-05-17T21:58:06.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- 172.21.15.83:0/3243080342 wait complete. 2024-05-17T21:58:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 Processor -- start 2024-05-17T21:58:06.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.964+0000 7f742f513640 1 -- start start 2024-05-17T21:58:06.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74281541a0 0x7f7428071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74280719c0 0x7f7428157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74280b3fd0 con 0x7f7414004810 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f742815ee70 con 0x7f74280719c0 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74280b4880 con 0x7f74281541a0 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37766/0 (socket says 172.21.15.83:37766) 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 -- 172.21.15.83:0/1500806015 learned_addr learned my addr 172.21.15.83:0/1500806015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742dab0640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74280719c0 0x7f7428157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742caae640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74281541a0 0x7f7428071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 -- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74281541a0 msgr2=0x7f7428071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74281541a0 0x7f7428071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 -- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74280719c0 msgr2=0x7f7428157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74280719c0 0x7f7428157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 -- 172.21.15.83:0/1500806015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7428071dd0 con 0x7f7414004810 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742d2af640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74240108d0 tx=0x7f7424010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742400f040 con 0x7f7414004810 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7424004220 con 0x7f7414004810 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7428158130 con 0x7f7414004810 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742400e960 con 0x7f7414004810 2024-05-17T21:58:06.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281f1f00 con 0x7f7414004810 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742815e930 con 0x7f7414004810 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.968+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f742400eb00 con 0x7f7414004810 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.972+0000 7f741dffb640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 0x7f740c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.972+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f742408faf0 con 0x7f7414004810 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.972+0000 7f742caae640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 0x7f740c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:06.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.972+0000 7f742caae640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 0x7f740c071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74180ac000 tx=0x7f74180011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:06.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:06.976+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7428071dd0 con 0x7f7414004810 2024-05-17T21:58:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.160+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f742806d680 con 0x7f7414004810 2024-05-17T21:58:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.799+0000 7f741dffb640 1 -- 172.21.15.83:0/1500806015 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f742408f0a0 con 0x7f7414004810 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 msgr2=0x7f740c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 0x7f740c071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74180ac000 tx=0x7f74180011a0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 msgr2=0x7f74281f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f74240108d0 tx=0x7f7424010d90 comp rx=0 tx=0).stop 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 shutdown_connections 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f740c06f400 0x7f740c071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74280719c0 0x7f7428157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74281541a0 0x7f7428071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 --2- 172.21.15.83:0/1500806015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7414004810 0x7f74281f5960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 >> 172.21.15.83:0/1500806015 conn(0x7f742809b8b0 msgr2=0x7f74280b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 shutdown_connections 2024-05-17T21:58:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.803+0000 7f742f513640 1 -- 172.21.15.83:0/1500806015 wait complete. 2024-05-17T21:58:07.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T21:58:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.979+0000 7f96b5d22640 1 Processor -- start 2024-05-17T21:58:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.979+0000 7f96b5d22640 1 -- start start 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96b5d22640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96b00b66c0 0x7f96b00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96b5d22640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b015ec60 con 0x7f96b015df30 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96b5d22640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b00b5be0 con 0x7f96b0072e50 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96b5d22640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f96b00b6230 con 0x7f96b00b66c0 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96af7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96b00b66c0 0x7f96b00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96affff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f96b015df30 0x7f96b015e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56464/0 (socket says 172.21.15.83:56464) 2024-05-17T21:58:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96affff640 1 -- 172.21.15.83:0/1509676324 learned_addr learned my addr 172.21.15.83:0/1509676324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96af7fe640 1 -- 172.21.15.83:0/1509676324 >> v1:172.21.15.155:6789/0 conn(0x7f96b0072e50 legacy=0x7f96b0073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96af7fe640 1 -- 172.21.15.83:0/1509676324 >> v1:172.21.15.83:6789/0 conn(0x7f96b015df30 legacy=0x7f96b015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96af7fe640 1 -- 172.21.15.83:0/1509676324 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00b5ec0 con 0x7f96b00b66c0 2024-05-17T21:58:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96aeffd640 1 -- 172.21.15.83:0/1509676324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385292851 0 0) 0x7f96b015ec60 con 0x7f96b015df30 2024-05-17T21:58:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96af7fe640 1 --2- 172.21.15.83:0/1509676324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b00b0bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097c0 tx=0x7f96a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a3c2cea608be98 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96aeffd640 1 -- 172.21.15.83:0/1509676324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00b1070 con 0x7f96b00b66c0 2024-05-17T21:58:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96aeffd640 1 -- 172.21.15.83:0/1509676324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a00ac410 con 0x7f96b00b66c0 2024-05-17T21:58:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.983+0000 7f96aeffd640 1 -- 172.21.15.83:0/1509676324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a00ac7e0 con 0x7f96b00b66c0 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- 172.21.15.83:0/1509676324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 msgr2=0x7f96b00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 --2- 172.21.15.83:0/1509676324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b00b0bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097c0 tx=0x7f96a001c4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- 172.21.15.83:0/1509676324 shutdown_connections 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 --2- 172.21.15.83:0/1509676324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b00b0bc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- 172.21.15.83:0/1509676324 >> 172.21.15.83:0/1509676324 conn(0x7f96b006cd90 msgr2=0x7f96b00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- 172.21.15.83:0/1509676324 shutdown_connections 2024-05-17T21:58:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- 172.21.15.83:0/1509676324 wait complete. 2024-05-17T21:58:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 Processor -- start 2024-05-17T21:58:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- start start 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b0072e50 0x7f96b0143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 0x7f96b015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00b5060 con 0x7f96b00b66c0 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00b4ee0 con 0x7f96b0072e50 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96b5d22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f96b00b51e0 con 0x7f96b015df30 2024-05-17T21:58:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37780/0 (socket says 172.21.15.83:37780) 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96a7fff640 1 -- 172.21.15.83:0/480245205 learned_addr learned my addr 172.21.15.83:0/480245205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.987+0000 7f96affff640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 0x7f96b015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96af7fe640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b0072e50 0x7f96b0143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 -- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 msgr2=0x7f96b015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 0x7f96b015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 -- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b0072e50 msgr2=0x7f96b0143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b0072e50 0x7f96b0143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 -- 172.21.15.83:0/480245205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0142c00 con 0x7f96b00b66c0 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96affff640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 0x7f96b015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96a7fff640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f969800b410 tx=0x7f969800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698015030 con 0x7f96b00b66c0 2024-05-17T21:58:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698004020 con 0x7f96b00b66c0 2024-05-17T21:58:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b0142ee0 con 0x7f96b00b66c0 2024-05-17T21:58:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698004410 con 0x7f96b00b66c0 2024-05-17T21:58:07.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b0159a70 con 0x7f96b00b66c0 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b00b0bc0 con 0x7f96b00b66c0 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9698018070 con 0x7f96b00b66c0 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 0x7f9690071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96af7fe640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 0x7f9690071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f969808ee30 con 0x7f96b00b66c0 2024-05-17T21:58:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.991+0000 7f96af7fe640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 0x7f9690071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96a00ae000 tx=0x7f96a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:07.995+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f969805bd50 con 0x7f96b00b66c0 2024-05-17T21:58:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:08.159+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96b0143b50 con 0x7f96b00b66c0 2024-05-17T21:58:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.079+0000 7f96acff9640 1 -- 172.21.15.83:0/480245205 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f969805b650 con 0x7f96b00b66c0 2024-05-17T21:58:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T21:58:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 msgr2=0x7f9690071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 0x7f9690071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96a00ae000 tx=0x7f96a0008040 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 msgr2=0x7f96b0156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f969800b410 tx=0x7f969800b8d0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 shutdown_connections 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b015df30 0x7f96b015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b00b66c0 0x7f96b0156f40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f969006f430 0x7f9690071940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 --2- 172.21.15.83:0/480245205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b0072e50 0x7f96b0143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 >> 172.21.15.83:0/480245205 conn(0x7f96b006cd90 msgr2=0x7f96b00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 shutdown_connections 2024-05-17T21:58:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.083+0000 7f96b5d22640 1 -- 172.21.15.83:0/480245205 wait complete. 2024-05-17T21:58:09.106 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T21:58:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.271+0000 7f40be84d640 1 Processor -- start 2024-05-17T21:58:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.271+0000 7f40be84d640 1 -- start start 2024-05-17T21:58:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40be84d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40b80b66c0 0x7f40b80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40be84d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40b8072e50 0x7f40b8073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40be84d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b80737a0 con 0x7f40b815df30 2024-05-17T21:58:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40be84d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f40b80b6230 con 0x7f40b8072e50 2024-05-17T21:58:09.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40be84d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f40b815ec60 con 0x7f40b80b66c0 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40b8072e50 0x7f40b8073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f40b8072e50 0x7f40b8073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37796/0 (socket says 172.21.15.83:37796) 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 -- 172.21.15.83:0/1987292711 learned_addr learned my addr 172.21.15.83:0/1987292711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b7fff640 1 --2- 172.21.15.83:0/1987292711 >> v2:172.21.15.155:3300/0 conn(0x7f40b80b66c0 0x7f40b80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 -- 172.21.15.83:0/1987292711 >> v2:172.21.15.155:3300/0 conn(0x7f40b80b66c0 msgr2=0x7f40b80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 --2- 172.21.15.83:0/1987292711 >> v2:172.21.15.155:3300/0 conn(0x7f40b80b66c0 0x7f40b80b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 -- 172.21.15.83:0/1987292711 >> v1:172.21.15.155:6789/0 conn(0x7f40b815df30 legacy=0x7f40b815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 -- 172.21.15.83:0/1987292711 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80b5be0 con 0x7f40b8072e50 2024-05-17T21:58:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.275+0000 7f40b77fe640 1 --2- 172.21.15.83:0/1987292711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8073260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40b806d690 tx=0x7f40ac01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13678b639875d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40b6ffd640 1 -- 172.21.15.83:0/1987292711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0b0070 con 0x7f40b8072e50 2024-05-17T21:58:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40b6ffd640 1 -- 172.21.15.83:0/1987292711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac0ab600 con 0x7f40b8072e50 2024-05-17T21:58:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40b6ffd640 1 -- 172.21.15.83:0/1987292711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0ab9d0 con 0x7f40b8072e50 2024-05-17T21:58:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- 172.21.15.83:0/1987292711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 msgr2=0x7f40b8073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 --2- 172.21.15.83:0/1987292711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8073260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40b806d690 tx=0x7f40ac01c340 comp rx=0 tx=0).stop 2024-05-17T21:58:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- 172.21.15.83:0/1987292711 shutdown_connections 2024-05-17T21:58:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 --2- 172.21.15.83:0/1987292711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8073260 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 --2- 172.21.15.83:0/1987292711 >> v2:172.21.15.155:3300/0 conn(0x7f40b80b66c0 0x7f40b80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- 172.21.15.83:0/1987292711 >> 172.21.15.83:0/1987292711 conn(0x7f40b806cd90 msgr2=0x7f40b80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- 172.21.15.83:0/1987292711 shutdown_connections 2024-05-17T21:58:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- 172.21.15.83:0/1987292711 wait complete. 2024-05-17T21:58:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 Processor -- start 2024-05-17T21:58:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.279+0000 7f40be84d640 1 -- start start 2024-05-17T21:58:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40b80b66c0 0x7f40b813ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40b815df30 0x7f40b8146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80b5060 con 0x7f40b8072e50 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80b4ee0 con 0x7f40b815df30 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f40b80b51e0 con 0x7f40b80b66c0 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37806/0 (socket says 172.21.15.83:37806) 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 -- 172.21.15.83:0/1840213575 learned_addr learned my addr 172.21.15.83:0/1840213575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 -- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40b80b66c0 msgr2=0x7f40b813ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40b80b66c0 0x7f40b813ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 -- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40b815df30 msgr2=0x7f40b8146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40bcdea640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40b815df30 0x7f40b8146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40b815df30 0x7f40b8146160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 -- 172.21.15.83:0/1840213575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b8141b90 con 0x7f40b8072e50 2024-05-17T21:58:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b7fff640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d620 tx=0x7f40a800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800b620 con 0x7f40b8072e50 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8019070 con 0x7f40b8072e50 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800b9e0 con 0x7f40b8072e50 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b8142000 con 0x7f40b8072e50 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81425c0 con 0x7f40b8072e50 2024-05-17T21:58:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8073260 con 0x7f40b8072e50 2024-05-17T21:58:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.283+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40a800bb80 con 0x7f40b8072e50 2024-05-17T21:58:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.287+0000 7f40b4ff9640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 0x7f40a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.287+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7f40a808f600 con 0x7f40b8072e50 2024-05-17T21:58:09.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.287+0000 7f40b77fe640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 0x7f40a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:09.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.287+0000 7f40b77fe640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 0x7f40a4071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0ad000 tx=0x7f40ac004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.291+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f40a805c650 con 0x7f40b8072e50 2024-05-17T21:58:09.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:09.463+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f40b8158560 con 0x7f40b8072e50 2024-05-17T21:58:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.083+0000 7f40b4ff9640 1 -- 172.21.15.83:0/1840213575 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f40a805bf50 con 0x7f40b8072e50 2024-05-17T21:58:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 msgr2=0x7f40a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 0x7f40a4071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0ad000 tx=0x7f40ac004740 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 msgr2=0x7f40b8143990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d620 tx=0x7f40a800dae0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 shutdown_connections 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f40a406f400 0x7f40a4071910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40b815df30 0x7f40b8146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40b80b66c0 0x7f40b813ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 --2- 172.21.15.83:0/1840213575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40b8072e50 0x7f40b8143990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 >> 172.21.15.83:0/1840213575 conn(0x7f40b806cd90 msgr2=0x7f40b8140db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:10.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 shutdown_connections 2024-05-17T21:58:10.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.087+0000 7f40be84d640 1 -- 172.21.15.83:0/1840213575 wait complete. 2024-05-17T21:58:10.109 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T21:58:10.109 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T21:58:10.109 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T21:58:10.109 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:10.109 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T21:58:10.109 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T21:58:10.109 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T21:58:10.109 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:10.110 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T21:58:10.110 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T21:58:10.110 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T21:58:10.110 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:10.110 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T21:58:10.110 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T21:58:10.110 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T21:58:10.111 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:10.126 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T21:58:10.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T21:58:10.127 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T21:58:10.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T21:58:10.138 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T21:58:10.138 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T21:58:10.140 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T21:58:10.141 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T21:58:10.169 INFO:tasks.ceph.mds.c:Started 2024-05-17T21:58:10.169 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:10.169 INFO:tasks.ceph.mds.a:Started 2024-05-17T21:58:10.170 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:10.187 INFO:tasks.ceph.mds.d:Started 2024-05-17T21:58:10.187 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:10.189 INFO:tasks.ceph.mds.b:Started 2024-05-17T21:58:10.189 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:10.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T21:58:10.267 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T21:58:10.268 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:58:10.288 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T21:58:10.288 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T21:58:10.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 Processor -- start 2024-05-17T21:58:10.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 -- start start 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af006c790 0x7f2af00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2af0157010 0x7f2af0161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af00af9c0 con 0x7f2af0156c00 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2af00afca0 con 0x7f2af0157010 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af76fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2af00aff80 con 0x7f2af006c790 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af5c9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2af0157010 0x7f2af0161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af5c9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2af0157010 0x7f2af0161460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37830/0 (socket says 172.21.15.83:37830) 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af5c9a640 1 -- 172.21.15.83:0/2760638716 learned_addr learned my addr 172.21.15.83:0/2760638716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.343+0000 7f2af5499640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af006c790 0x7f2af00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af5c9a640 1 -- 172.21.15.83:0/2760638716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af006c790 msgr2=0x7f2af00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af5c9a640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af006c790 0x7f2af00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af5c9a640 1 -- 172.21.15.83:0/2760638716 >> v1:172.21.15.155:6789/0 conn(0x7f2af0156c00 legacy=0x7f2af00b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af5c9a640 1 -- 172.21.15.83:0/2760638716 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af00b0260 con 0x7f2af0157010 2024-05-17T21:58:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af5c9a640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af0157010 0x7f2af0161460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009e80 tx=0x7f2ae801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62a1495d0b5cf78 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2ae7fff640 1 -- 172.21.15.83:0/2760638716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80a5840 con 0x7f2af0157010 2024-05-17T21:58:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2ae7fff640 1 -- 172.21.15.83:0/2760638716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae80a59e0 con 0x7f2af0157010 2024-05-17T21:58:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2ae7fff640 1 -- 172.21.15.83:0/2760638716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80a5db0 con 0x7f2af0157010 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 -- 172.21.15.83:0/2760638716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af0157010 msgr2=0x7f2af0161460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af0157010 0x7f2af0161460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009e80 tx=0x7f2ae801c9e0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 -- 172.21.15.83:0/2760638716 shutdown_connections 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af0157010 0x7f2af0161460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 --2- 172.21.15.83:0/2760638716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af006c790 0x7f2af00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 -- 172.21.15.83:0/2760638716 >> 172.21.15.83:0/2760638716 conn(0x7f2af009b8b0 msgr2=0x7f2af00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 -- 172.21.15.83:0/2760638716 shutdown_connections 2024-05-17T21:58:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.347+0000 7f2af76fd640 1 -- 172.21.15.83:0/2760638716 wait complete. 2024-05-17T21:58:10.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 Processor -- start 2024-05-17T21:58:10.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- start start 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0156c00 0x7f2af0149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af0157010 0x7f2af015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2af00d1820 con 0x7f2af006c790 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0161db0 con 0x7f2af0156c00 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2af00d19a0 con 0x7f2af0157010 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37838/0 (socket says 172.21.15.83:37838) 2024-05-17T21:58:10.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 -- 172.21.15.83:0/382307015 learned_addr learned my addr 172.21.15.83:0/382307015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5c9a640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af0157010 0x7f2af015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af4c98640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0156c00 0x7f2af0149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 -- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af0157010 msgr2=0x7f2af015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af0157010 0x7f2af015df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 -- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0156c00 msgr2=0x7f2af0149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0156c00 0x7f2af0149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 -- 172.21.15.83:0/382307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af01495d0 con 0x7f2af006c790 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af5499640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00d5f0 tx=0x7f2aec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0125f0 con 0x7f2af006c790 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec019070 con 0x7f2af006c790 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af015f620 con 0x7f2af006c790 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0129b0 con 0x7f2af006c790 2024-05-17T21:58:10.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.351+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af015fbe0 con 0x7f2af006c790 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af00af3e0 con 0x7f2af006c790 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2aec012b50 con 0x7f2af006c790 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2ae5ffb640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 0x7f2ad4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f2aec08f1d0 con 0x7f2af006c790 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2af4c98640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 0x7f2ad4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.355+0000 7f2af4c98640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 0x7f2ad4071940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2af00763a0 tx=0x7f2ae0009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.359+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2aec05be00 con 0x7f2af006c790 2024-05-17T21:58:10.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:10.531+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2af0074ba0 con 0x7f2af006c790 2024-05-17T21:58:11.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.119+0000 7f2ae5ffb640 1 -- 172.21.15.83:0/382307015 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f2aec08de20 con 0x7f2af006c790 2024-05-17T21:58:11.113 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 msgr2=0x7f2ad4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 0x7f2ad4071940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2af00763a0 tx=0x7f2ae0009060 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 msgr2=0x7f2af0155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00d5f0 tx=0x7f2aec00dab0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 shutdown_connections 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2af0157010 0x7f2af015df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ad406f430 0x7f2ad4071940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0156c00 0x7f2af0149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 --2- 172.21.15.83:0/382307015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2af006c790 0x7f2af0155fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 >> 172.21.15.83:0/382307015 conn(0x7f2af009b8b0 msgr2=0x7f2af014fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 shutdown_connections 2024-05-17T21:58:11.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.123+0000 7f2af76fd640 1 -- 172.21.15.83:0/382307015 wait complete. 2024-05-17T21:58:11.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T21:58:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 Processor -- start 2024-05-17T21:58:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 -- start start 2024-05-17T21:58:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7e0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd7e0073540 0x7fd7e0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e015c830 con 0x7fd7e015fe20 2024-05-17T21:58:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e01608a0 con 0x7fd7e0073540 2024-05-17T21:58:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e843c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e0160b80 con 0x7fd7e015d2f0 2024-05-17T21:58:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e59d7640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd7e015fe20 0x7fd7e0072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56476/0 (socket says 172.21.15.83:56476) 2024-05-17T21:58:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e59d7640 1 -- 172.21.15.83:0/3432345176 learned_addr learned my addr 172.21.15.83:0/3432345176 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7e0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e69d9640 1 --2- 172.21.15.83:0/3432345176 >> v2:172.21.15.155:3300/0 conn(0x7fd7e0073540 0x7fd7e0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667594771 0 0) 0x7fd7e015c830 con 0x7fd7e015fe20 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7cc003530 con 0x7fd7e015fe20 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2159332146 0 0) 0x7fd7cc003530 con 0x7fd7e015fe20 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 msgr2=0x7fd7e0159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7e0159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 >> v2:172.21.15.155:3300/0 conn(0x7fd7e0073540 msgr2=0x7fd7e0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 --2- 172.21.15.83:0/3432345176 >> v2:172.21.15.155:3300/0 conn(0x7fd7e0073540 0x7fd7e0160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e015caf0 con 0x7fd7e015fe20 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fd7d0001700 con 0x7fd7e015fe20 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 >> v1:172.21.15.83:6789/0 conn(0x7fd7e015fe20 legacy=0x7fd7e0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7cc003a10 0x7fd7cc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc004d50 0x7fd7cc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01608a0 con 0x7fd7cc003a10 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0160b80 con 0x7fd7cc004d50 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc00bb00 con 0x7fd7e015d2f0 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7cc003a10 0x7fd7cc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e59d7640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e69d9640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc004d50 0x7fd7cc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 -- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 msgr2=0x7fd7cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e015d2f0 0x7fd7cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 -- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc004d50 msgr2=0x7fd7cc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc004d50 0x7fd7cc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.299+0000 7fd7e61d8640 1 -- 172.21.15.83:0/3432345176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc003530 con 0x7fd7cc003a10 2024-05-17T21:58:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7cc003a10 0x7fd7cc003e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009ae0 tx=0x7fd7dc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ed9f618d42c828 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0b2070 con 0x7fd7cc003a10 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0a5550 con 0x7fd7cc003a10 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e51d6640 1 -- 172.21.15.83:0/3432345176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc0096b0 con 0x7fd7cc003a10 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 -- 172.21.15.83:0/3432345176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7cc003a10 msgr2=0x7fd7cc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 --2- 172.21.15.83:0/3432345176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7cc003a10 0x7fd7cc003e80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009ae0 tx=0x7fd7dc01ca70 comp rx=0 tx=0).stop 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e69d9640 1 -- 172.21.15.83:0/3432345176 reap_dead start 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 -- 172.21.15.83:0/3432345176 shutdown_connections 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 -- 172.21.15.83:0/3432345176 >> 172.21.15.83:0/3432345176 conn(0x7fd7e006cd90 msgr2=0x7fd7e00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 -- 172.21.15.83:0/3432345176 shutdown_connections 2024-05-17T21:58:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.303+0000 7fd7e843c640 1 -- 172.21.15.83:0/3432345176 wait complete. 2024-05-17T21:58:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 Processor -- start 2024-05-17T21:58:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- start start 2024-05-17T21:58:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc003a10 0x7fd7e01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e00b2e10 0x7fd7e0158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e00b3db0 con 0x7fd7e00cb840 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e00b3c30 con 0x7fd7cc003a10 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e00b3f30 con 0x7fd7e00b2e10 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37854/0 (socket says 172.21.15.83:37854) 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 -- 172.21.15.83:0/789959 learned_addr learned my addr 172.21.15.83:0/789959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e59d7640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e00b2e10 0x7fd7e0158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc003a10 0x7fd7e01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 -- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e00b2e10 msgr2=0x7fd7e0158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e00b2e10 0x7fd7e0158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 -- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc003a10 msgr2=0x7fd7e01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc003a10 0x7fd7e01562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 -- 172.21.15.83:0/789959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e0159870 con 0x7fd7e00cb840 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e69d9640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0072be0 tx=0x7fd7d80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8011db0 con 0x7fd7e00cb840 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7d801b070 con 0x7fd7e00cb840 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80039a0 con 0x7fd7e00cb840 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e00caa80 con 0x7fd7e00cb840 2024-05-17T21:58:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.307+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e00cb040 con 0x7fd7e00cb840 2024-05-17T21:58:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e0160230 con 0x7fd7e00cb840 2024-05-17T21:58:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7d8003b40 con 0x7fd7e00cb840 2024-05-17T21:58:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7d6ffd640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 0x7fd7c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 0x7fd7c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fd7d808f580 con 0x7fd7e00cb840 2024-05-17T21:58:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.311+0000 7fd7e61d8640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 0x7fd7c4071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0af000 tx=0x7fd7dc0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.315+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd7e0159870 con 0x7fd7e00cb840 2024-05-17T21:58:11.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.535+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd7e0073590 con 0x7fd7e00cb840 2024-05-17T21:58:11.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7d6ffd640 1 -- 172.21.15.83:0/789959 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fd7d805c850 con 0x7fd7e00cb840 2024-05-17T21:58:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:11.532 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:58:11.532 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:11.534 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.535 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.537 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.539 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T21:58:11.540 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:11.541 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.542 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:11.543 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mds", 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdmE/ECExAAK8YL5zM7adNLDZIj+8BFeA==", 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mds" 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mgr", 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdmhQQDExAAohaOMXBMTuimNVgZq0jaeQ==", 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.544 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-osd", 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdmxBQDExAAUcHjiWTwaaIQTbIzx1vkzw==", 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-osd" 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd", 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdmkCQDExAAplCn5Mwz/Qk7Y2WqLPB1gQ==", 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-17T21:58:11.545 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdm4zQDExAAqBaSdX2+ARye8Vb9YOhtCg==", 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rgw", 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCy0kdmi0UDExAAFk6ZDDVRyHKgS/FPyb2gNg==", 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-17T21:58:11.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:11.547 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T21:58:11.548 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 msgr2=0x7fd7c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 0x7fd7c4071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0af000 tx=0x7fd7dc0046d0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 msgr2=0x7fd7e01f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0072be0 tx=0x7fd7d80115a0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 shutdown_connections 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7e00cb840 0x7fd7e01f86d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e00b2e10 0x7fd7e0158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7c406f430 0x7fd7c4071940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 --2- 172.21.15.83:0/789959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7cc003a10 0x7fd7e01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 >> 172.21.15.83:0/789959 conn(0x7fd7e006cd90 msgr2=0x7fd7e0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 shutdown_connections 2024-05-17T21:58:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.539+0000 7fd7e843c640 1 -- 172.21.15.83:0/789959 wait complete. 2024-05-17T21:58:11.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-17T21:58:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.695+0000 7f0c29d0c640 1 Processor -- start 2024-05-17T21:58:11.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.695+0000 7f0c29d0c640 1 -- start start 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c29d0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c240b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c29d0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0c240cdf40 0x7f0c2415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c29d0c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c240b5640 con 0x7f0c2415d530 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c29d0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0c240b58e0 con 0x7f0c240cdf40 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c29d0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2406f060 con 0x7f0c240cf830 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c23fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0c2415d530 0x7f0c2406f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56480/0 (socket says 172.21.15.83:56480) 2024-05-17T21:58:11.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c23fff640 1 -- 172.21.15.83:0/302714357 learned_addr learned my addr 172.21.15.83:0/302714357 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:11.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c240b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244994009 0 0) 0x7f0c240b5640 con 0x7f0c2415d530 2024-05-17T21:58:11.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c10003530 con 0x7f0c2415d530 2024-05-17T21:58:11.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c22ffd640 1 --2- 172.21.15.83:0/302714357 >> v2:172.21.15.155:3300/0 conn(0x7f0c240cdf40 0x7f0c2415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1346891067 0 0) 0x7f0c10003530 con 0x7f0c2415d530 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 msgr2=0x7f0c240b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c240b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 >> v2:172.21.15.155:3300/0 conn(0x7f0c240cdf40 msgr2=0x7f0c2415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 --2- 172.21.15.83:0/302714357 >> v2:172.21.15.155:3300/0 conn(0x7f0c240cdf40 0x7f0c2415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2406f340 con 0x7f0c2415d530 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f0c14001a80 con 0x7f0c2415d530 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 >> v1:172.21.15.83:6789/0 conn(0x7f0c2415d530 legacy=0x7f0c2406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c10003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c10004d50 0x7f0c10005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c240b58e0 con 0x7f0c10003a10 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2406f060 con 0x7f0c10004d50 2024-05-17T21:58:11.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1000ba80 con 0x7f0c240cf830 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c10003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c22ffd640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 -- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 msgr2=0x7f0c10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c240cf830 0x7f0c10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 -- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c10004d50 msgr2=0x7f0c10005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c10004d50 0x7f0c10005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.699+0000 7f0c237fe640 1 -- 172.21.15.83:0/302714357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c10003530 con 0x7f0c10003a10 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c237fe640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c10003e80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ae0 tx=0x7f0c1c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1120cca0bc35324 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b2070 con 0x7f0c10003a10 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0a5570 con 0x7f0c10003a10 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c227fc640 1 -- 172.21.15.83:0/302714357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0096b0 con 0x7f0c10003a10 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- 172.21.15.83:0/302714357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 msgr2=0x7f0c10003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/302714357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c10003e80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ae0 tx=0x7f0c1c01ca90 comp rx=0 tx=0).stop 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c23fff640 1 -- 172.21.15.83:0/302714357 reap_dead start 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- 172.21.15.83:0/302714357 shutdown_connections 2024-05-17T21:58:11.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- 172.21.15.83:0/302714357 >> 172.21.15.83:0/302714357 conn(0x7f0c2406cd90 msgr2=0x7f0c240b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:11.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- 172.21.15.83:0/302714357 shutdown_connections 2024-05-17T21:58:11.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- 172.21.15.83:0/302714357 wait complete. 2024-05-17T21:58:11.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 Processor -- start 2024-05-17T21:58:11.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.703+0000 7f0c29d0c640 1 -- start start 2024-05-17T21:58:11.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c240b0aa0 0x7f0c24156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c241570d0 0x7f0c2415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24167560 con 0x7f0c10003a10 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c241673e0 con 0x7f0c240b0aa0 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c241676e0 con 0x7f0c241570d0 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37882/0 (socket says 172.21.15.83:37882) 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 -- 172.21.15.83:0/1022246673 learned_addr learned my addr 172.21.15.83:0/1022246673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c23fff640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c241570d0 0x7f0c2415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c22ffd640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c240b0aa0 0x7f0c24156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 -- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c241570d0 msgr2=0x7f0c2415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c241570d0 0x7f0c2415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 -- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c240b0aa0 msgr2=0x7f0c24156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c240b0aa0 0x7f0c24156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 -- 172.21.15.83:0/1022246673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c241574e0 con 0x7f0c10003a10 2024-05-17T21:58:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c237fe640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c01c8c0 tx=0x7f0c10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0bd070 con 0x7f0c10003a10 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c004030 con 0x7f0c10003a10 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c241428f0 con 0x7f0c10003a10 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b2040 con 0x7f0c10003a10 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2413fa80 con 0x7f0c10003a10 2024-05-17T21:58:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2415cff0 con 0x7f0c10003a10 2024-05-17T21:58:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c1c0c0070 con 0x7f0c10003a10 2024-05-17T21:58:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 0x7f0c04071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f0c1c1292e0 con 0x7f0c10003a10 2024-05-17T21:58:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.707+0000 7f0c22ffd640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 0x7f0c04071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:11.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.711+0000 7f0c22ffd640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 0x7f0c04071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c24163d50 tx=0x7f0c0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.711+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0c1c0c7020 con 0x7f0c10003a10 2024-05-17T21:58:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.919+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0c240ce620 con 0x7f0c10003a10 2024-05-17T21:58:11.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c1bfff640 1 -- 172.21.15.83:0/1022246673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f0c241574e0 con 0x7f0c10003a10 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 msgr2=0x7f0c04071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 0x7f0c04071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c24163d50 tx=0x7f0c0c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 msgr2=0x7f0c241423b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c01c8c0 tx=0x7f0c10005840 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 shutdown_connections 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c241570d0 0x7f0c2415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c0406f430 0x7f0c04071940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c240b0aa0 0x7f0c24156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 --2- 172.21.15.83:0/1022246673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c10003a10 0x7f0c241423b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:11.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 >> 172.21.15.83:0/1022246673 conn(0x7f0c2406cd90 msgr2=0x7f0c240b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:11.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 shutdown_connections 2024-05-17T21:58:11.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:11.923+0000 7f0c29d0c640 1 -- 172.21.15.83:0/1022246673 wait complete. 2024-05-17T21:58:11.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-17T21:58:12.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 Processor -- start 2024-05-17T21:58:12.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 -- start start 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b409bcc0 0x7fd4b40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4b406cea0 0x7fd4b40b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b40af470 con 0x7fd4b40b1f00 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b40af750 con 0x7fd4b406cea0 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4bace1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b40afa30 con 0x7fd4b409bcc0 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b927e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4b406cea0 0x7fd4b40b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd4b40b1f00 0x7fd4b406c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56486/0 (socket says 172.21.15.83:56486) 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b3fff640 1 -- 172.21.15.83:0/91270428 learned_addr learned my addr 172.21.15.83:0/91270428 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:12.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b8a7d640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b409bcc0 0x7fd4b40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b927e640 1 -- 172.21.15.83:0/91270428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b409bcc0 msgr2=0x7fd4b40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b927e640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b409bcc0 0x7fd4b40b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b927e640 1 -- 172.21.15.83:0/91270428 >> v1:172.21.15.83:6789/0 conn(0x7fd4b40b1f00 legacy=0x7fd4b406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.091+0000 7fd4b37fe640 1 -- 172.21.15.83:0/91270428 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213775766 0 0) 0x7fd4b40af470 con 0x7fd4b40b1f00 2024-05-17T21:58:12.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4b927e640 1 -- 172.21.15.83:0/91270428 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b40afd10 con 0x7fd4b406cea0 2024-05-17T21:58:12.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4b927e640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b406cea0 0x7fd4b40b0510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097c0 tx=0x7fd4a401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b8570b6fd693e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4b37fe640 1 -- 172.21.15.83:0/91270428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40b0070 con 0x7fd4b406cea0 2024-05-17T21:58:12.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4b37fe640 1 -- 172.21.15.83:0/91270428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a40ab400 con 0x7fd4b406cea0 2024-05-17T21:58:12.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4b37fe640 1 -- 172.21.15.83:0/91270428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40ab7d0 con 0x7fd4b406cea0 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 -- 172.21.15.83:0/91270428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b406cea0 msgr2=0x7fd4b40b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b406cea0 0x7fd4b40b0510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097c0 tx=0x7fd4a401c3e0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 -- 172.21.15.83:0/91270428 shutdown_connections 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b406cea0 0x7fd4b40b0510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 --2- 172.21.15.83:0/91270428 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b409bcc0 0x7fd4b40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 -- 172.21.15.83:0/91270428 >> 172.21.15.83:0/91270428 conn(0x7fd4b409b8b0 msgr2=0x7fd4b40b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 -- 172.21.15.83:0/91270428 shutdown_connections 2024-05-17T21:58:12.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.095+0000 7fd4bace1640 1 -- 172.21.15.83:0/91270428 wait complete. 2024-05-17T21:58:12.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 Processor -- start 2024-05-17T21:58:12.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- start start 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 0x7fd4b4161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b4157a00 0x7fd4b4155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4170da0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4165660 con 0x7fd4b409bcc0 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b4170f20 con 0x7fd4b4157a00 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37888/0 (socket says 172.21.15.83:37888) 2024-05-17T21:58:12.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 -- 172.21.15.83:0/811302085 learned_addr learned my addr 172.21.15.83:0/811302085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b8a7d640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 0x7fd4b4161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b927e640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b4157a00 0x7fd4b4155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 -- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b4157a00 msgr2=0x7fd4b4155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b4157a00 0x7fd4b4155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 -- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 msgr2=0x7fd4b4161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 0x7fd4b4161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 -- 172.21.15.83:0/811302085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b415b1b0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b8a7d640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 0x7fd4b4161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b3fff640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c390 tx=0x7fd49c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0074e0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c0047f0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b415b620 con 0x7fd4b40b1f00 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c007680 con 0x7fd4b40b1f00 2024-05-17T21:58:12.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.099+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b4161200 con 0x7fd4b40b1f00 2024-05-17T21:58:12.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b40b0510 con 0x7fd4b40b1f00 2024-05-17T21:58:12.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd49c011070 con 0x7fd4b40b1f00 2024-05-17T21:58:12.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4b17fa640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 0x7fd494071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fd49c08e9e0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4b8a7d640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 0x7fd494071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.103+0000 7fd4b8a7d640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 0x7fd494071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009f50 tx=0x7fd4ac00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.107+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd49c05bcb0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.331+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd4b415b380 con 0x7fd4b40b1f00 2024-05-17T21:58:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4b17fa640 1 -- 172.21.15.83:0/811302085 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fd49c05b5b0 con 0x7fd4b40b1f00 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 msgr2=0x7fd494071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 0x7fd494071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009f50 tx=0x7fd4ac00b040 comp rx=0 tx=0).stop 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 msgr2=0x7fd4b41447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00c390 tx=0x7fd49c00c850 comp rx=0 tx=0).stop 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 shutdown_connections 2024-05-17T21:58:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4b4157a00 0x7fd4b4155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4b40b1f00 0x7fd4b41447b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd49406f400 0x7fd494071910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 --2- 172.21.15.83:0/811302085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4b409bcc0 0x7fd4b4161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 >> 172.21.15.83:0/811302085 conn(0x7fd4b409b8b0 msgr2=0x7fd4b41582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 shutdown_connections 2024-05-17T21:58:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.335+0000 7fd4bace1640 1 -- 172.21.15.83:0/811302085 wait complete. 2024-05-17T21:58:12.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-17T21:58:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.503+0000 7f2800a5d640 1 Processor -- start 2024-05-17T21:58:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.503+0000 7f2800a5d640 1 -- start start 2024-05-17T21:58:12.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27fc0d17d0 0x7f27fc0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 0x7f27fc06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27fc0b5020 con 0x7f27fc0d2120 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc0cdee0 con 0x7f27fc0d17d0 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f27fc09cde0 con 0x7f27fc0cf380 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27fc0d17d0 0x7f27fc0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27f9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f27fc0d2120 0x7f27fc06d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56488/0 (socket says 172.21.15.83:56488) 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27fc0d17d0 0x7f27fc0d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37904/0 (socket says 172.21.15.83:37904) 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27f9d9b640 1 -- 172.21.15.83:0/3317898702 learned_addr learned my addr 172.21.15.83:0/3317898702 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fad9d640 1 --2- 172.21.15.83:0/3317898702 >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 0x7f27fc06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 -- 172.21.15.83:0/3317898702 >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 msgr2=0x7f27fc06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 --2- 172.21.15.83:0/3317898702 >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 0x7f27fc06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 -- 172.21.15.83:0/3317898702 >> v1:172.21.15.83:6789/0 conn(0x7f27fc0d2120 legacy=0x7f27fc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 -- 172.21.15.83:0/3317898702 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc09c540 con 0x7f27fc0d17d0 2024-05-17T21:58:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fad9d640 1 --2- 172.21.15.83:0/3317898702 >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 0x7f27fc06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:12.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27fa59c640 1 --2- 172.21.15.83:0/3317898702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0d17d0 0x7f27fc0d1be0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0cdcb0 tx=0x7f27f001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76df37fe162eeb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27f959a640 1 -- 172.21.15.83:0/3317898702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0d2740 con 0x7f27fc0d17d0 2024-05-17T21:58:12.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27f959a640 1 -- 172.21.15.83:0/3317898702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f00a7610 con 0x7f27fc0d17d0 2024-05-17T21:58:12.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f27f959a640 1 -- 172.21.15.83:0/3317898702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00a79e0 con 0x7f27fc0d17d0 2024-05-17T21:58:12.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 -- 172.21.15.83:0/3317898702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0d17d0 msgr2=0x7f27fc0d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3317898702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0d17d0 0x7f27fc0d1be0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0cdcb0 tx=0x7f27f001bed0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 -- 172.21.15.83:0/3317898702 shutdown_connections 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3317898702 >> v2:172.21.15.155:3300/0 conn(0x7f27fc0cf380 0x7f27fc06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.507+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3317898702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0d17d0 0x7f27fc0d1be0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- 172.21.15.83:0/3317898702 >> 172.21.15.83:0/3317898702 conn(0x7f27fc0d36d0 msgr2=0x7f27fc0d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- 172.21.15.83:0/3317898702 shutdown_connections 2024-05-17T21:58:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- 172.21.15.83:0/3317898702 wait complete. 2024-05-17T21:58:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 Processor -- start 2024-05-17T21:58:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- start start 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27fc0d17d0 0x7f27fc157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27fc0d2120 0x7f27fc157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc16d5a0 con 0x7f27fc0cf380 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc06eb10 con 0x7f27fc0d2120 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f2800a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc16d720 con 0x7f27fc0d17d0 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37918/0 (socket says 172.21.15.83:37918) 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 -- 172.21.15.83:0/3847998588 learned_addr learned my addr 172.21.15.83:0/3847998588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fad9d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27fc0d2120 0x7f27fc157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27f9d9b640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27fc0d17d0 0x7f27fc157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 -- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27fc0d17d0 msgr2=0x7f27fc157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27fc0d17d0 0x7f27fc157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 -- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27fc0d2120 msgr2=0x7f27fc157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27fc0d2120 0x7f27fc157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 -- 172.21.15.83:0/3847998588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc13b310 con 0x7f27fc0cf380 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.511+0000 7f27fa59c640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f27f001c3b0 tx=0x7f27f001a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00b8610 con 0x7f27fc0cf380 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f00b87b0 con 0x7f27fc0cf380 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc13b5f0 con 0x7f27fc0cf380 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00b8b60 con 0x7f27fc0cf380 2024-05-17T21:58:12.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc14b7a0 con 0x7f27fc0cf380 2024-05-17T21:58:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc0d1be0 con 0x7f27fc0cf380 2024-05-17T21:58:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f27f00b6070 con 0x7f27fc0cf380 2024-05-17T21:58:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 0x7f27dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f27f0128270 con 0x7f27fc0cf380 2024-05-17T21:58:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27f9d9b640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 0x7f27dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.515+0000 7f27f9d9b640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 0x7f27dc071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c10 tx=0x7f27ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.519+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f27f00be060 con 0x7f27fc0cf380 2024-05-17T21:58:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.723+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f27fc15d7b0 con 0x7f27fc0cf380 2024-05-17T21:58:12.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.731+0000 7f27eaffd640 1 -- 172.21.15.83:0/3847998588 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f27f00ab030 con 0x7f27fc0cf380 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 msgr2=0x7f27dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 0x7f27dc071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27ec000c10 tx=0x7f27ec009040 comp rx=0 tx=0).stop 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 msgr2=0x7f27fc14b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f27f001c3b0 tx=0x7f27f001a770 comp rx=0 tx=0).stop 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 shutdown_connections 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f27dc06f430 0x7f27dc071940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27fc0d2120 0x7f27fc157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27fc0d17d0 0x7f27fc157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 --2- 172.21.15.83:0/3847998588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27fc0cf380 0x7f27fc14b260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 >> 172.21.15.83:0/3847998588 conn(0x7f27fc0d36d0 msgr2=0x7f27fc076090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:12.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 shutdown_connections 2024-05-17T21:58:12.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.735+0000 7f2800a5d640 1 -- 172.21.15.83:0/3847998588 wait complete. 2024-05-17T21:58:12.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-17T21:58:12.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 Processor -- start 2024-05-17T21:58:12.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 -- start start 2024-05-17T21:58:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3af00af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3af00af8b0 0x7f3af006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af00ce450 con 0x7f3af00b4bb0 2024-05-17T21:58:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3af00ce730 con 0x7f3af00af8b0 2024-05-17T21:58:12.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af77d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3af00cea10 con 0x7f3af00b3be0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af4d72640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3af00b4bb0 0x7f3af00cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56492/0 (socket says 172.21.15.83:56492) 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af4d72640 1 -- 172.21.15.83:0/213415561 learned_addr learned my addr 172.21.15.83:0/213415561 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af5573640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3af00af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3af5d74640 1 --2- 172.21.15.83:0/213415561 >> v2:172.21.15.155:3300/0 conn(0x7f3af00af8b0 0x7f3af006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2917700345 0 0) 0x7f3af00ce450 con 0x7f3af00b4bb0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.899+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3adc003530 con 0x7f3af00b4bb0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2366440968 0 0) 0x7f3adc003530 con 0x7f3af00b4bb0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 msgr2=0x7f3af00af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3af00af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 >> v2:172.21.15.155:3300/0 conn(0x7f3af00af8b0 msgr2=0x7f3af006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 --2- 172.21.15.83:0/213415561 >> v2:172.21.15.155:3300/0 conn(0x7f3af00af8b0 0x7f3af006c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af00cecf0 con 0x7f3af00b4bb0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f3ae4001700 con 0x7f3af00b4bb0 2024-05-17T21:58:12.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 >> v1:172.21.15.83:6789/0 conn(0x7f3af00b4bb0 legacy=0x7f3af00cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3adc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3adc003fe0 0x7f3adc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc004d50 0x7f3adc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00ce730 con 0x7f3adc004d50 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00cea10 con 0x7f3adc003fe0 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc00bad0 con 0x7f3af00b3be0 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc004d50 0x7f3adc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af4d72640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3adc003fe0 0x7f3adc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5573640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3adc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 -- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 msgr2=0x7f3adc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3adc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 -- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3adc003fe0 msgr2=0x7f3adc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3adc003fe0 0x7f3adc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 -- 172.21.15.83:0/213415561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc003530 con 0x7f3adc004d50 2024-05-17T21:58:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5573640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af00b3be0 0x7f3adc003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3af5d74640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc004d50 0x7f3adc005160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ab0 tx=0x7f3aec01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fab398c26306d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0b2070 con 0x7f3adc004d50 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec0a5640 con 0x7f3adc004d50 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.903+0000 7f3ae3fff640 1 -- 172.21.15.83:0/213415561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0096b0 con 0x7f3adc004d50 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- 172.21.15.83:0/213415561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc004d50 msgr2=0x7f3adc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 --2- 172.21.15.83:0/213415561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc004d50 0x7f3adc005160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ab0 tx=0x7f3aec01ca50 comp rx=0 tx=0).stop 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 -- 172.21.15.83:0/213415561 reap_dead start 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- 172.21.15.83:0/213415561 shutdown_connections 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- 172.21.15.83:0/213415561 >> 172.21.15.83:0/213415561 conn(0x7f3af00b4090 msgr2=0x7f3af00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- 172.21.15.83:0/213415561 shutdown_connections 2024-05-17T21:58:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- 172.21.15.83:0/213415561 wait complete. 2024-05-17T21:58:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 Processor -- start 2024-05-17T21:58:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- start start 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 0x7f3af0156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 0x7f3af015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00ce1c0 con 0x7f3af0160850 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00ce040 con 0x7f3af015a210 2024-05-17T21:58:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af77d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3af00cfc00 con 0x7f3af015f6e0 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37926/0 (socket says 172.21.15.83:37926) 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 -- 172.21.15.83:0/1949849830 learned_addr learned my addr 172.21.15.83:0/1949849830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5573640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 0x7f3af0156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af4d72640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 0x7f3af015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 -- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 msgr2=0x7f3af0156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 0x7f3af0156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 -- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 msgr2=0x7f3af015a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 0x7f3af015a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5d74640 1 -- 172.21.15.83:0/1949849830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af01611a0 con 0x7f3af0160850 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af5573640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 0x7f3af0156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.907+0000 7f3af4d72640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 0x7f3af015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af5d74640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ab0 tx=0x7f3adc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0b2040 con 0x7f3af0160850 2024-05-17T21:58:12.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec000e50 con 0x7f3af0160850 2024-05-17T21:58:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af015e640 con 0x7f3af0160850 2024-05-17T21:58:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0096b0 con 0x7f3af0160850 2024-05-17T21:58:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af015ec00 con 0x7f3af0160850 2024-05-17T21:58:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af00ce340 con 0x7f3af0160850 2024-05-17T21:58:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3aec004240 con 0x7f3af0160850 2024-05-17T21:58:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 0x7f3ad4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af5573640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 0x7f3ad4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f3aec128210 con 0x7f3af0160850 2024-05-17T21:58:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.911+0000 7f3af5573640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 0x7f3ad4071910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3af0158270 tx=0x7f3ae800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:12.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:12.915+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3af01611a0 con 0x7f3af0160850 2024-05-17T21:58:13.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.143+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3af00b7870 con 0x7f3af0160850 2024-05-17T21:58:13.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.147+0000 7f3ae1ffb640 1 -- 172.21.15.83:0/1949849830 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f3aec0167a0 con 0x7f3af0160850 2024-05-17T21:58:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 msgr2=0x7f3ad4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 0x7f3ad4071910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3af0158270 tx=0x7f3ae800d040 comp rx=0 tx=0).stop 2024-05-17T21:58:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 msgr2=0x7f3af0160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ab0 tx=0x7f3adc005840 comp rx=0 tx=0).stop 2024-05-17T21:58:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 shutdown_connections 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3af0160850 0x7f3af0160c60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ad406f400 0x7f3ad4071910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3af015a210 0x7f3af015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 --2- 172.21.15.83:0/1949849830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3af015f6e0 0x7f3af0156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 >> 172.21.15.83:0/1949849830 conn(0x7f3af00b4090 msgr2=0x7f3af00afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 shutdown_connections 2024-05-17T21:58:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.151+0000 7f3af77d7640 1 -- 172.21.15.83:0/1949849830 wait complete. 2024-05-17T21:58:13.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-17T21:58:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 Processor -- start 2024-05-17T21:58:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 -- start start 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f590815cb90 0x7f590815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080b65a0 con 0x7f590815d4e0 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080b5500 con 0x7f590815dcb0 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590e357640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f59080b6110 con 0x7f590815cb90 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f59077fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f590815cb90 0x7f590815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590c8f4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f590815d4e0 0x7f59080aff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56494/0 (socket says 172.21.15.83:56494) 2024-05-17T21:58:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f590c8f4640 1 -- 172.21.15.83:0/1275901155 learned_addr learned my addr 172.21.15.83:0/1275901155 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f59077fe640 1 -- 172.21.15.83:0/1275901155 >> v1:172.21.15.155:6789/0 conn(0x7f590815dcb0 legacy=0x7f59080b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f59077fe640 1 -- 172.21.15.83:0/1275901155 >> v1:172.21.15.83:6789/0 conn(0x7f590815d4e0 legacy=0x7f59080aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f59077fe640 1 -- 172.21.15.83:0/1275901155 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080b57e0 con 0x7f590815cb90 2024-05-17T21:58:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.307+0000 7f5906ffd640 1 -- 172.21.15.83:0/1275901155 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263568300 0 0) 0x7f59080b65a0 con 0x7f590815d4e0 2024-05-17T21:58:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f59077fe640 1 --2- 172.21.15.83:0/1275901155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f590815cfa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097c0 tx=0x7f58fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e72daa59348886 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f5906ffd640 1 -- 172.21.15.83:0/1275901155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0b2070 con 0x7f590815cb90 2024-05-17T21:58:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f5906ffd640 1 -- 172.21.15.83:0/1275901155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc0ad640 con 0x7f590815cb90 2024-05-17T21:58:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f5906ffd640 1 -- 172.21.15.83:0/1275901155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0ada10 con 0x7f590815cb90 2024-05-17T21:58:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 -- 172.21.15.83:0/1275901155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 msgr2=0x7f590815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 --2- 172.21.15.83:0/1275901155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f590815cfa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097c0 tx=0x7f58fc01c4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 -- 172.21.15.83:0/1275901155 shutdown_connections 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 --2- 172.21.15.83:0/1275901155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f590815cfa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 -- 172.21.15.83:0/1275901155 >> 172.21.15.83:0/1275901155 conn(0x7f590809b8b0 msgr2=0x7f59080b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 -- 172.21.15.83:0/1275901155 shutdown_connections 2024-05-17T21:58:13.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.311+0000 7f590e357640 1 -- 172.21.15.83:0/1275901155 wait complete. 2024-05-17T21:58:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 Processor -- start 2024-05-17T21:58:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- start start 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f590815d4e0 0x7f590806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 0x7f590815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59080b1020 con 0x7f590815cb90 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59080b0ea0 con 0x7f590815d4e0 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59080b11a0 con 0x7f590815dcb0 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37942/0 (socket says 172.21.15.83:37942) 2024-05-17T21:58:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 -- 172.21.15.83:0/3820712926 learned_addr learned my addr 172.21.15.83:0/3820712926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590c8f4640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 0x7f590815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f59077fe640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f590815d4e0 0x7f590806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 -- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 msgr2=0x7f590815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 0x7f590815bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 -- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f590815d4e0 msgr2=0x7f590806c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f590815d4e0 0x7f590806c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 -- 172.21.15.83:0/3820712926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5908152060 con 0x7f590815cb90 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590c8f4640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 0x7f590815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5907fff640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f590000b9f0 tx=0x7f590000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590000cbd0 con 0x7f590815cb90 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590000cd70 con 0x7f590815cb90 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5908152340 con 0x7f590815cb90 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900003bc0 con 0x7f590815cb90 2024-05-17T21:58:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.315+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081568f0 con 0x7f590815cb90 2024-05-17T21:58:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590815cfa0 con 0x7f590815cb90 2024-05-17T21:58:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5900006d10 con 0x7f590815cb90 2024-05-17T21:58:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f5904ff9640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 0x7f58f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f590008eba0 con 0x7f590815cb90 2024-05-17T21:58:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f59077fe640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 0x7f58f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.319+0000 7f59077fe640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 0x7f58f4071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0af000 tx=0x7f58fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.323+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f590005bd50 con 0x7f590815cb90 2024-05-17T21:58:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.527+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f59081588d0 con 0x7f590815cb90 2024-05-17T21:58:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.531+0000 7f5904ff9640 1 -- 172.21.15.83:0/3820712926 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f590005b650 con 0x7f590815cb90 2024-05-17T21:58:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 msgr2=0x7f58f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 0x7f58f4071910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0af000 tx=0x7f58fc008040 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 msgr2=0x7f5908167f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f590000b9f0 tx=0x7f590000beb0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 shutdown_connections 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f590815dcb0 0x7f590815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58f406f400 0x7f58f4071910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f590815d4e0 0x7f590806c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 --2- 172.21.15.83:0/3820712926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f590815cb90 0x7f5908167f90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 >> 172.21.15.83:0/3820712926 conn(0x7f590809b8b0 msgr2=0x7f5908158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 shutdown_connections 2024-05-17T21:58:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.535+0000 7f590e357640 1 -- 172.21.15.83:0/3820712926 wait complete. 2024-05-17T21:58:13.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-17T21:58:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 Processor -- start 2024-05-17T21:58:13.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 -- start start 2024-05-17T21:58:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57740af370 0x7f57740b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57740b58c0 con 0x7f57740b6ea0 2024-05-17T21:58:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f57740b5ba0 con 0x7f57740af370 2024-05-17T21:58:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f577b8f8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f57740b5e80 con 0x7f57740b5390 2024-05-17T21:58:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f5779694640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f5779694640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:57488/0 (socket says 172.21.15.83:57488) 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f5778e93640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57740af370 0x7f57740b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.687+0000 7f5779694640 1 -- 172.21.15.83:0/4291352804 learned_addr learned my addr 172.21.15.83:0/4291352804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f5779694640 1 -- 172.21.15.83:0/4291352804 >> v2:172.21.15.155:3300/0 conn(0x7f57740af370 msgr2=0x7f57740b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f5779694640 1 --2- 172.21.15.83:0/4291352804 >> v2:172.21.15.155:3300/0 conn(0x7f57740af370 0x7f57740b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f5779694640 1 -- 172.21.15.83:0/4291352804 >> v1:172.21.15.155:6789/0 conn(0x7f57740b6ea0 legacy=0x7f5774165720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f5779694640 1 -- 172.21.15.83:0/4291352804 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57740b6160 con 0x7f57740b5390 2024-05-17T21:58:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f5779694640 1 --2- 172.21.15.83:0/4291352804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5770009870 tx=0x7f577001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8aeb8bb4b0d29ec server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f576bfff640 1 -- 172.21.15.83:0/4291352804 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700a9640 con 0x7f57740b5390 2024-05-17T21:58:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f576bfff640 1 -- 172.21.15.83:0/4291352804 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57700a97e0 con 0x7f57740b5390 2024-05-17T21:58:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f576bfff640 1 -- 172.21.15.83:0/4291352804 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700a9bb0 con 0x7f57740b5390 2024-05-17T21:58:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 -- 172.21.15.83:0/4291352804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 msgr2=0x7f577415ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 --2- 172.21.15.83:0/4291352804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5770009870 tx=0x7f577001c560 comp rx=0 tx=0).stop 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 -- 172.21.15.83:0/4291352804 shutdown_connections 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 --2- 172.21.15.83:0/4291352804 >> v2:172.21.15.155:3300/0 conn(0x7f57740af370 0x7f57740b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 --2- 172.21.15.83:0/4291352804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740b5390 0x7f577415ca50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.691+0000 7f577b8f8640 1 -- 172.21.15.83:0/4291352804 >> 172.21.15.83:0/4291352804 conn(0x7f57740b4090 msgr2=0x7f57740b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- 172.21.15.83:0/4291352804 shutdown_connections 2024-05-17T21:58:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- 172.21.15.83:0/4291352804 wait complete. 2024-05-17T21:58:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 Processor -- start 2024-05-17T21:58:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- start start 2024-05-17T21:58:13.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740af370 0x7f57741517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57740b6ea0 0x7f5774155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5774167650 con 0x7f57740b5390 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57740b76c0 con 0x7f57740b6ea0 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.695+0000 7f577b8f8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f57741677d0 con 0x7f57740af370 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37944/0 (socket says 172.21.15.83:37944) 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 -- 172.21.15.83:0/2306691005 learned_addr learned my addr 172.21.15.83:0/2306691005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5779694640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740af370 0x7f57741517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5779e95640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57740b6ea0 0x7f5774155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 -- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740af370 msgr2=0x7f57741517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740af370 0x7f57741517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 -- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57740b6ea0 msgr2=0x7f5774155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57740b6ea0 0x7f5774155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 -- 172.21.15.83:0/2306691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577415c400 con 0x7f57740b5390 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5778e93640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f576400e8a0 tx=0x7f576400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764017070 con 0x7f57740b5390 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576400c940 con 0x7f57740b5390 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57740caf30 con 0x7f57740b5390 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576400ccf0 con 0x7f57740b5390 2024-05-17T21:58:13.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57740cb4f0 con 0x7f57740b5390 2024-05-17T21:58:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5774155a90 con 0x7f57740b5390 2024-05-17T21:58:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f576401a070 con 0x7f57740b5390 2024-05-17T21:58:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 0x7f5758071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f576408ee60 con 0x7f57740b5390 2024-05-17T21:58:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5779694640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 0x7f5758071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.699+0000 7f5779694640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 0x7f5758071940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57700ae000 tx=0x7f57700b3040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:13.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.703+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5774155a90 con 0x7f57740b5390 2024-05-17T21:58:13.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.907+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f5774167b80 con 0x7f57740b5390 2024-05-17T21:58:13.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.911+0000 7f5769ffb640 1 -- 172.21.15.83:0/2306691005 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f576405c0a0 con 0x7f57740b5390 2024-05-17T21:58:13.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 msgr2=0x7f5758071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 0x7f5758071940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f57700ae000 tx=0x7f57700b3040 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 msgr2=0x7f577415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f576400e8a0 tx=0x7f576400eca0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 shutdown_connections 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f575806f430 0x7f5758071940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57740b6ea0 0x7f5774155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57740b5390 0x7f577415bec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 --2- 172.21.15.83:0/2306691005 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57740af370 0x7f57741517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 >> 172.21.15.83:0/2306691005 conn(0x7f57740b4090 msgr2=0x7f577406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 shutdown_connections 2024-05-17T21:58:13.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:13.915+0000 7f577b8f8640 1 -- 172.21.15.83:0/2306691005 wait complete. 2024-05-17T21:58:13.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T21:58:14.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 Processor -- start 2024-05-17T21:58:14.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 -- start start 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e80072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e80073780 con 0x7f3e80073370 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e8015e790 con 0x7f3e800d1040 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e879d8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8015ea40 con 0x7f3e80144080 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85f75640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3e80073370 0x7f3e8006f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56502/0 (socket says 172.21.15.83:56502) 2024-05-17T21:58:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85f75640 1 -- 172.21.15.83:0/3894177676 learned_addr learned my addr 172.21.15.83:0/3894177676 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e84f73640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e80072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205043837 0 0) 0x7f3e78001120 con 0x7f3e80073370 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e68003530 con 0x7f3e80073370 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 206958681 0 0) 0x7f3e68003530 con 0x7f3e80073370 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 msgr2=0x7f3e80072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e80072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 >> v1:172.21.15.155:6789/0 conn(0x7f3e800d1040 legacy=0x7f3e8015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8015ed20 con 0x7f3e80073370 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f3e780018d0 con 0x7f3e80073370 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 >> v1:172.21.15.83:6789/0 conn(0x7f3e80073370 legacy=0x7f3e8006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68003a40 0x7f3e68003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68004d50 0x7f3e68005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8015ea40 con 0x7f3e68003a40 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6800bb90 con 0x7f3e68004d50 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6800bd90 con 0x7f3e80144080 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68003a40 0x7f3e68003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e84f73640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85f75640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68004d50 0x7f3e68005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 -- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 msgr2=0x7f3e68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e80144080 0x7f3e68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 -- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68004d50 msgr2=0x7f3e68005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68004d50 0x7f3e68005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85774640 1 -- 172.21.15.83:0/3894177676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e68003530 con 0x7f3e68003a40 2024-05-17T21:58:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.083+0000 7f3e85f75640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68004d50 0x7f3e68005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e85774640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68003a40 0x7f3e68003eb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001e90 tx=0x7f3e7001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583eb6599974326 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70001ae0 con 0x7f3e68003a40 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e70003b80 con 0x7f3e68003a40 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e7ffff640 1 -- 172.21.15.83:0/3894177676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700050d0 con 0x7f3e68003a40 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 -- 172.21.15.83:0/3894177676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68003a40 msgr2=0x7f3e68003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 --2- 172.21.15.83:0/3894177676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68003a40 0x7f3e68003eb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001e90 tx=0x7f3e7001c740 comp rx=0 tx=0).stop 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e85f75640 1 -- 172.21.15.83:0/3894177676 reap_dead start 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 -- 172.21.15.83:0/3894177676 shutdown_connections 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 -- 172.21.15.83:0/3894177676 >> 172.21.15.83:0/3894177676 conn(0x7f3e8006cd90 msgr2=0x7f3e800b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 -- 172.21.15.83:0/3894177676 shutdown_connections 2024-05-17T21:58:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.087+0000 7f3e879d8640 1 -- 172.21.15.83:0/3894177676 wait complete. 2024-05-17T21:58:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 Processor -- start 2024-05-17T21:58:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- start start 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68003a40 0x7f3e8015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e801ec780 0x7f3e8015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80070650 con 0x7f3e8015f780 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80144490 con 0x7f3e68003a40 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e800707d0 con 0x7f3e801ec780 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37970/0 (socket says 172.21.15.83:37970) 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 -- 172.21.15.83:0/2726119802 learned_addr learned my addr 172.21.15.83:0/2726119802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85774640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68003a40 0x7f3e8015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e84f73640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e801ec780 0x7f3e8015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 -- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e801ec780 msgr2=0x7f3e8015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e801ec780 0x7f3e8015f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 -- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68003a40 msgr2=0x7f3e8015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68003a40 0x7f3e8015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 -- 172.21.15.83:0/2726119802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e801ebff0 con 0x7f3e8015f780 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e85f75640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e800ceb70 tx=0x7f3e7800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78007440 con 0x7f3e8015f780 2024-05-17T21:58:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78007600 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78004a30 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e801623f0 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.091+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e80140070 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8013f100 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e7801b020 con 0x7f3e8015f780 2024-05-17T21:58:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e7dffb640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 0x7f3e6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e85774640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 0x7f3e6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f3e7808e910 con 0x7f3e8015f780 2024-05-17T21:58:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.095+0000 7f3e85774640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 0x7f3e6c071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e700ad000 tx=0x7f3e700aa040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.099+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3e80073780 con 0x7f3e8015f780 2024-05-17T21:58:14.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3e8006f730 con 0x7f3e8015f780 2024-05-17T21:58:14.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e7dffb640 1 -- 172.21.15.83:0/2726119802 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+65 (secure 0 0 0) 0x7f3e7805bbe0 con 0x7f3e8015f780 2024-05-17T21:58:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:14.245 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T21:58:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T21:58:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T21:58:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:14.246 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 msgr2=0x7f3e6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 0x7f3e6c071910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e700ad000 tx=0x7f3e700aa040 comp rx=0 tx=0).stop 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 msgr2=0x7f3e801fa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.251+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e800ceb70 tx=0x7f3e7800d9d0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 shutdown_connections 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e8015f780 0x7f3e801fa780 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e801ec780 0x7f3e8015f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e6c06f400 0x7f3e6c071910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 --2- 172.21.15.83:0/2726119802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e68003a40 0x7f3e8015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 >> 172.21.15.83:0/2726119802 conn(0x7f3e8006cd90 msgr2=0x7f3e80144610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 shutdown_connections 2024-05-17T21:58:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.255+0000 7f3e879d8640 1 -- 172.21.15.83:0/2726119802 wait complete. 2024-05-17T21:58:14.265 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T21:58:14.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T21:58:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 Processor -- start 2024-05-17T21:58:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 -- start start 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8e8160420 0x7ff8e8160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 0x7ff8e80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8e8160d70 con 0x7ff8e81570e0 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff8e80b4a90 con 0x7ff8e8160420 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8efa7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e80d0820 con 0x7ff8e80b4f20 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8ed017640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8e8160420 0x7ff8e8160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8ee019640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 0x7ff8e80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8ee019640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 0x7ff8e80b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:57530/0 (socket says 172.21.15.83:57530) 2024-05-17T21:58:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.411+0000 7ff8ee019640 1 -- 172.21.15.83:0/4266125963 learned_addr learned my addr 172.21.15.83:0/4266125963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ed017640 1 -- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 msgr2=0x7ff8e80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ed017640 1 --2- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 0x7ff8e80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ed017640 1 -- 172.21.15.83:0/4266125963 >> v1:172.21.15.155:6789/0 conn(0x7ff8e81570e0 legacy=0x7ff8e80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ed017640 1 -- 172.21.15.83:0/4266125963 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e80d0b00 con 0x7ff8e8160420 2024-05-17T21:58:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ed017640 1 --2- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e8160830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0097c0 tx=0x7ff8dc01e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2e081f3f77eb1e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ec816640 1 -- 172.21.15.83:0/4266125963 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0ab670 con 0x7ff8e8160420 2024-05-17T21:58:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ec816640 1 -- 172.21.15.83:0/4266125963 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0ab810 con 0x7ff8e8160420 2024-05-17T21:58:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8ec816640 1 -- 172.21.15.83:0/4266125963 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0abbe0 con 0x7ff8e8160420 2024-05-17T21:58:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 -- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 msgr2=0x7ff8e8160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e8160830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0097c0 tx=0x7ff8dc01e6b0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 -- 172.21.15.83:0/4266125963 shutdown_connections 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e80b4f20 0x7ff8e80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/4266125963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e8160830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 -- 172.21.15.83:0/4266125963 >> 172.21.15.83:0/4266125963 conn(0x7ff8e806cd90 msgr2=0x7ff8e80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 -- 172.21.15.83:0/4266125963 shutdown_connections 2024-05-17T21:58:14.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.415+0000 7ff8efa7c640 1 -- 172.21.15.83:0/4266125963 wait complete. 2024-05-17T21:58:14.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 Processor -- start 2024-05-17T21:58:14.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 -- start start 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e81570e0 0x7ff8e8155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e81564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e8072e50 con 0x7ff8e80b4f20 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e80d0600 con 0x7ff8e8160420 2024-05-17T21:58:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8efa7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e8072fd0 con 0x7ff8e81570e0 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8ed818640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8ed818640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37972/0 (socket says 172.21.15.83:37972) 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.419+0000 7ff8ed818640 1 -- 172.21.15.83:0/1042716821 learned_addr learned my addr 172.21.15.83:0/1042716821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ee019640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e81564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed017640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e81570e0 0x7ff8e8155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 -- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e81570e0 msgr2=0x7ff8e8155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e81570e0 0x7ff8e8155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 -- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 msgr2=0x7ff8e81564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e81564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 -- 172.21.15.83:0/1042716821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e81507b0 con 0x7ff8e80b4f20 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed818640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c3c0 tx=0x7ff8e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4017070 con 0x7ff8e80b4f20 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e40052d0 con 0x7ff8e80b4f20 2024-05-17T21:58:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e8150f80 con 0x7ff8e80b4f20 2024-05-17T21:58:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40054c0 con 0x7ff8e80b4f20 2024-05-17T21:58:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e8151540 con 0x7ff8e80b4f20 2024-05-17T21:58:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e80b5330 con 0x7ff8e80b4f20 2024-05-17T21:58:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff8e401a070 con 0x7ff8e80b4f20 2024-05-17T21:58:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 0x7ff8cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7ff8e408e310 con 0x7ff8e80b4f20 2024-05-17T21:58:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed017640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 0x7ff8cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.423+0000 7ff8ed017640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 0x7ff8cc071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0b0000 tx=0x7ff8dc0b5040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:14.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.427+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff8e8160d70 con 0x7ff8e80b4f20 2024-05-17T21:58:14.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:14.579+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff8e80b55d0 con 0x7ff8e80b4f20 2024-05-17T21:58:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.139+0000 7ff8da7fc640 1 -- 172.21.15.83:0/1042716821 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) ==== 138+0+0 (secure 0 0 0) 0x7ff8e405b560 con 0x7ff8e80b4f20 2024-05-17T21:58:15.133 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 msgr2=0x7ff8cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 0x7ff8cc071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0b0000 tx=0x7ff8dc0b5040 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 msgr2=0x7ff8e8150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c3c0 tx=0x7ff8e400c880 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 shutdown_connections 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff8cc06f400 0x7ff8cc071910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e8160420 0x7ff8e81564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8e81570e0 0x7ff8e8155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 --2- 172.21.15.83:0/1042716821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8e80b4f20 0x7ff8e8150270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 >> 172.21.15.83:0/1042716821 conn(0x7ff8e806cd90 msgr2=0x7ff8e80b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 shutdown_connections 2024-05-17T21:58:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.143+0000 7ff8efa7c640 1 -- 172.21.15.83:0/1042716821 wait complete. 2024-05-17T21:58:15.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T21:58:15.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.323+0000 7f1d7bab8640 1 Processor -- start 2024-05-17T21:58:15.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.323+0000 7f1d7bab8640 1 -- start start 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7bab8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d740d17d0 0x7f1d740d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7bab8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d7406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7bab8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d740b5020 con 0x7f1d7406dde0 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7bab8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d740cdee0 con 0x7f1d740d17d0 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7bab8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7409cde0 con 0x7f1d740d2120 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7a055640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1d7406dde0 0x7f1d7406e1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56516/0 (socket says 172.21.15.83:56516) 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7a055640 1 -- 172.21.15.83:0/3251308280 learned_addr learned my addr 172.21.15.83:0/3251308280 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d7406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79854640 1 --2- 172.21.15.83:0/3251308280 >> v2:172.21.15.155:3300/0 conn(0x7f1d740d17d0 0x7f1d740d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538811023 0 0) 0x7f1d6c0014a0 con 0x7f1d7406dde0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d60003530 con 0x7f1d7406dde0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 541365602 0 0) 0x7f1d60003530 con 0x7f1d7406dde0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 msgr2=0x7f1d7406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d7406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 >> v2:172.21.15.155:3300/0 conn(0x7f1d740d17d0 msgr2=0x7f1d740d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 --2- 172.21.15.83:0/3251308280 >> v2:172.21.15.155:3300/0 conn(0x7f1d740d17d0 0x7f1d740d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7409c540 con 0x7f1d7406dde0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1d6c001c50 con 0x7f1d7406dde0 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 >> v1:172.21.15.83:6789/0 conn(0x7f1d7406dde0 legacy=0x7f1d7406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d60003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d60003fe0 0x7f1d60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60004d50 0x7f1d60005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d740cdee0 con 0x7f1d60003fe0 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7409cde0 con 0x7f1d60004d50 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6000bc10 con 0x7f1d740d2120 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d60003fe0 0x7f1d60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79854640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d60003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d7a055640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60004d50 0x7f1d60005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 -- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 msgr2=0x7f1d60003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d60003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 -- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60004d50 msgr2=0x7f1d60005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60004d50 0x7f1d60005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79053640 1 -- 172.21.15.83:0/3251308280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60003530 con 0x7f1d60003fe0 2024-05-17T21:58:15.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.327+0000 7f1d79854640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d740d2120 0x7f1d60003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:15.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d79053640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d60003fe0 0x7f1d60004810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ab0 tx=0x7f1d6401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20d56534b0b790f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:15.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640b2070 con 0x7f1d60003fe0 2024-05-17T21:58:15.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d640a5530 con 0x7f1d60003fe0 2024-05-17T21:58:15.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d78852640 1 -- 172.21.15.83:0/3251308280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640096b0 con 0x7f1d60003fe0 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- 172.21.15.83:0/3251308280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d60003fe0 msgr2=0x7f1d60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/3251308280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d60003fe0 0x7f1d60004810 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ab0 tx=0x7f1d6401ca70 comp rx=0 tx=0).stop 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7a055640 1 -- 172.21.15.83:0/3251308280 reap_dead start 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- 172.21.15.83:0/3251308280 shutdown_connections 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- 172.21.15.83:0/3251308280 >> 172.21.15.83:0/3251308280 conn(0x7f1d740d36d0 msgr2=0x7f1d740d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- 172.21.15.83:0/3251308280 shutdown_connections 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- 172.21.15.83:0/3251308280 wait complete. 2024-05-17T21:58:15.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 Processor -- start 2024-05-17T21:58:15.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.331+0000 7f1d7bab8640 1 -- start start 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 0x7f1d740c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d740cbf70 0x7f1d740ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d740d26b0 con 0x7f1d7415aef0 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d740d2530 con 0x7f1d740cbf70 2024-05-17T21:58:15.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7406d8a0 con 0x7f1d60003fe0 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37994/0 (socket says 172.21.15.83:37994) 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 -- 172.21.15.83:0/2428158643 learned_addr learned my addr 172.21.15.83:0/2428158643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d79854640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 0x7f1d740c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d79053640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d740cbf70 0x7f1d740ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 -- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 msgr2=0x7f1d740c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 0x7f1d740c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 -- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d740cbf70 msgr2=0x7f1d740ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:15.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d740cbf70 0x7f1d740ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 -- 172.21.15.83:0/2428158643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d740caed0 con 0x7f1d7415aef0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d79854640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 0x7f1d740c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7a055640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d740ca730 tx=0x7f1d6c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00ea20 con 0x7f1d7415aef0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6c019070 con 0x7f1d7415aef0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d740cb1b0 con 0x7f1d7415aef0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00ee20 con 0x7f1d7415aef0 2024-05-17T21:58:15.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d740cb770 con 0x7f1d7415aef0 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d740d1be0 con 0x7f1d7415aef0 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d6c01e460 con 0x7f1d7415aef0 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 0x7f1d58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d79854640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 0x7f1d58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.335+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5227+0+0 (secure 0 0 0) 0x7f1d6c08fd90 con 0x7f1d7415aef0 2024-05-17T21:58:15.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.339+0000 7f1d79854640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 0x7f1d58071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009bf0 tx=0x7f1d70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:15.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.339+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1d740b5020 con 0x7f1d7415aef0 2024-05-17T21:58:15.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:15.487+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1d740b4990 con 0x7f1d7415aef0 2024-05-17T21:58:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.143+0000 7f1d6a7fc640 1 -- 172.21.15.83:0/2428158643 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) ==== 170+0+0 (secure 0 0 0) 0x7f1d6c05cfd0 con 0x7f1d7415aef0 2024-05-17T21:58:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 msgr2=0x7f1d58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 0x7f1d58071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009bf0 tx=0x7f1d70008040 comp rx=0 tx=0).stop 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 msgr2=0x7f1d740cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d740ca730 tx=0x7f1d6c007bc0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 shutdown_connections 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d7415aef0 0x7f1d740cd290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d5806f400 0x7f1d58071910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d740cbf70 0x7f1d740ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 --2- 172.21.15.83:0/2428158643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d60003fe0 0x7f1d740c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 >> 172.21.15.83:0/2428158643 conn(0x7f1d740d36d0 msgr2=0x7f1d74071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 shutdown_connections 2024-05-17T21:58:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.147+0000 7f1d7bab8640 1 -- 172.21.15.83:0/2428158643 wait complete. 2024-05-17T21:58:16.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T21:58:16.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 Processor -- start 2024-05-17T21:58:16.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 -- start start 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb000cf830 0x7fcb000b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb000cdf40 0x7fcb0015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb0015d530 0x7fcb0006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcb000b5640 con 0x7fcb0015d530 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcb000b58e0 con 0x7fcb000cf830 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcb05d57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0006f060 con 0x7fcb000cdf40 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcafffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb0015d530 0x7fcb0006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcafffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb0015d530 0x7fcb0006f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38008/0 (socket says 172.21.15.83:38008) 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcafffff640 1 -- 172.21.15.83:0/4036441098 learned_addr learned my addr 172.21.15.83:0/4036441098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcafeffd640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb000cdf40 0x7fcb0015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.347+0000 7fcaff7fe640 1 --2- 172.21.15.83:0/4036441098 >> v2:172.21.15.155:3300/0 conn(0x7fcb000cf830 0x7fcb000b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 -- 172.21.15.83:0/4036441098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb000cdf40 msgr2=0x7fcb0015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:16.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb000cdf40 0x7fcb0015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 -- 172.21.15.83:0/4036441098 >> v2:172.21.15.155:3300/0 conn(0x7fcb000cf830 msgr2=0x7fcb000b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:16.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 --2- 172.21.15.83:0/4036441098 >> v2:172.21.15.155:3300/0 conn(0x7fcb000cf830 0x7fcb000b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 -- 172.21.15.83:0/4036441098 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0006f340 con 0x7fcb0015d530 2024-05-17T21:58:16.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafffff640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0015d530 0x7fcb0006f860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb0006d6e0 tx=0x7fcaf801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca556e28344c9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:16.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafe7fc640 1 -- 172.21.15.83:0/4036441098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80a5780 con 0x7fcb0015d530 2024-05-17T21:58:16.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafe7fc640 1 -- 172.21.15.83:0/4036441098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf80a5920 con 0x7fcb0015d530 2024-05-17T21:58:16.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcafe7fc640 1 -- 172.21.15.83:0/4036441098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80a5cf0 con 0x7fcb0015d530 2024-05-17T21:58:16.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 -- 172.21.15.83:0/4036441098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0015d530 msgr2=0x7fcb0006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:16.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0015d530 0x7fcb0006f860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb0006d6e0 tx=0x7fcaf801c9f0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 -- 172.21.15.83:0/4036441098 shutdown_connections 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb0015d530 0x7fcb0006f860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 --2- 172.21.15.83:0/4036441098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb000cdf40 0x7fcb0015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 --2- 172.21.15.83:0/4036441098 >> v2:172.21.15.155:3300/0 conn(0x7fcb000cf830 0x7fcb000b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 -- 172.21.15.83:0/4036441098 >> 172.21.15.83:0/4036441098 conn(0x7fcb0006cd90 msgr2=0x7fcb000b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 -- 172.21.15.83:0/4036441098 shutdown_connections 2024-05-17T21:58:16.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.351+0000 7fcb05d57640 1 -- 172.21.15.83:0/4036441098 wait complete. 2024-05-17T21:58:16.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 Processor -- start 2024-05-17T21:58:16.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- start start 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb000cf830 0x7fcb00152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 0x7fcb00161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb001683e0 con 0x7fcb000cdf40 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb000707c0 con 0x7fcb000cf830 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb00168560 con 0x7fcb0015d530 2024-05-17T21:58:16.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38014/0 (socket says 172.21.15.83:38014) 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 -- 172.21.15.83:0/2809552491 learned_addr learned my addr 172.21.15.83:0/2809552491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcafeffd640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb000cf830 0x7fcb00152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcafffff640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 0x7fcb00161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 -- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 msgr2=0x7fcb00161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 0x7fcb00161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 -- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb000cf830 msgr2=0x7fcb00152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb000cf830 0x7fcb00152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 -- 172.21.15.83:0/2809552491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb00163970 con 0x7fcb000cdf40 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcafffff640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 0x7fcb00161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcaff7fe640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d620 tx=0x7fcaf000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:16.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000b620 con 0x7fcb000cdf40 2024-05-17T21:58:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0019070 con 0x7fcb000cdf40 2024-05-17T21:58:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb00158160 con 0x7fcb000cdf40 2024-05-17T21:58:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000b9e0 con 0x7fcb000cdf40 2024-05-17T21:58:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.355+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb00160e50 con 0x7fcb000cdf40 2024-05-17T21:58:16.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb000b3aa0 con 0x7fcb000cdf40 2024-05-17T21:58:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcaf000bb80 con 0x7fcb000cdf40 2024-05-17T21:58:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcae7fff640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 0x7fcaec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5609+0+0 (secure 0 0 0) 0x7fcaf008f510 con 0x7fcb000cdf40 2024-05-17T21:58:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcafeffd640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 0x7fcaec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:16.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.359+0000 7fcafeffd640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 0x7fcaec071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb001f9ac0 tx=0x7fcaf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:16.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.363+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcaf005c580 con 0x7fcb000cdf40 2024-05-17T21:58:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:16.559+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcb00158680 con 0x7fcb000cdf40 2024-05-17T21:58:17.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.203+0000 7fcae7fff640 1 -- 172.21.15.83:0/2809552491 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fcaf005be80 con 0x7fcb000cdf40 2024-05-17T21:58:17.200 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 4 and data pool 5 2024-05-17T21:58:17.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 msgr2=0x7fcaec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 0x7fcaec071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb001f9ac0 tx=0x7fcaf4008040 comp rx=0 tx=0).stop 2024-05-17T21:58:17.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 msgr2=0x7fcb00163430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000d620 tx=0x7fcaf000dae0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 shutdown_connections 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb0015d530 0x7fcb00161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcaec06f400 0x7fcaec071910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb000cf830 0x7fcb00152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 --2- 172.21.15.83:0/2809552491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb000cdf40 0x7fcb00163430 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.211+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 >> 172.21.15.83:0/2809552491 conn(0x7fcb0006cd90 msgr2=0x7fcb000cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:17.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.223+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 shutdown_connections 2024-05-17T21:58:17.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.223+0000 7fcb05d57640 1 -- 172.21.15.83:0/2809552491 wait complete. 2024-05-17T21:58:17.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 Processor -- start 2024-05-17T21:58:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 -- start start 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6240070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f624006d7a0 0x7f624006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624006e0f0 con 0x7f6240161460 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f62400709f0 con 0x7f624006d7a0 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f624508b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f62400700e0 con 0x7f62400af7c0 2024-05-17T21:58:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623f59e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6240161460 0x7f6240161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41566/0 (socket says 172.21.15.83:41566) 2024-05-17T21:58:17.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623f59e640 1 -- 172.21.15.83:0/683691644 learned_addr learned my addr 172.21.15.83:0/683691644 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:17.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623ed9d640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6240070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623e59c640 1 --2- 172.21.15.83:0/683691644 >> v2:172.21.15.155:3300/0 conn(0x7f624006d7a0 0x7f624006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877887129 0 0) 0x7f62380014a0 con 0x7f6240161460 2024-05-17T21:58:17.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.395+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6228003530 con 0x7f6240161460 2024-05-17T21:58:17.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 838083173 0 0) 0x7f6228003530 con 0x7f6240161460 2024-05-17T21:58:17.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 msgr2=0x7f6240070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6240070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 >> v2:172.21.15.155:3300/0 conn(0x7f624006d7a0 msgr2=0x7f624006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 --2- 172.21.15.83:0/683691644 >> v2:172.21.15.155:3300/0 conn(0x7f624006d7a0 0x7f624006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62400aff10 con 0x7f6240161460 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f6238001c50 con 0x7f6240161460 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 >> v1:172.21.15.83:6789/0 conn(0x7f6240161460 legacy=0x7f6240161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6228003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6228003fe0 0x7f6228004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f6228005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62400709f0 con 0x7f6228004d50 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62400700e0 con 0x7f6228003fe0 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f622800bad0 con 0x7f62400af7c0 2024-05-17T21:58:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f6228005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623ed9d640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6228003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623e59c640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6228003fe0 0x7f6228004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 -- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 msgr2=0x7f6228003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400af7c0 0x7f6228003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 -- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6228003fe0 msgr2=0x7f6228004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6228003fe0 0x7f6228004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 -- 172.21.15.83:0/683691644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228003530 con 0x7f6228004d50 2024-05-17T21:58:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f6228005160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f623800aa40 tx=0x7f623801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe1dcaac33dede server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623800a690 con 0x7f6228004d50 2024-05-17T21:58:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62380072d0 con 0x7f6228004d50 2024-05-17T21:58:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623dd9b640 1 -- 172.21.15.83:0/683691644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238007700 con 0x7f6228004d50 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f624508b640 1 -- 172.21.15.83:0/683691644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 msgr2=0x7f6228005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f624508b640 1 --2- 172.21.15.83:0/683691644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f6228005160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f623800aa40 tx=0x7f623801d2a0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.399+0000 7f623f59e640 1 -- 172.21.15.83:0/683691644 reap_dead start 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- 172.21.15.83:0/683691644 shutdown_connections 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- 172.21.15.83:0/683691644 >> 172.21.15.83:0/683691644 conn(0x7f6240073670 msgr2=0x7f6240073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- 172.21.15.83:0/683691644 shutdown_connections 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- 172.21.15.83:0/683691644 wait complete. 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 Processor -- start 2024-05-17T21:58:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- start start 2024-05-17T21:58:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 0x7f62400cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 0x7f62400caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62400af500 con 0x7f6228004d50 2024-05-17T21:58:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6240070680 con 0x7f62400caad0 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f624508b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f624015e8e0 con 0x7f62400cc090 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51676/0 (socket says 172.21.15.83:51676) 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 -- 172.21.15.83:0/2938181646 learned_addr learned my addr 172.21.15.83:0/2938181646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623f59e640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 0x7f62400caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623e59c640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 0x7f62400cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 -- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 msgr2=0x7f62400cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 0x7f62400cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 -- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 msgr2=0x7f62400caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 0x7f62400caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623ed9d640 1 -- 172.21.15.83:0/2938181646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62400d1a30 con 0x7f6228004d50 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623e59c640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 0x7f62400cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.403+0000 7f623f59e640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 0x7f62400caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f623ed9d640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6230011610 tx=0x7f6230011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000f8f0 con 0x7f6228004d50 2024-05-17T21:58:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623000fa90 con 0x7f6228004d50 2024-05-17T21:58:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62400c9a30 con 0x7f6228004d50 2024-05-17T21:58:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000fe40 con 0x7f6228004d50 2024-05-17T21:58:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62400c9ff0 con 0x7f6228004d50 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6240069c40 con 0x7f6228004d50 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6230015460 con 0x7f6228004d50 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 0x7f622c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f623008fb60 con 0x7f6228004d50 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f623e59c640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 0x7f622c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.407+0000 7f623e59c640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 0x7f622c071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624015e770 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.411+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6230019030 con 0x7f6228004d50 2024-05-17T21:58:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62400afbd0 con 0x7f6228004d50 2024-05-17T21:58:17.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f62277fe640 1 -- 172.21.15.83:0/2938181646 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16658 (secure 0 0 0) 0x7f623005cb70 con 0x7f6228004d50 2024-05-17T21:58:17.565 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:17.565 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:16.566258+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:14.593534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:15.504714+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 msgr2=0x7f622c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 0x7f622c071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624015e770 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-05-17T21:58:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 msgr2=0x7f62401f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6230011610 tx=0x7f6230011ad0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 shutdown_connections 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f622c06f400 0x7f622c071910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62400caad0 0x7f62400caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62400cc090 0x7f62400cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 --2- 172.21.15.83:0/2938181646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6228004d50 0x7f62401f3af0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 >> 172.21.15.83:0/2938181646 conn(0x7f6240073670 msgr2=0x7f6240070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 shutdown_connections 2024-05-17T21:58:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.571+0000 7f624508b640 1 -- 172.21.15.83:0/2938181646 wait complete. 2024-05-17T21:58:17.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.723+0000 7f45cd7ad640 1 Processor -- start 2024-05-17T21:58:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.723+0000 7f45cd7ad640 1 -- start start 2024-05-17T21:58:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45cd7ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45c80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45cd7ad640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f45c8072e30 0x7f45c8073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45cd7ad640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c8073780 con 0x7f45c815fef0 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45cd7ad640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f45c806ff60 con 0x7f45c8072e30 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45cd7ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80b49b0 con 0x7f45c80b5a50 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f45c815fef0 0x7f45c806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41578/0 (socket says 172.21.15.83:41578) 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c6ffd640 1 -- 172.21.15.83:0/476037718 learned_addr learned my addr 172.21.15.83:0/476037718 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c67fc640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45c80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 --2- 172.21.15.83:0/476037718 >> v2:172.21.15.155:3300/0 conn(0x7f45c8072e30 0x7f45c8073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480036365 0 0) 0x7f45b8001120 con 0x7f45c815fef0 2024-05-17T21:58:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b0003530 con 0x7f45c815fef0 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3456227307 0 0) 0x7f45b0003530 con 0x7f45c815fef0 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 msgr2=0x7f45c80b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45c80b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 >> v2:172.21.15.155:3300/0 conn(0x7f45c8072e30 msgr2=0x7f45c8073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 --2- 172.21.15.83:0/476037718 >> v2:172.21.15.155:3300/0 conn(0x7f45c8072e30 0x7f45c8073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c80b4c90 con 0x7f45c815fef0 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f45b80018d0 con 0x7f45c815fef0 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 >> v1:172.21.15.83:6789/0 conn(0x7f45c815fef0 legacy=0x7f45c806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b0003a10 0x7f45b0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b0004d50 0x7f45b0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45c806ff60 con 0x7f45b0004d50 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b49b0 con 0x7f45b0003a10 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45b000bab0 con 0x7f45c80b5a50 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b0004d50 0x7f45b0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c6ffd640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b0003a10 0x7f45b0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c67fc640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 -- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 msgr2=0x7f45b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c80b5a50 0x7f45b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 -- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b0003a10 msgr2=0x7f45b0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45b0003a10 0x7f45b0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.727+0000 7f45c77fe640 1 -- 172.21.15.83:0/476037718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b0003530 con 0x7f45b0004d50 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45c77fe640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b0004d50 0x7f45b0005160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009f90 tx=0x7f45c001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5effa461c705125f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00b2070 con 0x7f45b0004d50 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c00a5c90 con 0x7f45b0004d50 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45c5ffb640 1 -- 172.21.15.83:0/476037718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0009c10 con 0x7f45b0004d50 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- 172.21.15.83:0/476037718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b0004d50 msgr2=0x7f45b0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/476037718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45b0004d50 0x7f45b0005160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009f90 tx=0x7f45c001cc80 comp rx=0 tx=0).stop 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45c77fe640 1 -- 172.21.15.83:0/476037718 reap_dead start 2024-05-17T21:58:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- 172.21.15.83:0/476037718 shutdown_connections 2024-05-17T21:58:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- 172.21.15.83:0/476037718 >> 172.21.15.83:0/476037718 conn(0x7f45c806cd90 msgr2=0x7f45c80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- 172.21.15.83:0/476037718 shutdown_connections 2024-05-17T21:58:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- 172.21.15.83:0/476037718 wait complete. 2024-05-17T21:58:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 Processor -- start 2024-05-17T21:58:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.731+0000 7f45cd7ad640 1 -- start start 2024-05-17T21:58:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 0x7f45c80d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 0x7f45c815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b5530 con 0x7f45c8163b90 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b5250 con 0x7f45c815ec00 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80b45d0 con 0x7f45c8161850 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51698/0 (socket says 172.21.15.83:51698) 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 -- 172.21.15.83:0/3239160740 learned_addr learned my addr 172.21.15.83:0/3239160740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c6ffd640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 0x7f45c80d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c77fe640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 0x7f45c815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 -- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 msgr2=0x7f45c80d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 0x7f45c80d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 -- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 msgr2=0x7f45c815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 0x7f45c815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 -- 172.21.15.83:0/3239160740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c815f550 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c6ffd640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 0x7f45c80d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c77fe640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 0x7f45c815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45c67fc640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0108c0 tx=0x7f45bc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00f040 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc00eb60 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c8161f50 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc01c400 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c8162510 con 0x7f45c8163b90 2024-05-17T21:58:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8073240 con 0x7f45c8163b90 2024-05-17T21:58:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.735+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f45bc01c5a0 con 0x7f45c8163b90 2024-05-17T21:58:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.739+0000 7f45af7fe640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 0x7f45b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.739+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f45bc08f8f0 con 0x7f45c8163b90 2024-05-17T21:58:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.739+0000 7f45c6ffd640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 0x7f45b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.739+0000 7f45c6ffd640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 0x7f45b4071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cda00 tx=0x7f45b8001300 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.743+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f45c815f550 con 0x7f45c8163b90 2024-05-17T21:58:17.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.903+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45c8070150 con 0x7f45c8163b90 2024-05-17T21:58:17.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.903+0000 7f45af7fe640 1 -- 172.21.15.83:0/3239160740 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16658 (secure 0 0 0) 0x7f45bc018030 con 0x7f45c8163b90 2024-05-17T21:58:17.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:17.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:16.566258+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:14.593534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:15.504714+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 msgr2=0x7f45b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 0x7f45b4071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cda00 tx=0x7f45b8001300 comp rx=0 tx=0).stop 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 msgr2=0x7f45c8162ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0108c0 tx=0x7f45bc010d80 comp rx=0 tx=0).stop 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 shutdown_connections 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f45b406f400 0x7f45b4071910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45c815ec00 0x7f45c815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c8163b90 0x7f45c8162ff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 --2- 172.21.15.83:0/3239160740 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45c8161850 0x7f45c80d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 >> 172.21.15.83:0/3239160740 conn(0x7f45c806cd90 msgr2=0x7f45c80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 shutdown_connections 2024-05-17T21:58:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:17.907+0000 7f45cd7ad640 1 -- 172.21.15.83:0/3239160740 wait complete. 2024-05-17T21:58:17.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T21:58:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 Processor -- start 2024-05-17T21:58:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 -- start start 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda280d2710 0x7fda280d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda280d4170 con 0x7fda280d5210 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda280d4450 con 0x7fda280b4810 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda3079d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fda280d5620 con 0x7fda280d2710 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda280d2710 0x7fda280d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda280d2710 0x7fda280d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51700/0 (socket says 172.21.15.83:51700) 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 -- 172.21.15.83:0/427567771 learned_addr learned my addr 172.21.15.83:0/427567771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 -- 172.21.15.83:0/427567771 >> v1:172.21.15.155:6789/0 conn(0x7fda280b4810 legacy=0x7fda2815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 -- 172.21.15.83:0/427567771 >> v1:172.21.15.83:6789/0 conn(0x7fda280d5210 legacy=0x7fda280d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2ed3a640 1 -- 172.21.15.83:0/427567771 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda280d4730 con 0x7fda280d2710 2024-05-17T21:58:18.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.079+0000 7fda2d537640 1 -- 172.21.15.83:0/427567771 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782292446 0 0) 0x7fda280d4170 con 0x7fda280d5210 2024-05-17T21:58:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda2ed3a640 1 --2- 172.21.15.83:0/427567771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280d2710 0x7fda280d0290 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda20009e50 tx=0x7fda2001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53f370dbd74db27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda2d537640 1 -- 172.21.15.83:0/427567771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200a5920 con 0x7fda280d2710 2024-05-17T21:58:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda2d537640 1 -- 172.21.15.83:0/427567771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda200a5ac0 con 0x7fda280d2710 2024-05-17T21:58:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda2d537640 1 -- 172.21.15.83:0/427567771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200ad3f0 con 0x7fda280d2710 2024-05-17T21:58:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- 172.21.15.83:0/427567771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280d2710 msgr2=0x7fda280d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 --2- 172.21.15.83:0/427567771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280d2710 0x7fda280d0290 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda20009e50 tx=0x7fda2001ca30 comp rx=0 tx=0).stop 2024-05-17T21:58:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- 172.21.15.83:0/427567771 shutdown_connections 2024-05-17T21:58:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 --2- 172.21.15.83:0/427567771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280d2710 0x7fda280d0290 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- 172.21.15.83:0/427567771 >> 172.21.15.83:0/427567771 conn(0x7fda2806e010 msgr2=0x7fda2806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- 172.21.15.83:0/427567771 shutdown_connections 2024-05-17T21:58:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- 172.21.15.83:0/427567771 wait complete. 2024-05-17T21:58:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 Processor -- start 2024-05-17T21:58:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.083+0000 7fda3079d640 1 -- start start 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 0x7fda2815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 0x7fda2815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda2816d840 con 0x7fda280b4810 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda2816d6c0 con 0x7fda280d2710 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda2816d9c0 con 0x7fda280d5210 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51714/0 (socket says 172.21.15.83:51714) 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 -- 172.21.15.83:0/507181774 learned_addr learned my addr 172.21.15.83:0/507181774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2ed3a640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 0x7fda2815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2dd38640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 0x7fda2815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 -- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 msgr2=0x7fda2815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 0x7fda2815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 -- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 msgr2=0x7fda2815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 0x7fda2815d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 -- 172.21.15.83:0/507181774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda280bacb0 con 0x7fda280b4810 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2dd38640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 0x7fda2815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2ed3a640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 0x7fda2815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2e539640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b440 tx=0x7fda2400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24015020 con 0x7fda280b4810 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda24011890 con 0x7fda280b4810 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda28159310 con 0x7fda280b4810 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24011c80 con 0x7fda280b4810 2024-05-17T21:58:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda281598d0 con 0x7fda280b4810 2024-05-17T21:58:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda280d0290 con 0x7fda280b4810 2024-05-17T21:58:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda24018020 con 0x7fda280b4810 2024-05-17T21:58:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda2dd38640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.087+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fda2408eff0 con 0x7fda280b4810 2024-05-17T21:58:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.091+0000 7fda2dd38640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda2806a6a0 tx=0x7fda18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.091+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda2401f030 con 0x7fda280b4810 2024-05-17T21:58:18.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:18.303+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fda28152080 con 0x7fda280b4810 2024-05-17T21:58:19.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.219+0000 7fda1effd640 1 -- 172.21.15.83:0/507181774 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fda2400d040 con 0x7fda280b4810 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 msgr2=0x7fda0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda2806a6a0 tx=0x7fda18009040 comp rx=0 tx=0).stop 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 msgr2=0x7fda28151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda2400b440 tx=0x7fda2400b900 comp rx=0 tx=0).stop 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 shutdown_connections 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda280d5210 0x7fda2815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d2710 0x7fda2815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 --2- 172.21.15.83:0/507181774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda280b4810 0x7fda28151b40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 >> 172.21.15.83:0/507181774 conn(0x7fda2806e010 msgr2=0x7fda2815fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:19.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 shutdown_connections 2024-05-17T21:58:19.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.223+0000 7fda3079d640 1 -- 172.21.15.83:0/507181774 wait complete. 2024-05-17T21:58:19.239 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T21:58:19.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T21:58:19.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.399+0000 7f2721e25640 1 Processor -- start 2024-05-17T21:58:19.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.399+0000 7f2721e25640 1 -- start start 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2721e25640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f271c0b3d00 0x7f271c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2721e25640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f271c143f20 0x7f271c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2721e25640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f271c0737d0 con 0x7f271c0733c0 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2721e25640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f271c0b4110 con 0x7f271c143f20 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2721e25640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f271c070770 con 0x7f271c0b3d00 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f271c143f20 0x7f271c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f271c0733c0 0x7f271c06eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41600/0 (socket says 172.21.15.83:41600) 2024-05-17T21:58:19.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271bfff640 1 -- 172.21.15.83:0/1845518034 learned_addr learned my addr 172.21.15.83:0/1845518034 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271b7fe640 1 --2- 172.21.15.83:0/1845518034 >> v2:172.21.15.155:3300/0 conn(0x7f271c0b3d00 0x7f271c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 -- 172.21.15.83:0/1845518034 >> v2:172.21.15.155:3300/0 conn(0x7f271c0b3d00 msgr2=0x7f271c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 --2- 172.21.15.83:0/1845518034 >> v2:172.21.15.155:3300/0 conn(0x7f271c0b3d00 0x7f271c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 -- 172.21.15.83:0/1845518034 >> v1:172.21.15.83:6789/0 conn(0x7f271c0733c0 legacy=0x7f271c06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271affd640 1 -- 172.21.15.83:0/1845518034 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585496856 0 0) 0x7f271c0737d0 con 0x7f271c0733c0 2024-05-17T21:58:19.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 -- 172.21.15.83:0/1845518034 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c0d10d0 con 0x7f271c143f20 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f2712ffd640 1 --2- 172.21.15.83:0/1845518034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c143f20 0x7f271c072e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f271c06d6e0 tx=0x7f270401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecf56a461489dad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271affd640 1 -- 172.21.15.83:0/1845518034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040b0070 con 0x7f271c143f20 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271affd640 1 -- 172.21.15.83:0/1845518034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27040ab600 con 0x7f271c143f20 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.403+0000 7f271affd640 1 -- 172.21.15.83:0/1845518034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040ab9d0 con 0x7f271c143f20 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- 172.21.15.83:0/1845518034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c143f20 msgr2=0x7f271c072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- 172.21.15.83:0/1845518034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c143f20 0x7f271c072e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f271c06d6e0 tx=0x7f270401c3f0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- 172.21.15.83:0/1845518034 shutdown_connections 2024-05-17T21:58:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- 172.21.15.83:0/1845518034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c143f20 0x7f271c072e80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- 172.21.15.83:0/1845518034 >> v2:172.21.15.155:3300/0 conn(0x7f271c0b3d00 0x7f271c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- 172.21.15.83:0/1845518034 >> 172.21.15.83:0/1845518034 conn(0x7f271c06cd90 msgr2=0x7f271c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- 172.21.15.83:0/1845518034 shutdown_connections 2024-05-17T21:58:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- 172.21.15.83:0/1845518034 wait complete. 2024-05-17T21:58:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 Processor -- start 2024-05-17T21:58:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- start start 2024-05-17T21:58:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0733c0 0x7f271c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 0x7f271c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f271c070c00 con 0x7f271c0b3d00 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f271c144330 con 0x7f271c0733c0 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2721e25640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f271c0c98a0 con 0x7f271c143f20 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51720/0 (socket says 172.21.15.83:51720) 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 -- 172.21.15.83:0/780040062 learned_addr learned my addr 172.21.15.83:0/780040062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f271b7fe640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0733c0 0x7f271c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f271bfff640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 0x7f271c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 -- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 msgr2=0x7f271c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 0x7f271c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 -- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0733c0 msgr2=0x7f271c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0733c0 0x7f271c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f2712ffd640 1 -- 172.21.15.83:0/780040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c0d22b0 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.407+0000 7f271bfff640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 0x7f271c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2712ffd640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27040ae040 tx=0x7f2704019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040b0040 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27040b9070 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c0c8310 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040b4690 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c0c64d0 con 0x7f271c0b3d00 2024-05-17T21:58:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c0d0a10 con 0x7f271c0b3d00 2024-05-17T21:58:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f27040b4830 con 0x7f271c0b3d00 2024-05-17T21:58:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 0x7f26fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2704128fa0 con 0x7f271c0b3d00 2024-05-17T21:58:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f271b7fe640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 0x7f26fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.411+0000 7f271b7fe640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 0x7f26fc071940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b930 tx=0x7f270c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.415+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2704016070 con 0x7f271c0b3d00 2024-05-17T21:58:19.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:19.615+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f271c13e0a0 con 0x7f271c0b3d00 2024-05-17T21:58:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.223+0000 7f2718ff9640 1 -- 172.21.15.83:0/780040062 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f27040f5f20 con 0x7f271c0b3d00 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 msgr2=0x7f26fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 0x7f26fc071940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f270c00b930 tx=0x7f270c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 msgr2=0x7f271c0c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27040ae040 tx=0x7f2704019190 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 shutdown_connections 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f271c143f20 0x7f271c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c0b3d00 0x7f271c0c7290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26fc06f430 0x7f26fc071940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 --2- 172.21.15.83:0/780040062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271c0733c0 0x7f271c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 >> 172.21.15.83:0/780040062 conn(0x7f271c06cd90 msgr2=0x7f271c13d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:20.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 shutdown_connections 2024-05-17T21:58:20.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.227+0000 7f2721e25640 1 -- 172.21.15.83:0/780040062 wait complete. 2024-05-17T21:58:20.246 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T21:58:20.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:20.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 Processor -- start 2024-05-17T21:58:20.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 -- start start 2024-05-17T21:58:20.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa39406ffd0 0x7fa3940703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3941605a0 0x7fa3941609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3941465d0 con 0x7fa3941461c0 2024-05-17T21:58:20.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa394160ef0 con 0x7fa3941605a0 2024-05-17T21:58:20.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa39a6aa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa39406ef30 con 0x7fa39406ffd0 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3941605a0 0x7fa3941609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa38b7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa39406ffd0 0x7fa3940703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3941605a0 0x7fa3941609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:39304/0 (socket says 172.21.15.83:39304) 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 -- 172.21.15.83:0/1760622378 learned_addr learned my addr 172.21.15.83:0/1760622378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 -- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa39406ffd0 msgr2=0x7fa3940703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 --2- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa39406ffd0 0x7fa3940703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 -- 172.21.15.83:0/1760622378 >> v1:172.21.15.155:6789/0 conn(0x7fa3941461c0 legacy=0x7fa39415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.407+0000 7fa398c47640 1 -- 172.21.15.83:0/1760622378 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394070920 con 0x7fa3941605a0 2024-05-17T21:58:20.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa398c47640 1 --2- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941605a0 0x7fa3941609b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009e50 tx=0x7fa38c01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78840dfc69a377 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa3937fe640 1 -- 172.21.15.83:0/1760622378 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0a58d0 con 0x7fa3941605a0 2024-05-17T21:58:20.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa3937fe640 1 -- 172.21.15.83:0/1760622378 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c0a5a70 con 0x7fa3941605a0 2024-05-17T21:58:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa3937fe640 1 -- 172.21.15.83:0/1760622378 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0a5e40 con 0x7fa3941605a0 2024-05-17T21:58:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941605a0 msgr2=0x7fa3941609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941605a0 0x7fa3941609b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009e50 tx=0x7fa38c01ca00 comp rx=0 tx=0).stop 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1760622378 shutdown_connections 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941605a0 0x7fa3941609b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1760622378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa39406ffd0 0x7fa3940703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1760622378 >> 172.21.15.83:0/1760622378 conn(0x7fa39406cd90 msgr2=0x7fa3940b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1760622378 shutdown_connections 2024-05-17T21:58:20.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.411+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1760622378 wait complete. 2024-05-17T21:58:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 Processor -- start 2024-05-17T21:58:20.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- start start 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 0x7fa39415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3941605a0 0x7fa39413b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa39415e880 con 0x7fa39406ffd0 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa39406fab0 con 0x7fa3941461c0 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940d10a0 con 0x7fa3941605a0 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51722/0 (socket says 172.21.15.83:51722) 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 -- 172.21.15.83:0/1414855973 learned_addr learned my addr 172.21.15.83:0/1414855973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa38b7fe640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 0x7fa39415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa398c47640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3941605a0 0x7fa39413b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 -- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3941605a0 msgr2=0x7fa39413b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3941605a0 0x7fa39413b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 -- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 msgr2=0x7fa39415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 0x7fa39415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 -- 172.21.15.83:0/1414855973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39415bdc0 con 0x7fa39406ffd0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa38b7fe640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 0x7fa39415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa393fff640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c3f0 tx=0x7fa38400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384017070 con 0x7fa39406ffd0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3840052d0 con 0x7fa39406ffd0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39413c340 con 0x7fa39406ffd0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384003840 con 0x7fa39406ffd0 2024-05-17T21:58:20.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39413cec0 con 0x7fa39406ffd0 2024-05-17T21:58:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.415+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3940703e0 con 0x7fa39406ffd0 2024-05-17T21:58:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.419+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa38401a070 con 0x7fa39406ffd0 2024-05-17T21:58:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.419+0000 7fa3917fa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 0x7fa374071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.419+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa38408e8f0 con 0x7fa39406ffd0 2024-05-17T21:58:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.419+0000 7fa38b7fe640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 0x7fa374071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.419+0000 7fa38b7fe640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 0x7fa374071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa39415fcc0 tx=0x7fa37c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.423+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa384021020 con 0x7fa39406ffd0 2024-05-17T21:58:20.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.615+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3940b41a0 con 0x7fa39406ffd0 2024-05-17T21:58:20.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.615+0000 7fa3917fa640 1 -- 172.21.15.83:0/1414855973 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fa384005480 con 0x7fa39406ffd0 2024-05-17T21:58:20.610 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:20.610 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 msgr2=0x7fa374071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 0x7fa374071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa39415fcc0 tx=0x7fa37c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 msgr2=0x7fa39413ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c3f0 tx=0x7fa38400c8b0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 shutdown_connections 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3941605a0 0x7fa39413b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa37406f430 0x7fa374071940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3941461c0 0x7fa39415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 --2- 172.21.15.83:0/1414855973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa39406ffd0 0x7fa39413ed60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 >> 172.21.15.83:0/1414855973 conn(0x7fa39406cd90 msgr2=0x7fa3940d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 shutdown_connections 2024-05-17T21:58:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.619+0000 7fa39a6aa640 1 -- 172.21.15.83:0/1414855973 wait complete. 2024-05-17T21:58:20.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.767+0000 7fe15da83640 1 Processor -- start 2024-05-17T21:58:20.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.767+0000 7fe15da83640 1 -- start start 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe15da83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe158073540 0x7fe158168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe15da83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15815e790 con 0x7fe1580d1040 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe15da83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15815ea40 con 0x7fe158144080 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe15da83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe15815ed20 con 0x7fe158073540 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1577fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1580d1040 0x7fe15815e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41616/0 (socket says 172.21.15.83:41616) 2024-05-17T21:58:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1577fe640 1 -- 172.21.15.83:0/28819180 learned_addr learned my addr 172.21.15.83:0/28819180 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe157fff640 1 --2- 172.21.15.83:0/28819180 >> v2:172.21.15.155:3300/0 conn(0x7fe158073540 0x7fe158168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636298079 0 0) 0x7fe15815e790 con 0x7fe1580d1040 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe144003530 con 0x7fe1580d1040 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2847942459 0 0) 0x7fe144003530 con 0x7fe1580d1040 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 >> v2:172.21.15.155:3300/0 conn(0x7fe158073540 msgr2=0x7fe158168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 --2- 172.21.15.83:0/28819180 >> v2:172.21.15.155:3300/0 conn(0x7fe158073540 0x7fe158168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 >> v1:172.21.15.155:6789/0 conn(0x7fe158144080 legacy=0x7fe158072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15815f000 con 0x7fe1580d1040 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe148001700 con 0x7fe1580d1040 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 >> v1:172.21.15.83:6789/0 conn(0x7fe1580d1040 legacy=0x7fe15815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe144003a60 0x7fe144003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 0x7fe144004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144005260 0x7fe14400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe15815ed20 con 0x7fe144004810 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe14400c1f0 con 0x7fe144005260 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe14400c3f0 con 0x7fe144003a60 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 0x7fe144004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1577fe640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe144003a60 0x7fe144003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe157fff640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144005260 0x7fe14400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 -- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe144003a60 msgr2=0x7fe144003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe144003a60 0x7fe144003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 -- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144005260 msgr2=0x7fe14400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144005260 0x7fe14400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe156ffd640 1 -- 172.21.15.83:0/28819180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe144003530 con 0x7fe144004810 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe157fff640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144005260 0x7fe14400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:20.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe1577fe640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe144003a60 0x7fe144003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.771+0000 7fe157fff640 1 -- 172.21.15.83:0/28819180 reap_dead start 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe156ffd640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 0x7fe144004c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe1580ceb70 tx=0x7fe14001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8db7a445f1a896 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400013e0 con 0x7fe144004810 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1400052d0 con 0x7fe144004810 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe1567fc640 1 -- 172.21.15.83:0/28819180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140003840 con 0x7fe144004810 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 -- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 msgr2=0x7fe144004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 0x7fe144004c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe1580ceb70 tx=0x7fe14001c370 comp rx=0 tx=0).stop 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 -- 172.21.15.83:0/28819180 shutdown_connections 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 --2- 172.21.15.83:0/28819180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe144004810 0x7fe144004c20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 -- 172.21.15.83:0/28819180 >> 172.21.15.83:0/28819180 conn(0x7fe15806cd90 msgr2=0x7fe1580b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 -- 172.21.15.83:0/28819180 shutdown_connections 2024-05-17T21:58:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.775+0000 7fe15da83640 1 -- 172.21.15.83:0/28819180 wait complete. 2024-05-17T21:58:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 Processor -- start 2024-05-17T21:58:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- start start 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144004810 0x7fe15815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1580af670 0x7fe1580b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe158144490 con 0x7fe1581552d0 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe15806f210 con 0x7fe144004810 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe15815e080 con 0x7fe1580af670 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51742/0 (socket says 172.21.15.83:51742) 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 -- 172.21.15.83:0/1701386940 learned_addr learned my addr 172.21.15.83:0/1701386940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe157fff640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1580af670 0x7fe1580b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe1577fe640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144004810 0x7fe15815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 -- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1580af670 msgr2=0x7fe1580b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1580af670 0x7fe1580b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 -- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144004810 msgr2=0x7fe15815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144004810 0x7fe15815f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 -- 172.21.15.83:0/1701386940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe158146dd0 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe156ffd640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe14001aa70 tx=0x7fe144005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400013c0 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe140005480 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140007020 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15815f240 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe158147350 con 0x7fe1581552d0 2024-05-17T21:58:20.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.779+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15815e3b0 con 0x7fe1581552d0 2024-05-17T21:58:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.783+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1400bc060 con 0x7fe1581552d0 2024-05-17T21:58:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.783+0000 7fe14ffff640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 0x7fe138071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.783+0000 7fe1577fe640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 0x7fe138071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.783+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe140128cc0 con 0x7fe1581552d0 2024-05-17T21:58:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.783+0000 7fe1577fe640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 0x7fe138071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe148001870 tx=0x7fe148001120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.787+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe1400160d0 con 0x7fe1581552d0 2024-05-17T21:58:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.943+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe15815e3b0 con 0x7fe1581552d0 2024-05-17T21:58:20.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.943+0000 7fe14ffff640 1 -- 172.21.15.83:0/1701386940 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16645 (secure 0 0 0) 0x7fe1400f5c90 con 0x7fe1581552d0 2024-05-17T21:58:20.939 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:20.940 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:18.200289+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:14.593534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:15.504714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:20.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 msgr2=0x7fe138071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 0x7fe138071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe148001870 tx=0x7fe148001120 comp rx=0 tx=0).stop 2024-05-17T21:58:20.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 msgr2=0x7fe1580af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe14001aa70 tx=0x7fe144005840 comp rx=0 tx=0).stop 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 shutdown_connections 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1580af670 0x7fe1580b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1581552d0 0x7fe1580af130 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe13806f430 0x7fe138071940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 --2- 172.21.15.83:0/1701386940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144004810 0x7fe15815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 >> 172.21.15.83:0/1701386940 conn(0x7fe15806cd90 msgr2=0x7fe1580b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 shutdown_connections 2024-05-17T21:58:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:20.947+0000 7fe15da83640 1 -- 172.21.15.83:0/1701386940 wait complete. 2024-05-17T21:58:20.959 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T21:58:20.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:58:21.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.111+0000 7f8b43237640 1 Processor -- start 2024-05-17T21:58:21.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 -- start start 2024-05-17T21:58:21.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b3c15fe20 0x7f8b3c160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c072e30 0x7f8b3c073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b3c073780 con 0x7f8b3c15d2f0 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8b3c15c830 con 0x7f8b3c15fe20 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b43237640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c1608a0 con 0x7f8b3c072e30 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b40fd3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8b3c15d2f0 0x7f8b3c159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41622/0 (socket says 172.21.15.83:41622) 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b40fd3640 1 -- 172.21.15.83:0/2864092461 learned_addr learned my addr 172.21.15.83:0/2864092461 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/2864092461 >> v2:172.21.15.83:3300/0 conn(0x7f8b3c15fe20 0x7f8b3c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3b7fe640 1 -- 172.21.15.83:0/2864092461 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3729878089 0 0) 0x7f8b2c001120 con 0x7f8b3c15d2f0 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3b7fe640 1 -- 172.21.15.83:0/2864092461 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b24003530 con 0x7f8b3c15d2f0 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b417d4640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c072e30 0x7f8b3c073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3bfff640 1 -- 172.21.15.83:0/2864092461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c072e30 msgr2=0x7f8b3c073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c072e30 0x7f8b3c073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3bfff640 1 -- 172.21.15.83:0/2864092461 >> v1:172.21.15.83:6789/0 conn(0x7f8b3c15d2f0 legacy=0x7f8b3c159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.115+0000 7f8b3bfff640 1 -- 172.21.15.83:0/2864092461 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c160b80 con 0x7f8b3c15fe20 2024-05-17T21:58:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c160230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000a960 tx=0x7f8b3001d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dca3d96fa31434 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b3b7fe640 1 -- 172.21.15.83:0/2864092461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300b3070 con 0x7f8b3c15fe20 2024-05-17T21:58:21.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b3b7fe640 1 -- 172.21.15.83:0/2864092461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b300a6740 con 0x7f8b3c15fe20 2024-05-17T21:58:21.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b3b7fe640 1 -- 172.21.15.83:0/2864092461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000b040 con 0x7f8b3c15fe20 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 -- 172.21.15.83:0/2864092461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c15fe20 msgr2=0x7f8b3c160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c160230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000a960 tx=0x7f8b3001d7e0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 -- 172.21.15.83:0/2864092461 shutdown_connections 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c072e30 0x7f8b3c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 --2- 172.21.15.83:0/2864092461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c160230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 -- 172.21.15.83:0/2864092461 >> 172.21.15.83:0/2864092461 conn(0x7f8b3c06cd90 msgr2=0x7f8b3c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 -- 172.21.15.83:0/2864092461 shutdown_connections 2024-05-17T21:58:21.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.119+0000 7f8b43237640 1 -- 172.21.15.83:0/2864092461 wait complete. 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 Processor -- start 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- start start 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c15d2f0 0x7f8b3c0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c0b04f0 con 0x7f8b3c072e30 2024-05-17T21:58:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c0b0370 con 0x7f8b3c15fe20 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c0b0670 con 0x7f8b3c15d2f0 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51758/0 (socket says 172.21.15.83:51758) 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 -- 172.21.15.83:0/318863642 learned_addr learned my addr 172.21.15.83:0/318863642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c15d2f0 0x7f8b3c0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b417d4640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 -- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c15d2f0 msgr2=0x7f8b3c0aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c15d2f0 0x7f8b3c0aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 -- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b3c15fe20 msgr2=0x7f8b3c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c0d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 -- 172.21.15.83:0/318863642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c157bc0 con 0x7f8b3c072e30 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b40fd3640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c700 tx=0x7f8b2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c017070 con 0x7f8b3c072e30 2024-05-17T21:58:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c001d40 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3c157ea0 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c005cc0 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3c152630 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.123+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3c073240 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b2c01a070 con 0x7f8b3c072e30 2024-05-17T21:58:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f8b397fa640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 0x7f8b28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c08e470 con 0x7f8b3c072e30 2024-05-17T21:58:21.122 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f3579108640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:58:21.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 0x7f8b28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.127+0000 7f8b3bfff640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 0x7f8b28071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b300b0000 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.131+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8b3c073780 con 0x7f8b3c072e30 2024-05-17T21:58:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.279+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b3c1f6f10 con 0x7f8b2806f400 2024-05-17T21:58:21.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.287+0000 7f8b397fa640 1 -- 172.21.15.83:0/318863642 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f8b3c1f6f10 con 0x7f8b2806f400 2024-05-17T21:58:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 msgr2=0x7f8b28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 0x7f8b28071910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b300b0000 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 msgr2=0x7f8b3c1520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c700 tx=0x7f8b2c00cbc0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 shutdown_connections 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b2806f400 0x7f8b28071910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b3c15fe20 0x7f8b3c0d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b3c15d2f0 0x7f8b3c0aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 --2- 172.21.15.83:0/318863642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3c072e30 0x7f8b3c1520f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 >> 172.21.15.83:0/318863642 conn(0x7f8b3c06cd90 msgr2=0x7f8b3c159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 shutdown_connections 2024-05-17T21:58:21.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.291+0000 7f8b43237640 1 -- 172.21.15.83:0/318863642 wait complete. 2024-05-17T21:58:21.303 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-05-17T21:58:20.982266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220080,"kb_used_data":3884,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754640,"statfs":{"total":773094113280,"available":772868751360,"internally_reserved":0,"allocated":3977216,"data_stored":2227064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.836966"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.250745+0000","last_change":"2024-05-17T21:58:17.231532+0000","last_active":"2024-05-17T21:58:17.250745+0000","last_peered":"2024-05-17T21:58:17.250745+0000","last_clean":"2024-05-17T21:58:17.250745+0000","last_became_active":"2024-05-17T21:58:17.230885+0000","last_became_peered":"2024-05-17T21:58:17.230885+0000","last_unstale":"2024-05-17T21:58:17.250745+0000","last_undegraded":"2024-05-17T21:58:17.250745+0000","last_fullsized":"2024-05-17T21:58:17.250745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:27.206495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.220164+0000","last_change":"2024-05-17T21:58:17.215826+0000","last_active":"2024-05-17T21:58:17.220164+0000","last_peered":"2024-05-17T21:58:17.220164+0000","last_clean":"2024-05-17T21:58:17.220164+0000","last_became_active":"2024-05-17T21:58:17.215515+0000","last_became_peered":"2024-05-17T21:58:17.215515+0000","last_unstale":"2024-05-17T21:58:17.220164+0000","last_undegraded":"2024-05-17T21:58:17.220164+0000","last_fullsized":"2024-05-17T21:58:17.220164+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215848+0000","last_change":"2024-05-17T21:58:17.219641+0000","last_active":"2024-05-17T21:58:18.215848+0000","last_peered":"2024-05-17T21:58:18.215848+0000","last_clean":"2024-05-17T21:58:18.215848+0000","last_became_active":"2024-05-17T21:58:17.218240+0000","last_became_peered":"2024-05-17T21:58:17.218240+0000","last_unstale":"2024-05-17T21:58:18.215848+0000","last_undegraded":"2024-05-17T21:58:18.215848+0000","last_fullsized":"2024-05-17T21:58:18.215848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213744+0000","last_change":"2024-05-17T21:58:17.245796+0000","last_active":"2024-05-17T21:58:18.213744+0000","last_peered":"2024-05-17T21:58:18.213744+0000","last_clean":"2024-05-17T21:58:18.213744+0000","last_became_active":"2024-05-17T21:58:17.245431+0000","last_became_peered":"2024-05-17T21:58:17.245431+0000","last_unstale":"2024-05-17T21:58:18.213744+0000","last_undegraded":"2024-05-17T21:58:18.213744+0000","last_fullsized":"2024-05-17T21:58:18.213744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214759+0000","last_change":"2024-05-17T21:58:17.213900+0000","last_active":"2024-05-17T21:58:18.214759+0000","last_peered":"2024-05-17T21:58:18.214759+0000","last_clean":"2024-05-17T21:58:18.214759+0000","last_became_active":"2024-05-17T21:58:17.213595+0000","last_became_peered":"2024-05-17T21:58:17.213595+0000","last_unstale":"2024-05-17T21:58:18.214759+0000","last_undegraded":"2024-05-17T21:58:18.214759+0000","last_fullsized":"2024-05-17T21:58:18.214759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:42.482659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.242311+0000","last_change":"2024-05-17T21:58:17.238615+0000","last_active":"2024-05-17T21:58:17.242311+0000","last_peered":"2024-05-17T21:58:17.242311+0000","last_clean":"2024-05-17T21:58:17.242311+0000","last_became_active":"2024-05-17T21:58:17.238244+0000","last_became_peered":"2024-05-17T21:58:17.238244+0000","last_unstale":"2024-05-17T21:58:17.242311+0000","last_undegraded":"2024-05-17T21:58:17.242311+0000","last_fullsized":"2024-05-17T21:58:17.242311+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253940+0000","last_change":"2024-05-17T21:58:17.238596+0000","last_active":"2024-05-17T21:58:17.253940+0000","last_peered":"2024-05-17T21:58:17.253940+0000","last_clean":"2024-05-17T21:58:17.253940+0000","last_became_active":"2024-05-17T21:58:17.235764+0000","last_became_peered":"2024-05-17T21:58:17.235764+0000","last_unstale":"2024-05-17T21:58:17.253940+0000","last_undegraded":"2024-05-17T21:58:17.253940+0000","last_fullsized":"2024-05-17T21:58:17.253940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:08.532377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253428+0000","last_change":"2024-05-17T21:58:17.222410+0000","last_active":"2024-05-17T21:58:17.253428+0000","last_peered":"2024-05-17T21:58:17.253428+0000","last_clean":"2024-05-17T21:58:17.253428+0000","last_became_active":"2024-05-17T21:58:17.221463+0000","last_became_peered":"2024-05-17T21:58:17.221463+0000","last_unstale":"2024-05-17T21:58:17.253428+0000","last_undegraded":"2024-05-17T21:58:17.253428+0000","last_fullsized":"2024-05-17T21:58:17.253428+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711578+0000","last_change":"2024-05-17T21:58:17.219945+0000","last_active":"2024-05-17T21:58:18.711578+0000","last_peered":"2024-05-17T21:58:18.711578+0000","last_clean":"2024-05-17T21:58:18.711578+0000","last_became_active":"2024-05-17T21:58:17.212404+0000","last_became_peered":"2024-05-17T21:58:17.212404+0000","last_unstale":"2024-05-17T21:58:18.711578+0000","last_undegraded":"2024-05-17T21:58:18.711578+0000","last_fullsized":"2024-05-17T21:58:18.711578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214526+0000","last_change":"2024-05-17T21:58:17.220337+0000","last_active":"2024-05-17T21:58:18.214526+0000","last_peered":"2024-05-17T21:58:18.214526+0000","last_clean":"2024-05-17T21:58:18.214526+0000","last_became_active":"2024-05-17T21:58:17.218394+0000","last_became_peered":"2024-05-17T21:58:17.218394+0000","last_unstale":"2024-05-17T21:58:18.214526+0000","last_undegraded":"2024-05-17T21:58:18.214526+0000","last_fullsized":"2024-05-17T21:58:18.214526+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253843+0000","last_change":"2024-05-17T21:58:17.235549+0000","last_active":"2024-05-17T21:58:17.253843+0000","last_peered":"2024-05-17T21:58:17.253843+0000","last_clean":"2024-05-17T21:58:17.253843+0000","last_became_active":"2024-05-17T21:58:17.234843+0000","last_became_peered":"2024-05-17T21:58:17.234843+0000","last_unstale":"2024-05-17T21:58:17.253843+0000","last_undegraded":"2024-05-17T21:58:17.253843+0000","last_fullsized":"2024-05-17T21:58:17.253843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:38.897601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216160+0000","last_change":"2024-05-17T21:58:17.223144+0000","last_active":"2024-05-17T21:58:18.216160+0000","last_peered":"2024-05-17T21:58:18.216160+0000","last_clean":"2024-05-17T21:58:18.216160+0000","last_became_active":"2024-05-17T21:58:17.222360+0000","last_became_peered":"2024-05-17T21:58:17.222360+0000","last_unstale":"2024-05-17T21:58:18.216160+0000","last_undegraded":"2024-05-17T21:58:18.216160+0000","last_fullsized":"2024-05-17T21:58:18.216160+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215157+0000","last_change":"2024-05-17T21:58:17.218946+0000","last_active":"2024-05-17T21:58:18.215157+0000","last_peered":"2024-05-17T21:58:18.215157+0000","last_clean":"2024-05-17T21:58:18.215157+0000","last_became_active":"2024-05-17T21:58:17.218234+0000","last_became_peered":"2024-05-17T21:58:17.218234+0000","last_unstale":"2024-05-17T21:58:18.215157+0000","last_undegraded":"2024-05-17T21:58:18.215157+0000","last_fullsized":"2024-05-17T21:58:18.215157+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:24.371583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.218890+0000","last_change":"2024-05-17T21:58:17.215489+0000","last_active":"2024-05-17T21:58:17.218890+0000","last_peered":"2024-05-17T21:58:17.218890+0000","last_clean":"2024-05-17T21:58:17.218890+0000","last_became_active":"2024-05-17T21:58:17.214842+0000","last_became_peered":"2024-05-17T21:58:17.214842+0000","last_unstale":"2024-05-17T21:58:17.218890+0000","last_undegraded":"2024-05-17T21:58:17.218890+0000","last_fullsized":"2024-05-17T21:58:17.218890+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215952+0000","last_change":"2024-05-17T21:58:17.210803+0000","last_active":"2024-05-17T21:58:18.215952+0000","last_peered":"2024-05-17T21:58:18.215952+0000","last_clean":"2024-05-17T21:58:18.215952+0000","last_became_active":"2024-05-17T21:58:17.209984+0000","last_became_peered":"2024-05-17T21:58:17.209984+0000","last_unstale":"2024-05-17T21:58:18.215952+0000","last_undegraded":"2024-05-17T21:58:18.215952+0000","last_fullsized":"2024-05-17T21:58:18.215952+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253794+0000","last_change":"2024-05-17T21:58:17.236222+0000","last_active":"2024-05-17T21:58:17.253794+0000","last_peered":"2024-05-17T21:58:17.253794+0000","last_clean":"2024-05-17T21:58:17.253794+0000","last_became_active":"2024-05-17T21:58:17.223206+0000","last_became_peered":"2024-05-17T21:58:17.223206+0000","last_unstale":"2024-05-17T21:58:17.253794+0000","last_undegraded":"2024-05-17T21:58:17.253794+0000","last_fullsized":"2024-05-17T21:58:17.253794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:39:56.935731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253562+0000","last_change":"2024-05-17T21:58:17.222192+0000","last_active":"2024-05-17T21:58:17.253562+0000","last_peered":"2024-05-17T21:58:17.253562+0000","last_clean":"2024-05-17T21:58:17.253562+0000","last_became_active":"2024-05-17T21:58:17.221388+0000","last_became_peered":"2024-05-17T21:58:17.221388+0000","last_unstale":"2024-05-17T21:58:17.253562+0000","last_undegraded":"2024-05-17T21:58:17.253562+0000","last_fullsized":"2024-05-17T21:58:17.253562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229355+0000","last_change":"2024-05-17T21:58:17.224969+0000","last_active":"2024-05-17T21:58:17.229355+0000","last_peered":"2024-05-17T21:58:17.229355+0000","last_clean":"2024-05-17T21:58:17.229355+0000","last_became_active":"2024-05-17T21:58:17.224588+0000","last_became_peered":"2024-05-17T21:58:17.224588+0000","last_unstale":"2024-05-17T21:58:17.229355+0000","last_undegraded":"2024-05-17T21:58:17.229355+0000","last_fullsized":"2024-05-17T21:58:17.229355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.230797+0000","last_change":"2024-05-17T21:58:17.225311+0000","last_active":"2024-05-17T21:58:17.230797+0000","last_peered":"2024-05-17T21:58:17.230797+0000","last_clean":"2024-05-17T21:58:17.230797+0000","last_became_active":"2024-05-17T21:58:17.224988+0000","last_became_peered":"2024-05-17T21:58:17.224988+0000","last_unstale":"2024-05-17T21:58:17.230797+0000","last_undegraded":"2024-05-17T21:58:17.230797+0000","last_fullsized":"2024-05-17T21:58:17.230797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215525+0000","last_change":"2024-05-17T21:58:17.206972+0000","last_active":"2024-05-17T21:58:18.215525+0000","last_peered":"2024-05-17T21:58:18.215525+0000","last_clean":"2024-05-17T21:58:18.215525+0000","last_became_active":"2024-05-17T21:58:17.206659+0000","last_became_peered":"2024-05-17T21:58:17.206659+0000","last_unstale":"2024-05-17T21:58:18.215525+0000","last_undegraded":"2024-05-17T21:58:18.215525+0000","last_fullsized":"2024-05-17T21:58:18.215525+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:29.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710638+0000","last_change":"2024-05-17T21:58:17.232619+0000","last_active":"2024-05-17T21:58:18.710638+0000","last_peered":"2024-05-17T21:58:18.710638+0000","last_clean":"2024-05-17T21:58:18.710638+0000","last_became_active":"2024-05-17T21:58:17.212476+0000","last_became_peered":"2024-05-17T21:58:17.212476+0000","last_unstale":"2024-05-17T21:58:18.710638+0000","last_undegraded":"2024-05-17T21:58:18.710638+0000","last_fullsized":"2024-05-17T21:58:18.710638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:47.375240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710747+0000","last_change":"2024-05-17T21:58:17.234499+0000","last_active":"2024-05-17T21:58:18.710747+0000","last_peered":"2024-05-17T21:58:18.710747+0000","last_clean":"2024-05-17T21:58:18.710747+0000","last_became_active":"2024-05-17T21:58:17.232854+0000","last_became_peered":"2024-05-17T21:58:17.232854+0000","last_unstale":"2024-05-17T21:58:18.710747+0000","last_undegraded":"2024-05-17T21:58:18.710747+0000","last_fullsized":"2024-05-17T21:58:18.710747+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.237062+0000","last_change":"2024-05-17T21:58:17.235353+0000","last_active":"2024-05-17T21:58:17.237062+0000","last_peered":"2024-05-17T21:58:17.237062+0000","last_clean":"2024-05-17T21:58:17.237062+0000","last_became_active":"2024-05-17T21:58:17.235030+0000","last_became_peered":"2024-05-17T21:58:17.235030+0000","last_unstale":"2024-05-17T21:58:17.237062+0000","last_undegraded":"2024-05-17T21:58:17.237062+0000","last_fullsized":"2024-05-17T21:58:17.237062+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:38.897601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217102+0000","last_change":"2024-05-17T21:58:17.200790+0000","last_active":"2024-05-17T21:58:18.217102+0000","last_peered":"2024-05-17T21:58:18.217102+0000","last_clean":"2024-05-17T21:58:18.217102+0000","last_became_active":"2024-05-17T21:58:17.199889+0000","last_became_peered":"2024-05-17T21:58:17.199889+0000","last_unstale":"2024-05-17T21:58:18.217102+0000","last_undegraded":"2024-05-17T21:58:18.217102+0000","last_fullsized":"2024-05-17T21:58:18.217102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:32:45.779428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213904+0000","last_change":"2024-05-17T21:58:17.240587+0000","last_active":"2024-05-17T21:58:18.213904+0000","last_peered":"2024-05-17T21:58:18.213904+0000","last_clean":"2024-05-17T21:58:18.213904+0000","last_became_active":"2024-05-17T21:58:17.240281+0000","last_became_peered":"2024-05-17T21:58:17.240281+0000","last_unstale":"2024-05-17T21:58:18.213904+0000","last_undegraded":"2024-05-17T21:58:18.213904+0000","last_fullsized":"2024-05-17T21:58:18.213904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.244944+0000","last_change":"2024-05-17T21:58:17.240665+0000","last_active":"2024-05-17T21:58:17.244944+0000","last_peered":"2024-05-17T21:58:17.244944+0000","last_clean":"2024-05-17T21:58:17.244944+0000","last_became_active":"2024-05-17T21:58:17.240205+0000","last_became_peered":"2024-05-17T21:58:17.240205+0000","last_unstale":"2024-05-17T21:58:17.244944+0000","last_undegraded":"2024-05-17T21:58:17.244944+0000","last_fullsized":"2024-05-17T21:58:17.244944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214133+0000","last_change":"2024-05-17T21:58:17.244345+0000","last_active":"2024-05-17T21:58:18.214133+0000","last_peered":"2024-05-17T21:58:18.214133+0000","last_clean":"2024-05-17T21:58:18.214133+0000","last_became_active":"2024-05-17T21:58:17.244065+0000","last_became_peered":"2024-05-17T21:58:17.244065+0000","last_unstale":"2024-05-17T21:58:18.214133+0000","last_undegraded":"2024-05-17T21:58:18.214133+0000","last_fullsized":"2024-05-17T21:58:18.214133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216245+0000","last_change":"2024-05-17T21:58:17.228068+0000","last_active":"2024-05-17T21:58:18.216245+0000","last_peered":"2024-05-17T21:58:18.216245+0000","last_clean":"2024-05-17T21:58:18.216245+0000","last_became_active":"2024-05-17T21:58:17.227234+0000","last_became_peered":"2024-05-17T21:58:17.227234+0000","last_unstale":"2024-05-17T21:58:18.216245+0000","last_undegraded":"2024-05-17T21:58:18.216245+0000","last_fullsized":"2024-05-17T21:58:18.216245+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:17.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711232+0000","last_change":"2024-05-17T21:58:17.234485+0000","last_active":"2024-05-17T21:58:18.711232+0000","last_peered":"2024-05-17T21:58:18.711232+0000","last_clean":"2024-05-17T21:58:18.711232+0000","last_became_active":"2024-05-17T21:58:17.232968+0000","last_became_peered":"2024-05-17T21:58:17.232968+0000","last_unstale":"2024-05-17T21:58:18.711232+0000","last_undegraded":"2024-05-17T21:58:18.711232+0000","last_fullsized":"2024-05-17T21:58:18.711232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.247682+0000","last_change":"2024-05-17T21:58:17.217853+0000","last_active":"2024-05-17T21:58:17.247682+0000","last_peered":"2024-05-17T21:58:17.247682+0000","last_clean":"2024-05-17T21:58:17.247682+0000","last_became_active":"2024-05-17T21:58:17.216989+0000","last_became_peered":"2024-05-17T21:58:17.216989+0000","last_unstale":"2024-05-17T21:58:17.247682+0000","last_undegraded":"2024-05-17T21:58:17.247682+0000","last_fullsized":"2024-05-17T21:58:17.247682+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:27.206495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.226275+0000","last_change":"2024-05-17T21:58:17.215039+0000","last_active":"2024-05-17T21:58:17.226275+0000","last_peered":"2024-05-17T21:58:17.226275+0000","last_clean":"2024-05-17T21:58:17.226275+0000","last_became_active":"2024-05-17T21:58:17.214238+0000","last_became_peered":"2024-05-17T21:58:17.214238+0000","last_unstale":"2024-05-17T21:58:17.226275+0000","last_undegraded":"2024-05-17T21:58:17.226275+0000","last_fullsized":"2024-05-17T21:58:17.226275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:57.616631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.256306+0000","last_change":"2024-05-17T21:58:17.249463+0000","last_active":"2024-05-17T21:58:17.256306+0000","last_peered":"2024-05-17T21:58:17.256306+0000","last_clean":"2024-05-17T21:58:17.256306+0000","last_became_active":"2024-05-17T21:58:17.246383+0000","last_became_peered":"2024-05-17T21:58:17.246383+0000","last_unstale":"2024-05-17T21:58:17.256306+0000","last_undegraded":"2024-05-17T21:58:17.256306+0000","last_fullsized":"2024-05-17T21:58:17.256306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.242182+0000","last_change":"2024-05-17T21:58:17.240722+0000","last_active":"2024-05-17T21:58:17.242182+0000","last_peered":"2024-05-17T21:58:17.242182+0000","last_clean":"2024-05-17T21:58:17.242182+0000","last_became_active":"2024-05-17T21:58:17.240358+0000","last_became_peered":"2024-05-17T21:58:17.240358+0000","last_unstale":"2024-05-17T21:58:17.242182+0000","last_undegraded":"2024-05-17T21:58:17.242182+0000","last_fullsized":"2024-05-17T21:58:17.242182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:48.736435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.250811+0000","last_change":"2024-05-17T21:58:16.192574+0000","last_active":"2024-05-17T21:58:17.250811+0000","last_peered":"2024-05-17T21:58:17.250811+0000","last_clean":"2024-05-17T21:58:17.250811+0000","last_became_active":"2024-05-17T21:58:16.191691+0000","last_became_peered":"2024-05-17T21:58:16.191691+0000","last_unstale":"2024-05-17T21:58:17.250811+0000","last_undegraded":"2024-05-17T21:58:17.250811+0000","last_fullsized":"2024-05-17T21:58:17.250811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:44.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.252479+0000","last_change":"2024-05-17T21:58:17.250324+0000","last_active":"2024-05-17T21:58:17.252479+0000","last_peered":"2024-05-17T21:58:17.252479+0000","last_clean":"2024-05-17T21:58:17.252479+0000","last_became_active":"2024-05-17T21:58:17.248055+0000","last_became_peered":"2024-05-17T21:58:17.248055+0000","last_unstale":"2024-05-17T21:58:17.252479+0000","last_undegraded":"2024-05-17T21:58:17.252479+0000","last_fullsized":"2024-05-17T21:58:17.252479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.254089+0000","last_change":"2024-05-17T21:58:16.181268+0000","last_active":"2024-05-17T21:58:17.254089+0000","last_peered":"2024-05-17T21:58:17.254089+0000","last_clean":"2024-05-17T21:58:17.254089+0000","last_became_active":"2024-05-17T21:58:16.180668+0000","last_became_peered":"2024-05-17T21:58:16.180668+0000","last_unstale":"2024-05-17T21:58:17.254089+0000","last_undegraded":"2024-05-17T21:58:17.254089+0000","last_fullsized":"2024-05-17T21:58:17.254089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711051+0000","last_change":"2024-05-17T21:58:17.240237+0000","last_active":"2024-05-17T21:58:18.711051+0000","last_peered":"2024-05-17T21:58:18.711051+0000","last_clean":"2024-05-17T21:58:18.711051+0000","last_became_active":"2024-05-17T21:58:17.231690+0000","last_became_peered":"2024-05-17T21:58:17.231690+0000","last_unstale":"2024-05-17T21:58:18.711051+0000","last_undegraded":"2024-05-17T21:58:18.711051+0000","last_fullsized":"2024-05-17T21:58:18.711051+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.234485+0000","last_change":"2024-05-17T21:58:16.183432+0000","last_active":"2024-05-17T21:58:17.234485+0000","last_peered":"2024-05-17T21:58:17.234485+0000","last_clean":"2024-05-17T21:58:17.234485+0000","last_became_active":"2024-05-17T21:58:16.182130+0000","last_became_peered":"2024-05-17T21:58:16.182130+0000","last_unstale":"2024-05-17T21:58:17.234485+0000","last_undegraded":"2024-05-17T21:58:17.234485+0000","last_fullsized":"2024-05-17T21:58:17.234485+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:46.364246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214310+0000","last_change":"2024-05-17T21:58:17.221739+0000","last_active":"2024-05-17T21:58:18.214310+0000","last_peered":"2024-05-17T21:58:18.214310+0000","last_clean":"2024-05-17T21:58:18.214310+0000","last_became_active":"2024-05-17T21:58:17.220418+0000","last_became_peered":"2024-05-17T21:58:17.220418+0000","last_unstale":"2024-05-17T21:58:18.214310+0000","last_undegraded":"2024-05-17T21:58:18.214310+0000","last_fullsized":"2024-05-17T21:58:18.214310+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215422+0000","last_change":"2024-05-17T21:58:16.166687+0000","last_active":"2024-05-17T21:58:18.215422+0000","last_peered":"2024-05-17T21:58:18.215422+0000","last_clean":"2024-05-17T21:58:18.215422+0000","last_became_active":"2024-05-17T21:58:16.164695+0000","last_became_peered":"2024-05-17T21:58:16.164695+0000","last_unstale":"2024-05-17T21:58:18.215422+0000","last_undegraded":"2024-05-17T21:58:18.215422+0000","last_fullsized":"2024-05-17T21:58:18.215422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:59.418642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.254103+0000","last_change":"2024-05-17T21:58:17.247347+0000","last_active":"2024-05-17T21:58:17.254103+0000","last_peered":"2024-05-17T21:58:17.254103+0000","last_clean":"2024-05-17T21:58:17.254103+0000","last_became_active":"2024-05-17T21:58:17.246687+0000","last_became_peered":"2024-05-17T21:58:17.246687+0000","last_unstale":"2024-05-17T21:58:17.254103+0000","last_undegraded":"2024-05-17T21:58:17.254103+0000","last_fullsized":"2024-05-17T21:58:17.254103+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:47.375240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711451+0000","last_change":"2024-05-17T21:58:16.196750+0000","last_active":"2024-05-17T21:58:18.711451+0000","last_peered":"2024-05-17T21:58:18.711451+0000","last_clean":"2024-05-17T21:58:18.711451+0000","last_became_active":"2024-05-17T21:58:16.195229+0000","last_became_peered":"2024-05-17T21:58:16.195229+0000","last_unstale":"2024-05-17T21:58:18.711451+0000","last_undegraded":"2024-05-17T21:58:18.711451+0000","last_fullsized":"2024-05-17T21:58:18.711451+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:23.360589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215819+0000","last_change":"2024-05-17T21:58:17.230033+0000","last_active":"2024-05-17T21:58:18.215819+0000","last_peered":"2024-05-17T21:58:18.215819+0000","last_clean":"2024-05-17T21:58:18.215819+0000","last_became_active":"2024-05-17T21:58:17.229201+0000","last_became_peered":"2024-05-17T21:58:17.229201+0000","last_unstale":"2024-05-17T21:58:18.215819+0000","last_undegraded":"2024-05-17T21:58:18.215819+0000","last_fullsized":"2024-05-17T21:58:18.215819+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710735+0000","last_change":"2024-05-17T21:58:16.187118+0000","last_active":"2024-05-17T21:58:18.710735+0000","last_peered":"2024-05-17T21:58:18.710735+0000","last_clean":"2024-05-17T21:58:18.710735+0000","last_became_active":"2024-05-17T21:58:16.186129+0000","last_became_peered":"2024-05-17T21:58:16.186129+0000","last_unstale":"2024-05-17T21:58:18.710735+0000","last_undegraded":"2024-05-17T21:58:18.710735+0000","last_fullsized":"2024-05-17T21:58:18.710735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:41.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710336+0000","last_change":"2024-05-17T21:58:17.231849+0000","last_active":"2024-05-17T21:58:18.710336+0000","last_peered":"2024-05-17T21:58:18.710336+0000","last_clean":"2024-05-17T21:58:18.710336+0000","last_became_active":"2024-05-17T21:58:17.225139+0000","last_became_peered":"2024-05-17T21:58:17.225139+0000","last_unstale":"2024-05-17T21:58:18.710336+0000","last_undegraded":"2024-05-17T21:58:18.710336+0000","last_fullsized":"2024-05-17T21:58:18.710336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:36.155454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.214988+0000","last_change":"2024-05-17T21:58:16.174650+0000","last_active":"2024-05-17T21:58:17.214988+0000","last_peered":"2024-05-17T21:58:17.214988+0000","last_clean":"2024-05-17T21:58:17.214988+0000","last_became_active":"2024-05-17T21:58:16.174080+0000","last_became_peered":"2024-05-17T21:58:16.174080+0000","last_unstale":"2024-05-17T21:58:17.214988+0000","last_undegraded":"2024-05-17T21:58:17.214988+0000","last_fullsized":"2024-05-17T21:58:17.214988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:39:55.924737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213714+0000","last_change":"2024-05-17T21:58:17.238673+0000","last_active":"2024-05-17T21:58:18.213714+0000","last_peered":"2024-05-17T21:58:18.213714+0000","last_clean":"2024-05-17T21:58:18.213714+0000","last_became_active":"2024-05-17T21:58:17.238376+0000","last_became_peered":"2024-05-17T21:58:17.238376+0000","last_unstale":"2024-05-17T21:58:18.213714+0000","last_undegraded":"2024-05-17T21:58:18.213714+0000","last_fullsized":"2024-05-17T21:58:18.213714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711539+0000","last_change":"2024-05-17T21:58:16.195135+0000","last_active":"2024-05-17T21:58:18.711539+0000","last_peered":"2024-05-17T21:58:18.711539+0000","last_clean":"2024-05-17T21:58:18.711539+0000","last_became_active":"2024-05-17T21:58:16.188660+0000","last_became_peered":"2024-05-17T21:58:16.188660+0000","last_unstale":"2024-05-17T21:58:18.711539+0000","last_undegraded":"2024-05-17T21:58:18.711539+0000","last_fullsized":"2024-05-17T21:58:18.711539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:38.551838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216632+0000","last_change":"2024-05-17T21:58:17.221595+0000","last_active":"2024-05-17T21:58:18.216632+0000","last_peered":"2024-05-17T21:58:18.216632+0000","last_clean":"2024-05-17T21:58:18.216632+0000","last_became_active":"2024-05-17T21:58:17.220686+0000","last_became_peered":"2024-05-17T21:58:17.220686+0000","last_unstale":"2024-05-17T21:58:18.216632+0000","last_undegraded":"2024-05-17T21:58:18.216632+0000","last_fullsized":"2024-05-17T21:58:18.216632+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:10:12.112834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.215362+0000","last_change":"2024-05-17T21:58:16.190739+0000","last_active":"2024-05-17T21:58:17.215362+0000","last_peered":"2024-05-17T21:58:17.215362+0000","last_clean":"2024-05-17T21:58:17.215362+0000","last_became_active":"2024-05-17T21:58:16.190135+0000","last_became_peered":"2024-05-17T21:58:16.190135+0000","last_unstale":"2024-05-17T21:58:17.215362+0000","last_undegraded":"2024-05-17T21:58:17.215362+0000","last_fullsized":"2024-05-17T21:58:17.215362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:07.521383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229510+0000","last_change":"2024-05-17T21:58:17.227593+0000","last_active":"2024-05-17T21:58:17.229510+0000","last_peered":"2024-05-17T21:58:17.229510+0000","last_clean":"2024-05-17T21:58:17.229510+0000","last_became_active":"2024-05-17T21:58:17.227322+0000","last_became_peered":"2024-05-17T21:58:17.227322+0000","last_unstale":"2024-05-17T21:58:17.229510+0000","last_undegraded":"2024-05-17T21:58:17.229510+0000","last_fullsized":"2024-05-17T21:58:17.229510+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.208602+0000","last_change":"2024-05-17T21:58:16.176414+0000","last_active":"2024-05-17T21:58:17.208602+0000","last_peered":"2024-05-17T21:58:17.208602+0000","last_clean":"2024-05-17T21:58:17.208602+0000","last_became_active":"2024-05-17T21:58:16.175832+0000","last_became_peered":"2024-05-17T21:58:16.175832+0000","last_unstale":"2024-05-17T21:58:17.208602+0000","last_undegraded":"2024-05-17T21:58:17.208602+0000","last_fullsized":"2024-05-17T21:58:17.208602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:44.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.244921+0000","last_change":"2024-05-17T21:58:17.240669+0000","last_active":"2024-05-17T21:58:17.244921+0000","last_peered":"2024-05-17T21:58:17.244921+0000","last_clean":"2024-05-17T21:58:17.244921+0000","last_became_active":"2024-05-17T21:58:17.240243+0000","last_became_peered":"2024-05-17T21:58:17.240243+0000","last_unstale":"2024-05-17T21:58:17.244921+0000","last_undegraded":"2024-05-17T21:58:17.244921+0000","last_fullsized":"2024-05-17T21:58:17.244921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216166+0000","last_change":"2024-05-17T21:58:16.192088+0000","last_active":"2024-05-17T21:58:18.216166+0000","last_peered":"2024-05-17T21:58:18.216166+0000","last_clean":"2024-05-17T21:58:18.216166+0000","last_became_active":"2024-05-17T21:58:16.191207+0000","last_became_peered":"2024-05-17T21:58:16.191207+0000","last_unstale":"2024-05-17T21:58:18.216166+0000","last_undegraded":"2024-05-17T21:58:18.216166+0000","last_fullsized":"2024-05-17T21:58:18.216166+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215772+0000","last_change":"2024-05-17T21:58:17.220429+0000","last_active":"2024-05-17T21:58:18.215772+0000","last_peered":"2024-05-17T21:58:18.215772+0000","last_clean":"2024-05-17T21:58:18.215772+0000","last_became_active":"2024-05-17T21:58:17.219570+0000","last_became_peered":"2024-05-17T21:58:17.219570+0000","last_unstale":"2024-05-17T21:58:18.215772+0000","last_undegraded":"2024-05-17T21:58:18.215772+0000","last_fullsized":"2024-05-17T21:58:18.215772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.255688+0000","last_change":"2024-05-17T21:58:16.182543+0000","last_active":"2024-05-17T21:58:17.255688+0000","last_peered":"2024-05-17T21:58:17.255688+0000","last_clean":"2024-05-17T21:58:17.255688+0000","last_became_active":"2024-05-17T21:58:16.182127+0000","last_became_peered":"2024-05-17T21:58:16.182127+0000","last_unstale":"2024-05-17T21:58:17.255688+0000","last_undegraded":"2024-05-17T21:58:17.255688+0000","last_fullsized":"2024-05-17T21:58:17.255688+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:37.886606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213390+0000","last_change":"2024-05-17T21:58:17.238765+0000","last_active":"2024-05-17T21:58:18.213390+0000","last_peered":"2024-05-17T21:58:18.213390+0000","last_clean":"2024-05-17T21:58:18.213390+0000","last_became_active":"2024-05-17T21:58:17.238371+0000","last_became_peered":"2024-05-17T21:58:17.238371+0000","last_unstale":"2024-05-17T21:58:18.213390+0000","last_undegraded":"2024-05-17T21:58:18.213390+0000","last_fullsized":"2024-05-17T21:58:18.213390+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711479+0000","last_change":"2024-05-17T21:58:16.199687+0000","last_active":"2024-05-17T21:58:18.711479+0000","last_peered":"2024-05-17T21:58:18.711479+0000","last_clean":"2024-05-17T21:58:18.711479+0000","last_became_active":"2024-05-17T21:58:16.196296+0000","last_became_peered":"2024-05-17T21:58:16.196296+0000","last_unstale":"2024-05-17T21:58:18.711479+0000","last_undegraded":"2024-05-17T21:58:18.711479+0000","last_fullsized":"2024-05-17T21:58:18.711479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:26.586942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.225820+0000","last_change":"2024-05-17T21:58:17.219009+0000","last_active":"2024-05-17T21:58:17.225820+0000","last_peered":"2024-05-17T21:58:17.225820+0000","last_clean":"2024-05-17T21:58:17.225820+0000","last_became_active":"2024-05-17T21:58:17.218693+0000","last_became_peered":"2024-05-17T21:58:17.218693+0000","last_unstale":"2024-05-17T21:58:17.225820+0000","last_undegraded":"2024-05-17T21:58:17.225820+0000","last_fullsized":"2024-05-17T21:58:17.225820+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213385+0000","last_change":"2024-05-17T21:58:16.178401+0000","last_active":"2024-05-17T21:58:18.213385+0000","last_peered":"2024-05-17T21:58:18.213385+0000","last_clean":"2024-05-17T21:58:18.213385+0000","last_became_active":"2024-05-17T21:58:16.177954+0000","last_became_peered":"2024-05-17T21:58:16.177954+0000","last_unstale":"2024-05-17T21:58:18.213385+0000","last_undegraded":"2024-05-17T21:58:18.213385+0000","last_fullsized":"2024-05-17T21:58:18.213385+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:57.087708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.259966+0000","last_change":"2024-05-17T21:58:17.249924+0000","last_active":"2024-05-17T21:58:17.259966+0000","last_peered":"2024-05-17T21:58:17.259966+0000","last_clean":"2024-05-17T21:58:17.259966+0000","last_became_active":"2024-05-17T21:58:17.248884+0000","last_became_peered":"2024-05-17T21:58:17.248884+0000","last_unstale":"2024-05-17T21:58:17.259966+0000","last_undegraded":"2024-05-17T21:58:17.259966+0000","last_fullsized":"2024-05-17T21:58:17.259966+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.215191+0000","last_change":"2024-05-17T21:58:16.190511+0000","last_active":"2024-05-17T21:58:17.215191+0000","last_peered":"2024-05-17T21:58:17.215191+0000","last_clean":"2024-05-17T21:58:17.215191+0000","last_became_active":"2024-05-17T21:58:16.190064+0000","last_became_peered":"2024-05-17T21:58:16.190064+0000","last_unstale":"2024-05-17T21:58:17.215191+0000","last_undegraded":"2024-05-17T21:58:17.215191+0000","last_fullsized":"2024-05-17T21:58:17.215191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:35.144459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.252453+0000","last_change":"2024-05-17T21:58:17.247221+0000","last_active":"2024-05-17T21:58:17.252453+0000","last_peered":"2024-05-17T21:58:17.252453+0000","last_clean":"2024-05-17T21:58:17.252453+0000","last_became_active":"2024-05-17T21:58:17.246461+0000","last_became_peered":"2024-05-17T21:58:17.246461+0000","last_unstale":"2024-05-17T21:58:17.252453+0000","last_undegraded":"2024-05-17T21:58:17.252453+0000","last_fullsized":"2024-05-17T21:58:17.252453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.234506+0000","last_change":"2024-05-17T21:58:16.181475+0000","last_active":"2024-05-17T21:58:17.234506+0000","last_peered":"2024-05-17T21:58:17.234506+0000","last_clean":"2024-05-17T21:58:17.234506+0000","last_became_active":"2024-05-17T21:58:16.180870+0000","last_became_peered":"2024-05-17T21:58:16.180870+0000","last_unstale":"2024-05-17T21:58:17.234506+0000","last_undegraded":"2024-05-17T21:58:17.234506+0000","last_fullsized":"2024-05-17T21:58:17.234506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710894+0000","last_change":"2024-05-17T21:58:17.223906+0000","last_active":"2024-05-17T21:58:18.710894+0000","last_peered":"2024-05-17T21:58:18.710894+0000","last_clean":"2024-05-17T21:58:18.710894+0000","last_became_active":"2024-05-17T21:58:17.222931+0000","last_became_peered":"2024-05-17T21:58:17.222931+0000","last_unstale":"2024-05-17T21:58:18.710894+0000","last_undegraded":"2024-05-17T21:58:18.710894+0000","last_fullsized":"2024-05-17T21:58:18.710894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214081+0000","last_change":"2024-05-17T21:58:16.177379+0000","last_active":"2024-05-17T21:58:18.214081+0000","last_peered":"2024-05-17T21:58:18.214081+0000","last_clean":"2024-05-17T21:58:18.214081+0000","last_became_active":"2024-05-17T21:58:16.176504+0000","last_became_peered":"2024-05-17T21:58:16.176504+0000","last_unstale":"2024-05-17T21:58:18.214081+0000","last_undegraded":"2024-05-17T21:58:18.214081+0000","last_fullsized":"2024-05-17T21:58:18.214081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:47.007631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217074+0000","last_change":"2024-05-17T21:58:17.206699+0000","last_active":"2024-05-17T21:58:18.217074+0000","last_peered":"2024-05-17T21:58:18.217074+0000","last_clean":"2024-05-17T21:58:18.217074+0000","last_became_active":"2024-05-17T21:58:17.203627+0000","last_became_peered":"2024-05-17T21:58:17.203627+0000","last_unstale":"2024-05-17T21:58:18.217074+0000","last_undegraded":"2024-05-17T21:58:18.217074+0000","last_fullsized":"2024-05-17T21:58:18.217074+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:36:43.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711371+0000","last_change":"2024-05-17T21:58:16.194836+0000","last_active":"2024-05-17T21:58:18.711371+0000","last_peered":"2024-05-17T21:58:18.711371+0000","last_clean":"2024-05-17T21:58:18.711371+0000","last_became_active":"2024-05-17T21:58:16.188468+0000","last_became_peered":"2024-05-17T21:58:16.188468+0000","last_unstale":"2024-05-17T21:58:18.711371+0000","last_undegraded":"2024-05-17T21:58:18.711371+0000","last_fullsized":"2024-05-17T21:58:18.711371+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213932+0000","last_change":"2024-05-17T21:58:17.246098+0000","last_active":"2024-05-17T21:58:18.213932+0000","last_peered":"2024-05-17T21:58:18.213932+0000","last_clean":"2024-05-17T21:58:18.213932+0000","last_became_active":"2024-05-17T21:58:17.245475+0000","last_became_peered":"2024-05-17T21:58:17.245475+0000","last_unstale":"2024-05-17T21:58:18.213932+0000","last_undegraded":"2024-05-17T21:58:18.213932+0000","last_fullsized":"2024-05-17T21:58:18.213932+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711264+0000","last_change":"2024-05-17T21:58:16.196629+0000","last_active":"2024-05-17T21:58:18.711264+0000","last_peered":"2024-05-17T21:58:18.711264+0000","last_clean":"2024-05-17T21:58:18.711264+0000","last_became_active":"2024-05-17T21:58:16.195479+0000","last_became_peered":"2024-05-17T21:58:16.195479+0000","last_unstale":"2024-05-17T21:58:18.711264+0000","last_undegraded":"2024-05-17T21:58:18.711264+0000","last_fullsized":"2024-05-17T21:58:18.711264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:28.078457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215134+0000","last_change":"2024-05-17T21:58:17.218218+0000","last_active":"2024-05-17T21:58:18.215134+0000","last_peered":"2024-05-17T21:58:18.215134+0000","last_clean":"2024-05-17T21:58:18.215134+0000","last_became_active":"2024-05-17T21:58:17.217919+0000","last_became_peered":"2024-05-17T21:58:17.217919+0000","last_unstale":"2024-05-17T21:58:18.215134+0000","last_undegraded":"2024-05-17T21:58:18.215134+0000","last_fullsized":"2024-05-17T21:58:18.215134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:44.113794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711089+0000","last_change":"2024-05-17T21:58:16.186846+0000","last_active":"2024-05-17T21:58:18.711089+0000","last_peered":"2024-05-17T21:58:18.711089+0000","last_clean":"2024-05-17T21:58:18.711089+0000","last_became_active":"2024-05-17T21:58:16.186046+0000","last_became_peered":"2024-05-17T21:58:16.186046+0000","last_unstale":"2024-05-17T21:58:18.711089+0000","last_undegraded":"2024-05-17T21:58:18.711089+0000","last_fullsized":"2024-05-17T21:58:18.711089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:26.739018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215389+0000","last_change":"2024-05-17T21:58:17.224515+0000","last_active":"2024-05-17T21:58:18.215389+0000","last_peered":"2024-05-17T21:58:18.215389+0000","last_clean":"2024-05-17T21:58:18.215389+0000","last_became_active":"2024-05-17T21:58:17.224171+0000","last_became_peered":"2024-05-17T21:58:17.224171+0000","last_unstale":"2024-05-17T21:58:18.215389+0000","last_undegraded":"2024-05-17T21:58:18.215389+0000","last_fullsized":"2024-05-17T21:58:18.215389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:27.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.232609+0000","last_change":"2024-05-17T21:58:16.179936+0000","last_active":"2024-05-17T21:58:17.232609+0000","last_peered":"2024-05-17T21:58:17.232609+0000","last_clean":"2024-05-17T21:58:17.232609+0000","last_became_active":"2024-05-17T21:58:16.178769+0000","last_became_peered":"2024-05-17T21:58:16.178769+0000","last_unstale":"2024-05-17T21:58:17.232609+0000","last_undegraded":"2024-05-17T21:58:17.232609+0000","last_fullsized":"2024-05-17T21:58:17.232609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:26.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215329+0000","last_change":"2024-05-17T21:58:17.219269+0000","last_active":"2024-05-17T21:58:18.215329+0000","last_peered":"2024-05-17T21:58:18.215329+0000","last_clean":"2024-05-17T21:58:18.215329+0000","last_became_active":"2024-05-17T21:58:17.218643+0000","last_became_peered":"2024-05-17T21:58:17.218643+0000","last_unstale":"2024-05-17T21:58:18.215329+0000","last_undegraded":"2024-05-17T21:58:18.215329+0000","last_fullsized":"2024-05-17T21:58:18.215329+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:27.597936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.208767+0000","last_change":"2024-05-17T21:58:16.185006+0000","last_active":"2024-05-17T21:58:17.208767+0000","last_peered":"2024-05-17T21:58:17.208767+0000","last_clean":"2024-05-17T21:58:17.208767+0000","last_became_active":"2024-05-17T21:58:16.184174+0000","last_became_peered":"2024-05-17T21:58:16.184174+0000","last_unstale":"2024-05-17T21:58:17.208767+0000","last_undegraded":"2024-05-17T21:58:17.208767+0000","last_fullsized":"2024-05-17T21:58:17.208767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:28.078457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214936+0000","last_change":"2024-05-17T21:58:17.215503+0000","last_active":"2024-05-17T21:58:18.214936+0000","last_peered":"2024-05-17T21:58:18.214936+0000","last_clean":"2024-05-17T21:58:18.214936+0000","last_became_active":"2024-05-17T21:58:17.215185+0000","last_became_peered":"2024-05-17T21:58:17.215185+0000","last_unstale":"2024-05-17T21:58:18.214936+0000","last_undegraded":"2024-05-17T21:58:18.214936+0000","last_fullsized":"2024-05-17T21:58:18.214936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:50:50.740232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710374+0000","last_change":"2024-05-17T21:58:16.187376+0000","last_active":"2024-05-17T21:58:18.710374+0000","last_peered":"2024-05-17T21:58:18.710374+0000","last_clean":"2024-05-17T21:58:18.710374+0000","last_became_active":"2024-05-17T21:58:16.186501+0000","last_became_peered":"2024-05-17T21:58:16.186501+0000","last_unstale":"2024-05-17T21:58:18.710374+0000","last_undegraded":"2024-05-17T21:58:18.710374+0000","last_fullsized":"2024-05-17T21:58:18.710374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:50:49.729238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253904+0000","last_change":"2024-05-17T21:58:17.238044+0000","last_active":"2024-05-17T21:58:17.253904+0000","last_peered":"2024-05-17T21:58:17.253904+0000","last_clean":"2024-05-17T21:58:17.253904+0000","last_became_active":"2024-05-17T21:58:17.236492+0000","last_became_peered":"2024-05-17T21:58:17.236492+0000","last_unstale":"2024-05-17T21:58:17.253904+0000","last_undegraded":"2024-05-17T21:58:17.253904+0000","last_fullsized":"2024-05-17T21:58:17.253904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:36.155454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215345+0000","last_change":"2024-05-17T21:58:16.184943+0000","last_active":"2024-05-17T21:58:18.215345+0000","last_peered":"2024-05-17T21:58:18.215345+0000","last_clean":"2024-05-17T21:58:18.215345+0000","last_became_active":"2024-05-17T21:58:16.183529+0000","last_became_peered":"2024-05-17T21:58:16.183529+0000","last_unstale":"2024-05-17T21:58:18.215345+0000","last_undegraded":"2024-05-17T21:58:18.215345+0000","last_fullsized":"2024-05-17T21:58:18.215345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:37.886606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216405+0000","last_change":"2024-05-17T21:58:17.205479+0000","last_active":"2024-05-17T21:58:18.216405+0000","last_peered":"2024-05-17T21:58:18.216405+0000","last_clean":"2024-05-17T21:58:18.216405+0000","last_became_active":"2024-05-17T21:58:17.204901+0000","last_became_peered":"2024-05-17T21:58:17.204901+0000","last_unstale":"2024-05-17T21:58:18.216405+0000","last_undegraded":"2024-05-17T21:58:18.216405+0000","last_fullsized":"2024-05-17T21:58:18.216405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:44.569768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214351+0000","last_change":"2024-05-17T21:58:16.183756+0000","last_active":"2024-05-17T21:58:18.214351+0000","last_peered":"2024-05-17T21:58:18.214351+0000","last_clean":"2024-05-17T21:58:18.214351+0000","last_became_active":"2024-05-17T21:58:16.181527+0000","last_became_peered":"2024-05-17T21:58:16.181527+0000","last_unstale":"2024-05-17T21:58:18.214351+0000","last_undegraded":"2024-05-17T21:58:18.214351+0000","last_fullsized":"2024-05-17T21:58:18.214351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217422+0000","last_change":"2024-05-17T21:58:17.219478+0000","last_active":"2024-05-17T21:58:18.217422+0000","last_peered":"2024-05-17T21:58:18.217422+0000","last_clean":"2024-05-17T21:58:18.217422+0000","last_became_active":"2024-05-17T21:58:17.218451+0000","last_became_peered":"2024-05-17T21:58:17.218451+0000","last_unstale":"2024-05-17T21:58:18.217422+0000","last_undegraded":"2024-05-17T21:58:18.217422+0000","last_fullsized":"2024-05-17T21:58:18.217422+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:25:20.341116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.208352+0000","last_change":"2024-05-17T21:58:16.179380+0000","last_active":"2024-05-17T21:58:18.208352+0000","last_peered":"2024-05-17T21:58:18.208352+0000","last_clean":"2024-05-17T21:58:18.208352+0000","last_became_active":"2024-05-17T21:58:16.178966+0000","last_became_peered":"2024-05-17T21:58:16.178966+0000","last_unstale":"2024-05-17T21:58:18.208352+0000","last_undegraded":"2024-05-17T21:58:18.208352+0000","last_fullsized":"2024-05-17T21:58:18.208352+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:57.087708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.226127+0000","last_change":"2024-05-17T21:58:17.224753+0000","last_active":"2024-05-17T21:58:17.226127+0000","last_peered":"2024-05-17T21:58:17.226127+0000","last_clean":"2024-05-17T21:58:17.226127+0000","last_became_active":"2024-05-17T21:58:17.224411+0000","last_became_peered":"2024-05-17T21:58:17.224411+0000","last_unstale":"2024-05-17T21:58:17.226127+0000","last_undegraded":"2024-05-17T21:58:17.226127+0000","last_fullsized":"2024-05-17T21:58:17.226127+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214143+0000","last_change":"2024-05-17T21:58:16.160805+0000","last_active":"2024-05-17T21:58:18.214143+0000","last_peered":"2024-05-17T21:58:18.214143+0000","last_clean":"2024-05-17T21:58:18.214143+0000","last_became_active":"2024-05-17T21:58:16.160333+0000","last_became_peered":"2024-05-17T21:58:16.160333+0000","last_unstale":"2024-05-17T21:58:18.214143+0000","last_undegraded":"2024-05-17T21:58:18.214143+0000","last_fullsized":"2024-05-17T21:58:18.214143+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:26.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.220416+0000","last_change":"2024-05-17T21:58:17.214929+0000","last_active":"2024-05-17T21:58:17.220416+0000","last_peered":"2024-05-17T21:58:17.220416+0000","last_clean":"2024-05-17T21:58:17.220416+0000","last_became_active":"2024-05-17T21:58:17.213949+0000","last_became_peered":"2024-05-17T21:58:17.213949+0000","last_unstale":"2024-05-17T21:58:17.220416+0000","last_undegraded":"2024-05-17T21:58:17.220416+0000","last_fullsized":"2024-05-17T21:58:17.220416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:58.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.209253+0000","last_change":"2024-05-17T21:58:16.179937+0000","last_active":"2024-05-17T21:58:17.209253+0000","last_peered":"2024-05-17T21:58:17.209253+0000","last_clean":"2024-05-17T21:58:17.209253+0000","last_became_active":"2024-05-17T21:58:16.179484+0000","last_became_peered":"2024-05-17T21:58:16.179484+0000","last_unstale":"2024-05-17T21:58:17.209253+0000","last_undegraded":"2024-05-17T21:58:17.209253+0000","last_fullsized":"2024-05-17T21:58:17.209253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:56.605637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.219863+0000","last_change":"2024-05-17T21:58:17.217200+0000","last_active":"2024-05-17T21:58:17.219863+0000","last_peered":"2024-05-17T21:58:17.219863+0000","last_clean":"2024-05-17T21:58:17.219863+0000","last_became_active":"2024-05-17T21:58:17.216861+0000","last_became_peered":"2024-05-17T21:58:17.216861+0000","last_unstale":"2024-05-17T21:58:17.219863+0000","last_undegraded":"2024-05-17T21:58:17.219863+0000","last_fullsized":"2024-05-17T21:58:17.219863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213543+0000","last_change":"2024-05-17T21:58:16.164394+0000","last_active":"2024-05-17T21:58:18.213543+0000","last_peered":"2024-05-17T21:58:18.213543+0000","last_clean":"2024-05-17T21:58:18.213543+0000","last_became_active":"2024-05-17T21:58:16.163934+0000","last_became_peered":"2024-05-17T21:58:16.163934+0000","last_unstale":"2024-05-17T21:58:18.213543+0000","last_undegraded":"2024-05-17T21:58:18.213543+0000","last_fullsized":"2024-05-17T21:58:18.213543+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:56.605637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229708+0000","last_change":"2024-05-17T21:58:17.225182+0000","last_active":"2024-05-17T21:58:17.229708+0000","last_peered":"2024-05-17T21:58:17.229708+0000","last_clean":"2024-05-17T21:58:17.229708+0000","last_became_active":"2024-05-17T21:58:17.224735+0000","last_became_peered":"2024-05-17T21:58:17.224735+0000","last_unstale":"2024-05-17T21:58:17.229708+0000","last_undegraded":"2024-05-17T21:58:17.229708+0000","last_fullsized":"2024-05-17T21:58:17.229708+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711044+0000","last_change":"2024-05-17T21:58:16.196829+0000","last_active":"2024-05-17T21:58:18.711044+0000","last_peered":"2024-05-17T21:58:18.711044+0000","last_clean":"2024-05-17T21:58:18.711044+0000","last_became_active":"2024-05-17T21:58:16.195797+0000","last_became_peered":"2024-05-17T21:58:16.195797+0000","last_unstale":"2024-05-17T21:58:18.711044+0000","last_undegraded":"2024-05-17T21:58:18.711044+0000","last_fullsized":"2024-05-17T21:58:18.711044+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:43.102799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710389+0000","last_change":"2024-05-17T21:58:17.224316+0000","last_active":"2024-05-17T21:58:18.710389+0000","last_peered":"2024-05-17T21:58:18.710389+0000","last_clean":"2024-05-17T21:58:18.710389+0000","last_became_active":"2024-05-17T21:58:17.221751+0000","last_became_peered":"2024-05-17T21:58:17.221751+0000","last_unstale":"2024-05-17T21:58:18.710389+0000","last_undegraded":"2024-05-17T21:58:18.710389+0000","last_fullsized":"2024-05-17T21:58:18.710389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:08.532377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.214833+0000","last_change":"2024-05-17T21:58:16.190454+0000","last_active":"2024-05-17T21:58:17.214833+0000","last_peered":"2024-05-17T21:58:17.214833+0000","last_clean":"2024-05-17T21:58:17.214833+0000","last_became_active":"2024-05-17T21:58:16.190016+0000","last_became_peered":"2024-05-17T21:58:16.190016+0000","last_unstale":"2024-05-17T21:58:17.214833+0000","last_undegraded":"2024-05-17T21:58:17.214833+0000","last_fullsized":"2024-05-17T21:58:17.214833+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:47.725441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253736+0000","last_change":"2024-05-17T21:58:17.235801+0000","last_active":"2024-05-17T21:58:17.253736+0000","last_peered":"2024-05-17T21:58:17.253736+0000","last_clean":"2024-05-17T21:58:17.253736+0000","last_became_active":"2024-05-17T21:58:17.217044+0000","last_became_peered":"2024-05-17T21:58:17.217044+0000","last_unstale":"2024-05-17T21:58:17.253736+0000","last_undegraded":"2024-05-17T21:58:17.253736+0000","last_fullsized":"2024-05-17T21:58:17.253736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:48.736435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.213183+0000","last_change":"2024-05-17T21:58:16.203141+0000","last_active":"2024-05-17T21:58:17.213183+0000","last_peered":"2024-05-17T21:58:17.213183+0000","last_clean":"2024-05-17T21:58:17.213183+0000","last_became_active":"2024-05-17T21:58:16.202480+0000","last_became_peered":"2024-05-17T21:58:16.202480+0000","last_unstale":"2024-05-17T21:58:17.213183+0000","last_undegraded":"2024-05-17T21:58:17.213183+0000","last_fullsized":"2024-05-17T21:58:17.213183+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":128,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.205693+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:58:17.205693+0000","last_peered":"2024-05-17T21:58:17.205693+0000","last_clean":"2024-05-17T21:58:17.205693+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:58:17.205693+0000","last_undegraded":"2024-05-17T21:58:17.205693+0000","last_fullsized":"2024-05-17T21:58:17.205693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":718284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":134798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":127930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":129460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":134196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":135760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":718194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:58:21.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:58:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 Processor -- start 2024-05-17T21:58:21.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 -- start start 2024-05-17T21:58:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0415df20 0x7f9a0415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a040afb60 0x7f9a0406c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a040b5e80 con 0x7f9a040b5780 2024-05-17T21:58:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9a040b1760 con 0x7f9a040afb60 2024-05-17T21:58:21.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a0a91b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a040b6160 con 0x7f9a0415df20 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a040afb60 0x7f9a0406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a040afb60 0x7f9a0406c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51760/0 (socket says 172.21.15.83:51760) 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1996714642 learned_addr learned my addr 172.21.15.83:0/1996714642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a03fff640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0415df20 0x7f9a0415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1996714642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0415df20 msgr2=0x7f9a0415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0415df20 0x7f9a0415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1996714642 >> v1:172.21.15.155:6789/0 conn(0x7f9a040b5780 legacy=0x7f9a040af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.499+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1996714642 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a040b6440 con 0x7f9a040afb60 2024-05-17T21:58:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a08eb8640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a040afb60 0x7f9a0406c940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009e50 tx=0x7f99fc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c5039b840f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a037fe640 1 -- 172.21.15.83:0/1996714642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0a5810 con 0x7f9a040afb60 2024-05-17T21:58:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a037fe640 1 -- 172.21.15.83:0/1996714642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0a59b0 con 0x7f9a040afb60 2024-05-17T21:58:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a037fe640 1 -- 172.21.15.83:0/1996714642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0a5d80 con 0x7f9a040afb60 2024-05-17T21:58:21.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1996714642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a040afb60 msgr2=0x7f9a0406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a040afb60 0x7f9a0406c940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009e50 tx=0x7f99fc01ca50 comp rx=0 tx=0).stop 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1996714642 shutdown_connections 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a040afb60 0x7f9a0406c940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1996714642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a0415df20 0x7f9a0415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1996714642 >> 172.21.15.83:0/1996714642 conn(0x7f9a040d1ee0 msgr2=0x7f9a040d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1996714642 shutdown_connections 2024-05-17T21:58:21.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1996714642 wait complete. 2024-05-17T21:58:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 Processor -- start 2024-05-17T21:58:21.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.503+0000 7f9a0a91b640 1 -- start start 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a040b5780 0x7f9a04153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a0415df20 0x7f9a040d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0406d810 con 0x7f9a0415a5b0 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0406d690 con 0x7f9a0415df20 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0406d990 con 0x7f9a040b5780 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51774/0 (socket says 172.21.15.83:51774) 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1812255457 learned_addr learned my addr 172.21.15.83:0/1812255457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a03fff640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a040b5780 0x7f9a04153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f99fbfff640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a0415df20 0x7f9a040d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a040b5780 msgr2=0x7f9a04153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a040b5780 0x7f9a04153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a0415df20 msgr2=0x7f9a040d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a0415df20 0x7f9a040d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 -- 172.21.15.83:0/1812255457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a04154050 con 0x7f9a0415a5b0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a08eb8640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0af040 tx=0x7f99fc0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b8470 con 0x7f9a0415a5b0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc0b8630 con 0x7f9a0415a5b0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a040d0ee0 con 0x7f9a0415a5b0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b8a20 con 0x7f9a0415a5b0 2024-05-17T21:58:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a04159ad0 con 0x7f9a0415a5b0 2024-05-17T21:58:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a040b5560 con 0x7f9a0415a5b0 2024-05-17T21:58:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.507+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99fc0c0030 con 0x7f9a0415a5b0 2024-05-17T21:58:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.511+0000 7f9a017fa640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 0x7f99e4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.511+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f99fc12ab80 con 0x7f9a0415a5b0 2024-05-17T21:58:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.511+0000 7f9a03fff640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 0x7f99e4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.511+0000 7f9a03fff640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 0x7f99e4071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99f40096f0 tx=0x7f99f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.515+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f99fc0c7020 con 0x7f9a0415a5b0 2024-05-17T21:58:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.671+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a04158470 con 0x7f99e406f430 2024-05-17T21:58:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.679+0000 7f9a017fa640 1 -- 172.21.15.83:0/1812255457 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f9a04158470 con 0x7f99e406f430 2024-05-17T21:58:21.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 msgr2=0x7f99e4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 0x7f99e4071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99f40096f0 tx=0x7f99f4008040 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 msgr2=0x7f9a040ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0af040 tx=0x7f99fc0197f0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 shutdown_connections 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a0415a5b0 0x7f9a040ce9a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f99e406f430 0x7f99e4071940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a0415df20 0x7f9a040d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 --2- 172.21.15.83:0/1812255457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a040b5780 0x7f9a04153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 >> 172.21.15.83:0/1812255457 conn(0x7f9a040d1ee0 msgr2=0x7f9a040b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 shutdown_connections 2024-05-17T21:58:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.683+0000 7f9a0a91b640 1 -- 172.21.15.83:0/1812255457 wait complete. 2024-05-17T21:58:21.697 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-05-17T21:58:20.982266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220080,"kb_used_data":3884,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754640,"statfs":{"total":773094113280,"available":772868751360,"internally_reserved":0,"allocated":3977216,"data_stored":2227064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.836966"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.250745+0000","last_change":"2024-05-17T21:58:17.231532+0000","last_active":"2024-05-17T21:58:17.250745+0000","last_peered":"2024-05-17T21:58:17.250745+0000","last_clean":"2024-05-17T21:58:17.250745+0000","last_became_active":"2024-05-17T21:58:17.230885+0000","last_became_peered":"2024-05-17T21:58:17.230885+0000","last_unstale":"2024-05-17T21:58:17.250745+0000","last_undegraded":"2024-05-17T21:58:17.250745+0000","last_fullsized":"2024-05-17T21:58:17.250745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:27.206495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.220164+0000","last_change":"2024-05-17T21:58:17.215826+0000","last_active":"2024-05-17T21:58:17.220164+0000","last_peered":"2024-05-17T21:58:17.220164+0000","last_clean":"2024-05-17T21:58:17.220164+0000","last_became_active":"2024-05-17T21:58:17.215515+0000","last_became_peered":"2024-05-17T21:58:17.215515+0000","last_unstale":"2024-05-17T21:58:17.220164+0000","last_undegraded":"2024-05-17T21:58:17.220164+0000","last_fullsized":"2024-05-17T21:58:17.220164+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215848+0000","last_change":"2024-05-17T21:58:17.219641+0000","last_active":"2024-05-17T21:58:18.215848+0000","last_peered":"2024-05-17T21:58:18.215848+0000","last_clean":"2024-05-17T21:58:18.215848+0000","last_became_active":"2024-05-17T21:58:17.218240+0000","last_became_peered":"2024-05-17T21:58:17.218240+0000","last_unstale":"2024-05-17T21:58:18.215848+0000","last_undegraded":"2024-05-17T21:58:18.215848+0000","last_fullsized":"2024-05-17T21:58:18.215848+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213744+0000","last_change":"2024-05-17T21:58:17.245796+0000","last_active":"2024-05-17T21:58:18.213744+0000","last_peered":"2024-05-17T21:58:18.213744+0000","last_clean":"2024-05-17T21:58:18.213744+0000","last_became_active":"2024-05-17T21:58:17.245431+0000","last_became_peered":"2024-05-17T21:58:17.245431+0000","last_unstale":"2024-05-17T21:58:18.213744+0000","last_undegraded":"2024-05-17T21:58:18.213744+0000","last_fullsized":"2024-05-17T21:58:18.213744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214759+0000","last_change":"2024-05-17T21:58:17.213900+0000","last_active":"2024-05-17T21:58:18.214759+0000","last_peered":"2024-05-17T21:58:18.214759+0000","last_clean":"2024-05-17T21:58:18.214759+0000","last_became_active":"2024-05-17T21:58:17.213595+0000","last_became_peered":"2024-05-17T21:58:17.213595+0000","last_unstale":"2024-05-17T21:58:18.214759+0000","last_undegraded":"2024-05-17T21:58:18.214759+0000","last_fullsized":"2024-05-17T21:58:18.214759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:42.482659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.242311+0000","last_change":"2024-05-17T21:58:17.238615+0000","last_active":"2024-05-17T21:58:17.242311+0000","last_peered":"2024-05-17T21:58:17.242311+0000","last_clean":"2024-05-17T21:58:17.242311+0000","last_became_active":"2024-05-17T21:58:17.238244+0000","last_became_peered":"2024-05-17T21:58:17.238244+0000","last_unstale":"2024-05-17T21:58:17.242311+0000","last_undegraded":"2024-05-17T21:58:17.242311+0000","last_fullsized":"2024-05-17T21:58:17.242311+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253940+0000","last_change":"2024-05-17T21:58:17.238596+0000","last_active":"2024-05-17T21:58:17.253940+0000","last_peered":"2024-05-17T21:58:17.253940+0000","last_clean":"2024-05-17T21:58:17.253940+0000","last_became_active":"2024-05-17T21:58:17.235764+0000","last_became_peered":"2024-05-17T21:58:17.235764+0000","last_unstale":"2024-05-17T21:58:17.253940+0000","last_undegraded":"2024-05-17T21:58:17.253940+0000","last_fullsized":"2024-05-17T21:58:17.253940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:08.532377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253428+0000","last_change":"2024-05-17T21:58:17.222410+0000","last_active":"2024-05-17T21:58:17.253428+0000","last_peered":"2024-05-17T21:58:17.253428+0000","last_clean":"2024-05-17T21:58:17.253428+0000","last_became_active":"2024-05-17T21:58:17.221463+0000","last_became_peered":"2024-05-17T21:58:17.221463+0000","last_unstale":"2024-05-17T21:58:17.253428+0000","last_undegraded":"2024-05-17T21:58:17.253428+0000","last_fullsized":"2024-05-17T21:58:17.253428+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711578+0000","last_change":"2024-05-17T21:58:17.219945+0000","last_active":"2024-05-17T21:58:18.711578+0000","last_peered":"2024-05-17T21:58:18.711578+0000","last_clean":"2024-05-17T21:58:18.711578+0000","last_became_active":"2024-05-17T21:58:17.212404+0000","last_became_peered":"2024-05-17T21:58:17.212404+0000","last_unstale":"2024-05-17T21:58:18.711578+0000","last_undegraded":"2024-05-17T21:58:18.711578+0000","last_fullsized":"2024-05-17T21:58:18.711578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214526+0000","last_change":"2024-05-17T21:58:17.220337+0000","last_active":"2024-05-17T21:58:18.214526+0000","last_peered":"2024-05-17T21:58:18.214526+0000","last_clean":"2024-05-17T21:58:18.214526+0000","last_became_active":"2024-05-17T21:58:17.218394+0000","last_became_peered":"2024-05-17T21:58:17.218394+0000","last_unstale":"2024-05-17T21:58:18.214526+0000","last_undegraded":"2024-05-17T21:58:18.214526+0000","last_fullsized":"2024-05-17T21:58:18.214526+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253843+0000","last_change":"2024-05-17T21:58:17.235549+0000","last_active":"2024-05-17T21:58:17.253843+0000","last_peered":"2024-05-17T21:58:17.253843+0000","last_clean":"2024-05-17T21:58:17.253843+0000","last_became_active":"2024-05-17T21:58:17.234843+0000","last_became_peered":"2024-05-17T21:58:17.234843+0000","last_unstale":"2024-05-17T21:58:17.253843+0000","last_undegraded":"2024-05-17T21:58:17.253843+0000","last_fullsized":"2024-05-17T21:58:17.253843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:38.897601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216160+0000","last_change":"2024-05-17T21:58:17.223144+0000","last_active":"2024-05-17T21:58:18.216160+0000","last_peered":"2024-05-17T21:58:18.216160+0000","last_clean":"2024-05-17T21:58:18.216160+0000","last_became_active":"2024-05-17T21:58:17.222360+0000","last_became_peered":"2024-05-17T21:58:17.222360+0000","last_unstale":"2024-05-17T21:58:18.216160+0000","last_undegraded":"2024-05-17T21:58:18.216160+0000","last_fullsized":"2024-05-17T21:58:18.216160+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215157+0000","last_change":"2024-05-17T21:58:17.218946+0000","last_active":"2024-05-17T21:58:18.215157+0000","last_peered":"2024-05-17T21:58:18.215157+0000","last_clean":"2024-05-17T21:58:18.215157+0000","last_became_active":"2024-05-17T21:58:17.218234+0000","last_became_peered":"2024-05-17T21:58:17.218234+0000","last_unstale":"2024-05-17T21:58:18.215157+0000","last_undegraded":"2024-05-17T21:58:18.215157+0000","last_fullsized":"2024-05-17T21:58:18.215157+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:24.371583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.218890+0000","last_change":"2024-05-17T21:58:17.215489+0000","last_active":"2024-05-17T21:58:17.218890+0000","last_peered":"2024-05-17T21:58:17.218890+0000","last_clean":"2024-05-17T21:58:17.218890+0000","last_became_active":"2024-05-17T21:58:17.214842+0000","last_became_peered":"2024-05-17T21:58:17.214842+0000","last_unstale":"2024-05-17T21:58:17.218890+0000","last_undegraded":"2024-05-17T21:58:17.218890+0000","last_fullsized":"2024-05-17T21:58:17.218890+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215952+0000","last_change":"2024-05-17T21:58:17.210803+0000","last_active":"2024-05-17T21:58:18.215952+0000","last_peered":"2024-05-17T21:58:18.215952+0000","last_clean":"2024-05-17T21:58:18.215952+0000","last_became_active":"2024-05-17T21:58:17.209984+0000","last_became_peered":"2024-05-17T21:58:17.209984+0000","last_unstale":"2024-05-17T21:58:18.215952+0000","last_undegraded":"2024-05-17T21:58:18.215952+0000","last_fullsized":"2024-05-17T21:58:18.215952+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253794+0000","last_change":"2024-05-17T21:58:17.236222+0000","last_active":"2024-05-17T21:58:17.253794+0000","last_peered":"2024-05-17T21:58:17.253794+0000","last_clean":"2024-05-17T21:58:17.253794+0000","last_became_active":"2024-05-17T21:58:17.223206+0000","last_became_peered":"2024-05-17T21:58:17.223206+0000","last_unstale":"2024-05-17T21:58:17.253794+0000","last_undegraded":"2024-05-17T21:58:17.253794+0000","last_fullsized":"2024-05-17T21:58:17.253794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:39:56.935731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253562+0000","last_change":"2024-05-17T21:58:17.222192+0000","last_active":"2024-05-17T21:58:17.253562+0000","last_peered":"2024-05-17T21:58:17.253562+0000","last_clean":"2024-05-17T21:58:17.253562+0000","last_became_active":"2024-05-17T21:58:17.221388+0000","last_became_peered":"2024-05-17T21:58:17.221388+0000","last_unstale":"2024-05-17T21:58:17.253562+0000","last_undegraded":"2024-05-17T21:58:17.253562+0000","last_fullsized":"2024-05-17T21:58:17.253562+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229355+0000","last_change":"2024-05-17T21:58:17.224969+0000","last_active":"2024-05-17T21:58:17.229355+0000","last_peered":"2024-05-17T21:58:17.229355+0000","last_clean":"2024-05-17T21:58:17.229355+0000","last_became_active":"2024-05-17T21:58:17.224588+0000","last_became_peered":"2024-05-17T21:58:17.224588+0000","last_unstale":"2024-05-17T21:58:17.229355+0000","last_undegraded":"2024-05-17T21:58:17.229355+0000","last_fullsized":"2024-05-17T21:58:17.229355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.230797+0000","last_change":"2024-05-17T21:58:17.225311+0000","last_active":"2024-05-17T21:58:17.230797+0000","last_peered":"2024-05-17T21:58:17.230797+0000","last_clean":"2024-05-17T21:58:17.230797+0000","last_became_active":"2024-05-17T21:58:17.224988+0000","last_became_peered":"2024-05-17T21:58:17.224988+0000","last_unstale":"2024-05-17T21:58:17.230797+0000","last_undegraded":"2024-05-17T21:58:17.230797+0000","last_fullsized":"2024-05-17T21:58:17.230797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215525+0000","last_change":"2024-05-17T21:58:17.206972+0000","last_active":"2024-05-17T21:58:18.215525+0000","last_peered":"2024-05-17T21:58:18.215525+0000","last_clean":"2024-05-17T21:58:18.215525+0000","last_became_active":"2024-05-17T21:58:17.206659+0000","last_became_peered":"2024-05-17T21:58:17.206659+0000","last_unstale":"2024-05-17T21:58:18.215525+0000","last_undegraded":"2024-05-17T21:58:18.215525+0000","last_fullsized":"2024-05-17T21:58:18.215525+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:29.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710638+0000","last_change":"2024-05-17T21:58:17.232619+0000","last_active":"2024-05-17T21:58:18.710638+0000","last_peered":"2024-05-17T21:58:18.710638+0000","last_clean":"2024-05-17T21:58:18.710638+0000","last_became_active":"2024-05-17T21:58:17.212476+0000","last_became_peered":"2024-05-17T21:58:17.212476+0000","last_unstale":"2024-05-17T21:58:18.710638+0000","last_undegraded":"2024-05-17T21:58:18.710638+0000","last_fullsized":"2024-05-17T21:58:18.710638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:47.375240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710747+0000","last_change":"2024-05-17T21:58:17.234499+0000","last_active":"2024-05-17T21:58:18.710747+0000","last_peered":"2024-05-17T21:58:18.710747+0000","last_clean":"2024-05-17T21:58:18.710747+0000","last_became_active":"2024-05-17T21:58:17.232854+0000","last_became_peered":"2024-05-17T21:58:17.232854+0000","last_unstale":"2024-05-17T21:58:18.710747+0000","last_undegraded":"2024-05-17T21:58:18.710747+0000","last_fullsized":"2024-05-17T21:58:18.710747+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.237062+0000","last_change":"2024-05-17T21:58:17.235353+0000","last_active":"2024-05-17T21:58:17.237062+0000","last_peered":"2024-05-17T21:58:17.237062+0000","last_clean":"2024-05-17T21:58:17.237062+0000","last_became_active":"2024-05-17T21:58:17.235030+0000","last_became_peered":"2024-05-17T21:58:17.235030+0000","last_unstale":"2024-05-17T21:58:17.237062+0000","last_undegraded":"2024-05-17T21:58:17.237062+0000","last_fullsized":"2024-05-17T21:58:17.237062+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:38.897601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217102+0000","last_change":"2024-05-17T21:58:17.200790+0000","last_active":"2024-05-17T21:58:18.217102+0000","last_peered":"2024-05-17T21:58:18.217102+0000","last_clean":"2024-05-17T21:58:18.217102+0000","last_became_active":"2024-05-17T21:58:17.199889+0000","last_became_peered":"2024-05-17T21:58:17.199889+0000","last_unstale":"2024-05-17T21:58:18.217102+0000","last_undegraded":"2024-05-17T21:58:18.217102+0000","last_fullsized":"2024-05-17T21:58:18.217102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:32:45.779428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213904+0000","last_change":"2024-05-17T21:58:17.240587+0000","last_active":"2024-05-17T21:58:18.213904+0000","last_peered":"2024-05-17T21:58:18.213904+0000","last_clean":"2024-05-17T21:58:18.213904+0000","last_became_active":"2024-05-17T21:58:17.240281+0000","last_became_peered":"2024-05-17T21:58:17.240281+0000","last_unstale":"2024-05-17T21:58:18.213904+0000","last_undegraded":"2024-05-17T21:58:18.213904+0000","last_fullsized":"2024-05-17T21:58:18.213904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.244944+0000","last_change":"2024-05-17T21:58:17.240665+0000","last_active":"2024-05-17T21:58:17.244944+0000","last_peered":"2024-05-17T21:58:17.244944+0000","last_clean":"2024-05-17T21:58:17.244944+0000","last_became_active":"2024-05-17T21:58:17.240205+0000","last_became_peered":"2024-05-17T21:58:17.240205+0000","last_unstale":"2024-05-17T21:58:17.244944+0000","last_undegraded":"2024-05-17T21:58:17.244944+0000","last_fullsized":"2024-05-17T21:58:17.244944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214133+0000","last_change":"2024-05-17T21:58:17.244345+0000","last_active":"2024-05-17T21:58:18.214133+0000","last_peered":"2024-05-17T21:58:18.214133+0000","last_clean":"2024-05-17T21:58:18.214133+0000","last_became_active":"2024-05-17T21:58:17.244065+0000","last_became_peered":"2024-05-17T21:58:17.244065+0000","last_unstale":"2024-05-17T21:58:18.214133+0000","last_undegraded":"2024-05-17T21:58:18.214133+0000","last_fullsized":"2024-05-17T21:58:18.214133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:54.389385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216245+0000","last_change":"2024-05-17T21:58:17.228068+0000","last_active":"2024-05-17T21:58:18.216245+0000","last_peered":"2024-05-17T21:58:18.216245+0000","last_clean":"2024-05-17T21:58:18.216245+0000","last_became_active":"2024-05-17T21:58:17.227234+0000","last_became_peered":"2024-05-17T21:58:17.227234+0000","last_unstale":"2024-05-17T21:58:18.216245+0000","last_undegraded":"2024-05-17T21:58:18.216245+0000","last_fullsized":"2024-05-17T21:58:18.216245+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:37:17.808915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711232+0000","last_change":"2024-05-17T21:58:17.234485+0000","last_active":"2024-05-17T21:58:18.711232+0000","last_peered":"2024-05-17T21:58:18.711232+0000","last_clean":"2024-05-17T21:58:18.711232+0000","last_became_active":"2024-05-17T21:58:17.232968+0000","last_became_peered":"2024-05-17T21:58:17.232968+0000","last_unstale":"2024-05-17T21:58:18.711232+0000","last_undegraded":"2024-05-17T21:58:18.711232+0000","last_fullsized":"2024-05-17T21:58:18.711232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.247682+0000","last_change":"2024-05-17T21:58:17.217853+0000","last_active":"2024-05-17T21:58:17.247682+0000","last_peered":"2024-05-17T21:58:17.247682+0000","last_clean":"2024-05-17T21:58:17.247682+0000","last_became_active":"2024-05-17T21:58:17.216989+0000","last_became_peered":"2024-05-17T21:58:17.216989+0000","last_unstale":"2024-05-17T21:58:17.247682+0000","last_undegraded":"2024-05-17T21:58:17.247682+0000","last_fullsized":"2024-05-17T21:58:17.247682+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:27.206495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.226275+0000","last_change":"2024-05-17T21:58:17.215039+0000","last_active":"2024-05-17T21:58:17.226275+0000","last_peered":"2024-05-17T21:58:17.226275+0000","last_clean":"2024-05-17T21:58:17.226275+0000","last_became_active":"2024-05-17T21:58:17.214238+0000","last_became_peered":"2024-05-17T21:58:17.214238+0000","last_unstale":"2024-05-17T21:58:17.226275+0000","last_undegraded":"2024-05-17T21:58:17.226275+0000","last_fullsized":"2024-05-17T21:58:17.226275+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:57.616631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.256306+0000","last_change":"2024-05-17T21:58:17.249463+0000","last_active":"2024-05-17T21:58:17.256306+0000","last_peered":"2024-05-17T21:58:17.256306+0000","last_clean":"2024-05-17T21:58:17.256306+0000","last_became_active":"2024-05-17T21:58:17.246383+0000","last_became_peered":"2024-05-17T21:58:17.246383+0000","last_unstale":"2024-05-17T21:58:17.256306+0000","last_undegraded":"2024-05-17T21:58:17.256306+0000","last_fullsized":"2024-05-17T21:58:17.256306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:55.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.242182+0000","last_change":"2024-05-17T21:58:17.240722+0000","last_active":"2024-05-17T21:58:17.242182+0000","last_peered":"2024-05-17T21:58:17.242182+0000","last_clean":"2024-05-17T21:58:17.242182+0000","last_became_active":"2024-05-17T21:58:17.240358+0000","last_became_peered":"2024-05-17T21:58:17.240358+0000","last_unstale":"2024-05-17T21:58:17.242182+0000","last_undegraded":"2024-05-17T21:58:17.242182+0000","last_fullsized":"2024-05-17T21:58:17.242182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:48.736435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.250811+0000","last_change":"2024-05-17T21:58:16.192574+0000","last_active":"2024-05-17T21:58:17.250811+0000","last_peered":"2024-05-17T21:58:17.250811+0000","last_clean":"2024-05-17T21:58:17.250811+0000","last_became_active":"2024-05-17T21:58:16.191691+0000","last_became_peered":"2024-05-17T21:58:16.191691+0000","last_unstale":"2024-05-17T21:58:17.250811+0000","last_undegraded":"2024-05-17T21:58:17.250811+0000","last_fullsized":"2024-05-17T21:58:17.250811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:44.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.252479+0000","last_change":"2024-05-17T21:58:17.250324+0000","last_active":"2024-05-17T21:58:17.252479+0000","last_peered":"2024-05-17T21:58:17.252479+0000","last_clean":"2024-05-17T21:58:17.252479+0000","last_became_active":"2024-05-17T21:58:17.248055+0000","last_became_peered":"2024-05-17T21:58:17.248055+0000","last_unstale":"2024-05-17T21:58:17.252479+0000","last_undegraded":"2024-05-17T21:58:17.252479+0000","last_fullsized":"2024-05-17T21:58:17.252479+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.254089+0000","last_change":"2024-05-17T21:58:16.181268+0000","last_active":"2024-05-17T21:58:17.254089+0000","last_peered":"2024-05-17T21:58:17.254089+0000","last_clean":"2024-05-17T21:58:17.254089+0000","last_became_active":"2024-05-17T21:58:16.180668+0000","last_became_peered":"2024-05-17T21:58:16.180668+0000","last_unstale":"2024-05-17T21:58:17.254089+0000","last_undegraded":"2024-05-17T21:58:17.254089+0000","last_fullsized":"2024-05-17T21:58:17.254089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711051+0000","last_change":"2024-05-17T21:58:17.240237+0000","last_active":"2024-05-17T21:58:18.711051+0000","last_peered":"2024-05-17T21:58:18.711051+0000","last_clean":"2024-05-17T21:58:18.711051+0000","last_became_active":"2024-05-17T21:58:17.231690+0000","last_became_peered":"2024-05-17T21:58:17.231690+0000","last_unstale":"2024-05-17T21:58:18.711051+0000","last_undegraded":"2024-05-17T21:58:18.711051+0000","last_fullsized":"2024-05-17T21:58:18.711051+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.234485+0000","last_change":"2024-05-17T21:58:16.183432+0000","last_active":"2024-05-17T21:58:17.234485+0000","last_peered":"2024-05-17T21:58:17.234485+0000","last_clean":"2024-05-17T21:58:17.234485+0000","last_became_active":"2024-05-17T21:58:16.182130+0000","last_became_peered":"2024-05-17T21:58:16.182130+0000","last_unstale":"2024-05-17T21:58:17.234485+0000","last_undegraded":"2024-05-17T21:58:17.234485+0000","last_fullsized":"2024-05-17T21:58:17.234485+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:46.364246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214310+0000","last_change":"2024-05-17T21:58:17.221739+0000","last_active":"2024-05-17T21:58:18.214310+0000","last_peered":"2024-05-17T21:58:18.214310+0000","last_clean":"2024-05-17T21:58:18.214310+0000","last_became_active":"2024-05-17T21:58:17.220418+0000","last_became_peered":"2024-05-17T21:58:17.220418+0000","last_unstale":"2024-05-17T21:58:18.214310+0000","last_undegraded":"2024-05-17T21:58:18.214310+0000","last_fullsized":"2024-05-17T21:58:18.214310+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:45.773456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215422+0000","last_change":"2024-05-17T21:58:16.166687+0000","last_active":"2024-05-17T21:58:18.215422+0000","last_peered":"2024-05-17T21:58:18.215422+0000","last_clean":"2024-05-17T21:58:18.215422+0000","last_became_active":"2024-05-17T21:58:16.164695+0000","last_became_peered":"2024-05-17T21:58:16.164695+0000","last_unstale":"2024-05-17T21:58:18.215422+0000","last_undegraded":"2024-05-17T21:58:18.215422+0000","last_fullsized":"2024-05-17T21:58:18.215422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:59.418642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.254103+0000","last_change":"2024-05-17T21:58:17.247347+0000","last_active":"2024-05-17T21:58:17.254103+0000","last_peered":"2024-05-17T21:58:17.254103+0000","last_clean":"2024-05-17T21:58:17.254103+0000","last_became_active":"2024-05-17T21:58:17.246687+0000","last_became_peered":"2024-05-17T21:58:17.246687+0000","last_unstale":"2024-05-17T21:58:17.254103+0000","last_undegraded":"2024-05-17T21:58:17.254103+0000","last_fullsized":"2024-05-17T21:58:17.254103+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:05:47.375240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711451+0000","last_change":"2024-05-17T21:58:16.196750+0000","last_active":"2024-05-17T21:58:18.711451+0000","last_peered":"2024-05-17T21:58:18.711451+0000","last_clean":"2024-05-17T21:58:18.711451+0000","last_became_active":"2024-05-17T21:58:16.195229+0000","last_became_peered":"2024-05-17T21:58:16.195229+0000","last_unstale":"2024-05-17T21:58:18.711451+0000","last_undegraded":"2024-05-17T21:58:18.711451+0000","last_fullsized":"2024-05-17T21:58:18.711451+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:23.360589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215819+0000","last_change":"2024-05-17T21:58:17.230033+0000","last_active":"2024-05-17T21:58:18.215819+0000","last_peered":"2024-05-17T21:58:18.215819+0000","last_clean":"2024-05-17T21:58:18.215819+0000","last_became_active":"2024-05-17T21:58:17.229201+0000","last_became_peered":"2024-05-17T21:58:17.229201+0000","last_unstale":"2024-05-17T21:58:18.215819+0000","last_undegraded":"2024-05-17T21:58:18.215819+0000","last_fullsized":"2024-05-17T21:58:18.215819+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710735+0000","last_change":"2024-05-17T21:58:16.187118+0000","last_active":"2024-05-17T21:58:18.710735+0000","last_peered":"2024-05-17T21:58:18.710735+0000","last_clean":"2024-05-17T21:58:18.710735+0000","last_became_active":"2024-05-17T21:58:16.186129+0000","last_became_peered":"2024-05-17T21:58:16.186129+0000","last_unstale":"2024-05-17T21:58:18.710735+0000","last_undegraded":"2024-05-17T21:58:18.710735+0000","last_fullsized":"2024-05-17T21:58:18.710735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:41.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710336+0000","last_change":"2024-05-17T21:58:17.231849+0000","last_active":"2024-05-17T21:58:18.710336+0000","last_peered":"2024-05-17T21:58:18.710336+0000","last_clean":"2024-05-17T21:58:18.710336+0000","last_became_active":"2024-05-17T21:58:17.225139+0000","last_became_peered":"2024-05-17T21:58:17.225139+0000","last_unstale":"2024-05-17T21:58:18.710336+0000","last_undegraded":"2024-05-17T21:58:18.710336+0000","last_fullsized":"2024-05-17T21:58:18.710336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:36.155454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.214988+0000","last_change":"2024-05-17T21:58:16.174650+0000","last_active":"2024-05-17T21:58:17.214988+0000","last_peered":"2024-05-17T21:58:17.214988+0000","last_clean":"2024-05-17T21:58:17.214988+0000","last_became_active":"2024-05-17T21:58:16.174080+0000","last_became_peered":"2024-05-17T21:58:16.174080+0000","last_unstale":"2024-05-17T21:58:17.214988+0000","last_undegraded":"2024-05-17T21:58:17.214988+0000","last_fullsized":"2024-05-17T21:58:17.214988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:39:55.924737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213714+0000","last_change":"2024-05-17T21:58:17.238673+0000","last_active":"2024-05-17T21:58:18.213714+0000","last_peered":"2024-05-17T21:58:18.213714+0000","last_clean":"2024-05-17T21:58:18.213714+0000","last_became_active":"2024-05-17T21:58:17.238376+0000","last_became_peered":"2024-05-17T21:58:17.238376+0000","last_unstale":"2024-05-17T21:58:18.213714+0000","last_undegraded":"2024-05-17T21:58:18.213714+0000","last_fullsized":"2024-05-17T21:58:18.213714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:08.207777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711539+0000","last_change":"2024-05-17T21:58:16.195135+0000","last_active":"2024-05-17T21:58:18.711539+0000","last_peered":"2024-05-17T21:58:18.711539+0000","last_clean":"2024-05-17T21:58:18.711539+0000","last_became_active":"2024-05-17T21:58:16.188660+0000","last_became_peered":"2024-05-17T21:58:16.188660+0000","last_unstale":"2024-05-17T21:58:18.711539+0000","last_undegraded":"2024-05-17T21:58:18.711539+0000","last_fullsized":"2024-05-17T21:58:18.711539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:38.551838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216632+0000","last_change":"2024-05-17T21:58:17.221595+0000","last_active":"2024-05-17T21:58:18.216632+0000","last_peered":"2024-05-17T21:58:18.216632+0000","last_clean":"2024-05-17T21:58:18.216632+0000","last_became_active":"2024-05-17T21:58:17.220686+0000","last_became_peered":"2024-05-17T21:58:17.220686+0000","last_unstale":"2024-05-17T21:58:18.216632+0000","last_undegraded":"2024-05-17T21:58:18.216632+0000","last_fullsized":"2024-05-17T21:58:18.216632+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:10:12.112834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.215362+0000","last_change":"2024-05-17T21:58:16.190739+0000","last_active":"2024-05-17T21:58:17.215362+0000","last_peered":"2024-05-17T21:58:17.215362+0000","last_clean":"2024-05-17T21:58:17.215362+0000","last_became_active":"2024-05-17T21:58:16.190135+0000","last_became_peered":"2024-05-17T21:58:16.190135+0000","last_unstale":"2024-05-17T21:58:17.215362+0000","last_undegraded":"2024-05-17T21:58:17.215362+0000","last_fullsized":"2024-05-17T21:58:17.215362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:07.521383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229510+0000","last_change":"2024-05-17T21:58:17.227593+0000","last_active":"2024-05-17T21:58:17.229510+0000","last_peered":"2024-05-17T21:58:17.229510+0000","last_clean":"2024-05-17T21:58:17.229510+0000","last_became_active":"2024-05-17T21:58:17.227322+0000","last_became_peered":"2024-05-17T21:58:17.227322+0000","last_unstale":"2024-05-17T21:58:17.229510+0000","last_undegraded":"2024-05-17T21:58:17.229510+0000","last_fullsized":"2024-05-17T21:58:17.229510+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.208602+0000","last_change":"2024-05-17T21:58:16.176414+0000","last_active":"2024-05-17T21:58:17.208602+0000","last_peered":"2024-05-17T21:58:17.208602+0000","last_clean":"2024-05-17T21:58:17.208602+0000","last_became_active":"2024-05-17T21:58:16.175832+0000","last_became_peered":"2024-05-17T21:58:16.175832+0000","last_unstale":"2024-05-17T21:58:17.208602+0000","last_undegraded":"2024-05-17T21:58:17.208602+0000","last_fullsized":"2024-05-17T21:58:17.208602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:44.993701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.244921+0000","last_change":"2024-05-17T21:58:17.240669+0000","last_active":"2024-05-17T21:58:17.244921+0000","last_peered":"2024-05-17T21:58:17.244921+0000","last_clean":"2024-05-17T21:58:17.244921+0000","last_became_active":"2024-05-17T21:58:17.240243+0000","last_became_peered":"2024-05-17T21:58:17.240243+0000","last_unstale":"2024-05-17T21:58:17.244921+0000","last_undegraded":"2024-05-17T21:58:17.244921+0000","last_fullsized":"2024-05-17T21:58:17.244921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:09.247683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216166+0000","last_change":"2024-05-17T21:58:16.192088+0000","last_active":"2024-05-17T21:58:18.216166+0000","last_peered":"2024-05-17T21:58:18.216166+0000","last_clean":"2024-05-17T21:58:18.216166+0000","last_became_active":"2024-05-17T21:58:16.191207+0000","last_became_peered":"2024-05-17T21:58:16.191207+0000","last_unstale":"2024-05-17T21:58:18.216166+0000","last_undegraded":"2024-05-17T21:58:18.216166+0000","last_fullsized":"2024-05-17T21:58:18.216166+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215772+0000","last_change":"2024-05-17T21:58:17.220429+0000","last_active":"2024-05-17T21:58:18.215772+0000","last_peered":"2024-05-17T21:58:18.215772+0000","last_clean":"2024-05-17T21:58:18.215772+0000","last_became_active":"2024-05-17T21:58:17.219570+0000","last_became_peered":"2024-05-17T21:58:17.219570+0000","last_unstale":"2024-05-17T21:58:18.215772+0000","last_undegraded":"2024-05-17T21:58:18.215772+0000","last_fullsized":"2024-05-17T21:58:18.215772+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:06.209813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.255688+0000","last_change":"2024-05-17T21:58:16.182543+0000","last_active":"2024-05-17T21:58:17.255688+0000","last_peered":"2024-05-17T21:58:17.255688+0000","last_clean":"2024-05-17T21:58:17.255688+0000","last_became_active":"2024-05-17T21:58:16.182127+0000","last_became_peered":"2024-05-17T21:58:16.182127+0000","last_unstale":"2024-05-17T21:58:17.255688+0000","last_undegraded":"2024-05-17T21:58:17.255688+0000","last_fullsized":"2024-05-17T21:58:17.255688+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:37.886606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213390+0000","last_change":"2024-05-17T21:58:17.238765+0000","last_active":"2024-05-17T21:58:18.213390+0000","last_peered":"2024-05-17T21:58:18.213390+0000","last_clean":"2024-05-17T21:58:18.213390+0000","last_became_active":"2024-05-17T21:58:17.238371+0000","last_became_peered":"2024-05-17T21:58:17.238371+0000","last_unstale":"2024-05-17T21:58:18.213390+0000","last_undegraded":"2024-05-17T21:58:18.213390+0000","last_fullsized":"2024-05-17T21:58:18.213390+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711479+0000","last_change":"2024-05-17T21:58:16.199687+0000","last_active":"2024-05-17T21:58:18.711479+0000","last_peered":"2024-05-17T21:58:18.711479+0000","last_clean":"2024-05-17T21:58:18.711479+0000","last_became_active":"2024-05-17T21:58:16.196296+0000","last_became_peered":"2024-05-17T21:58:16.196296+0000","last_unstale":"2024-05-17T21:58:18.711479+0000","last_undegraded":"2024-05-17T21:58:18.711479+0000","last_fullsized":"2024-05-17T21:58:18.711479+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:26.586942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.225820+0000","last_change":"2024-05-17T21:58:17.219009+0000","last_active":"2024-05-17T21:58:17.225820+0000","last_peered":"2024-05-17T21:58:17.225820+0000","last_clean":"2024-05-17T21:58:17.225820+0000","last_became_active":"2024-05-17T21:58:17.218693+0000","last_became_peered":"2024-05-17T21:58:17.218693+0000","last_unstale":"2024-05-17T21:58:17.225820+0000","last_undegraded":"2024-05-17T21:58:17.225820+0000","last_fullsized":"2024-05-17T21:58:17.225820+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:31.661575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213385+0000","last_change":"2024-05-17T21:58:16.178401+0000","last_active":"2024-05-17T21:58:18.213385+0000","last_peered":"2024-05-17T21:58:18.213385+0000","last_clean":"2024-05-17T21:58:18.213385+0000","last_became_active":"2024-05-17T21:58:16.177954+0000","last_became_peered":"2024-05-17T21:58:16.177954+0000","last_unstale":"2024-05-17T21:58:18.213385+0000","last_undegraded":"2024-05-17T21:58:18.213385+0000","last_fullsized":"2024-05-17T21:58:18.213385+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:57.087708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.259966+0000","last_change":"2024-05-17T21:58:17.249924+0000","last_active":"2024-05-17T21:58:17.259966+0000","last_peered":"2024-05-17T21:58:17.259966+0000","last_clean":"2024-05-17T21:58:17.259966+0000","last_became_active":"2024-05-17T21:58:17.248884+0000","last_became_peered":"2024-05-17T21:58:17.248884+0000","last_unstale":"2024-05-17T21:58:17.259966+0000","last_undegraded":"2024-05-17T21:58:17.259966+0000","last_fullsized":"2024-05-17T21:58:17.259966+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.215191+0000","last_change":"2024-05-17T21:58:16.190511+0000","last_active":"2024-05-17T21:58:17.215191+0000","last_peered":"2024-05-17T21:58:17.215191+0000","last_clean":"2024-05-17T21:58:17.215191+0000","last_became_active":"2024-05-17T21:58:16.190064+0000","last_became_peered":"2024-05-17T21:58:16.190064+0000","last_unstale":"2024-05-17T21:58:17.215191+0000","last_undegraded":"2024-05-17T21:58:17.215191+0000","last_fullsized":"2024-05-17T21:58:17.215191+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:35.144459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.252453+0000","last_change":"2024-05-17T21:58:17.247221+0000","last_active":"2024-05-17T21:58:17.252453+0000","last_peered":"2024-05-17T21:58:17.252453+0000","last_clean":"2024-05-17T21:58:17.252453+0000","last_became_active":"2024-05-17T21:58:17.246461+0000","last_became_peered":"2024-05-17T21:58:17.246461+0000","last_unstale":"2024-05-17T21:58:17.252453+0000","last_undegraded":"2024-05-17T21:58:17.252453+0000","last_fullsized":"2024-05-17T21:58:17.252453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.234506+0000","last_change":"2024-05-17T21:58:16.181475+0000","last_active":"2024-05-17T21:58:17.234506+0000","last_peered":"2024-05-17T21:58:17.234506+0000","last_clean":"2024-05-17T21:58:17.234506+0000","last_became_active":"2024-05-17T21:58:16.180870+0000","last_became_peered":"2024-05-17T21:58:16.180870+0000","last_unstale":"2024-05-17T21:58:17.234506+0000","last_undegraded":"2024-05-17T21:58:17.234506+0000","last_fullsized":"2024-05-17T21:58:17.234506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710894+0000","last_change":"2024-05-17T21:58:17.223906+0000","last_active":"2024-05-17T21:58:18.710894+0000","last_peered":"2024-05-17T21:58:18.710894+0000","last_clean":"2024-05-17T21:58:18.710894+0000","last_became_active":"2024-05-17T21:58:17.222931+0000","last_became_peered":"2024-05-17T21:58:17.222931+0000","last_unstale":"2024-05-17T21:58:18.710894+0000","last_undegraded":"2024-05-17T21:58:18.710894+0000","last_fullsized":"2024-05-17T21:58:18.710894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:01.142762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214081+0000","last_change":"2024-05-17T21:58:16.177379+0000","last_active":"2024-05-17T21:58:18.214081+0000","last_peered":"2024-05-17T21:58:18.214081+0000","last_clean":"2024-05-17T21:58:18.214081+0000","last_became_active":"2024-05-17T21:58:16.176504+0000","last_became_peered":"2024-05-17T21:58:16.176504+0000","last_unstale":"2024-05-17T21:58:18.214081+0000","last_undegraded":"2024-05-17T21:58:18.214081+0000","last_fullsized":"2024-05-17T21:58:18.214081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:47.007631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217074+0000","last_change":"2024-05-17T21:58:17.206699+0000","last_active":"2024-05-17T21:58:18.217074+0000","last_peered":"2024-05-17T21:58:18.217074+0000","last_clean":"2024-05-17T21:58:18.217074+0000","last_became_active":"2024-05-17T21:58:17.203627+0000","last_became_peered":"2024-05-17T21:58:17.203627+0000","last_unstale":"2024-05-17T21:58:18.217074+0000","last_undegraded":"2024-05-17T21:58:18.217074+0000","last_fullsized":"2024-05-17T21:58:18.217074+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:36:43.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711371+0000","last_change":"2024-05-17T21:58:16.194836+0000","last_active":"2024-05-17T21:58:18.711371+0000","last_peered":"2024-05-17T21:58:18.711371+0000","last_clean":"2024-05-17T21:58:18.711371+0000","last_became_active":"2024-05-17T21:58:16.188468+0000","last_became_peered":"2024-05-17T21:58:16.188468+0000","last_unstale":"2024-05-17T21:58:18.711371+0000","last_undegraded":"2024-05-17T21:58:18.711371+0000","last_fullsized":"2024-05-17T21:58:18.711371+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213932+0000","last_change":"2024-05-17T21:58:17.246098+0000","last_active":"2024-05-17T21:58:18.213932+0000","last_peered":"2024-05-17T21:58:18.213932+0000","last_clean":"2024-05-17T21:58:18.213932+0000","last_became_active":"2024-05-17T21:58:17.245475+0000","last_became_peered":"2024-05-17T21:58:17.245475+0000","last_unstale":"2024-05-17T21:58:18.213932+0000","last_undegraded":"2024-05-17T21:58:18.213932+0000","last_fullsized":"2024-05-17T21:58:18.213932+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:28.033345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711264+0000","last_change":"2024-05-17T21:58:16.196629+0000","last_active":"2024-05-17T21:58:18.711264+0000","last_peered":"2024-05-17T21:58:18.711264+0000","last_clean":"2024-05-17T21:58:18.711264+0000","last_became_active":"2024-05-17T21:58:16.195479+0000","last_became_peered":"2024-05-17T21:58:16.195479+0000","last_unstale":"2024-05-17T21:58:18.711264+0000","last_undegraded":"2024-05-17T21:58:18.711264+0000","last_fullsized":"2024-05-17T21:58:18.711264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:28.078457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215134+0000","last_change":"2024-05-17T21:58:17.218218+0000","last_active":"2024-05-17T21:58:18.215134+0000","last_peered":"2024-05-17T21:58:18.215134+0000","last_clean":"2024-05-17T21:58:18.215134+0000","last_became_active":"2024-05-17T21:58:17.217919+0000","last_became_peered":"2024-05-17T21:58:17.217919+0000","last_unstale":"2024-05-17T21:58:18.215134+0000","last_undegraded":"2024-05-17T21:58:18.215134+0000","last_fullsized":"2024-05-17T21:58:18.215134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:44.113794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711089+0000","last_change":"2024-05-17T21:58:16.186846+0000","last_active":"2024-05-17T21:58:18.711089+0000","last_peered":"2024-05-17T21:58:18.711089+0000","last_clean":"2024-05-17T21:58:18.711089+0000","last_became_active":"2024-05-17T21:58:16.186046+0000","last_became_peered":"2024-05-17T21:58:16.186046+0000","last_unstale":"2024-05-17T21:58:18.711089+0000","last_undegraded":"2024-05-17T21:58:18.711089+0000","last_fullsized":"2024-05-17T21:58:18.711089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:26.739018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215389+0000","last_change":"2024-05-17T21:58:17.224515+0000","last_active":"2024-05-17T21:58:18.215389+0000","last_peered":"2024-05-17T21:58:18.215389+0000","last_clean":"2024-05-17T21:58:18.215389+0000","last_became_active":"2024-05-17T21:58:17.224171+0000","last_became_peered":"2024-05-17T21:58:17.224171+0000","last_unstale":"2024-05-17T21:58:18.215389+0000","last_undegraded":"2024-05-17T21:58:18.215389+0000","last_fullsized":"2024-05-17T21:58:18.215389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:27.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.232609+0000","last_change":"2024-05-17T21:58:16.179936+0000","last_active":"2024-05-17T21:58:17.232609+0000","last_peered":"2024-05-17T21:58:17.232609+0000","last_clean":"2024-05-17T21:58:17.232609+0000","last_became_active":"2024-05-17T21:58:16.178769+0000","last_became_peered":"2024-05-17T21:58:16.178769+0000","last_unstale":"2024-05-17T21:58:17.232609+0000","last_undegraded":"2024-05-17T21:58:17.232609+0000","last_fullsized":"2024-05-17T21:58:17.232609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:26.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215329+0000","last_change":"2024-05-17T21:58:17.219269+0000","last_active":"2024-05-17T21:58:18.215329+0000","last_peered":"2024-05-17T21:58:18.215329+0000","last_clean":"2024-05-17T21:58:18.215329+0000","last_became_active":"2024-05-17T21:58:17.218643+0000","last_became_peered":"2024-05-17T21:58:17.218643+0000","last_unstale":"2024-05-17T21:58:18.215329+0000","last_undegraded":"2024-05-17T21:58:18.215329+0000","last_fullsized":"2024-05-17T21:58:18.215329+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:07:27.597936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.208767+0000","last_change":"2024-05-17T21:58:16.185006+0000","last_active":"2024-05-17T21:58:17.208767+0000","last_peered":"2024-05-17T21:58:17.208767+0000","last_clean":"2024-05-17T21:58:17.208767+0000","last_became_active":"2024-05-17T21:58:16.184174+0000","last_became_peered":"2024-05-17T21:58:16.184174+0000","last_unstale":"2024-05-17T21:58:17.208767+0000","last_undegraded":"2024-05-17T21:58:17.208767+0000","last_fullsized":"2024-05-17T21:58:17.208767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:28.078457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214936+0000","last_change":"2024-05-17T21:58:17.215503+0000","last_active":"2024-05-17T21:58:18.214936+0000","last_peered":"2024-05-17T21:58:18.214936+0000","last_clean":"2024-05-17T21:58:18.214936+0000","last_became_active":"2024-05-17T21:58:17.215185+0000","last_became_peered":"2024-05-17T21:58:17.215185+0000","last_unstale":"2024-05-17T21:58:18.214936+0000","last_undegraded":"2024-05-17T21:58:18.214936+0000","last_fullsized":"2024-05-17T21:58:18.214936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:50:50.740232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710374+0000","last_change":"2024-05-17T21:58:16.187376+0000","last_active":"2024-05-17T21:58:18.710374+0000","last_peered":"2024-05-17T21:58:18.710374+0000","last_clean":"2024-05-17T21:58:18.710374+0000","last_became_active":"2024-05-17T21:58:16.186501+0000","last_became_peered":"2024-05-17T21:58:16.186501+0000","last_unstale":"2024-05-17T21:58:18.710374+0000","last_undegraded":"2024-05-17T21:58:18.710374+0000","last_fullsized":"2024-05-17T21:58:18.710374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:50:49.729238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253904+0000","last_change":"2024-05-17T21:58:17.238044+0000","last_active":"2024-05-17T21:58:17.253904+0000","last_peered":"2024-05-17T21:58:17.253904+0000","last_clean":"2024-05-17T21:58:17.253904+0000","last_became_active":"2024-05-17T21:58:17.236492+0000","last_became_peered":"2024-05-17T21:58:17.236492+0000","last_unstale":"2024-05-17T21:58:17.253904+0000","last_undegraded":"2024-05-17T21:58:17.253904+0000","last_fullsized":"2024-05-17T21:58:17.253904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:42:36.155454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.215345+0000","last_change":"2024-05-17T21:58:16.184943+0000","last_active":"2024-05-17T21:58:18.215345+0000","last_peered":"2024-05-17T21:58:18.215345+0000","last_clean":"2024-05-17T21:58:18.215345+0000","last_became_active":"2024-05-17T21:58:16.183529+0000","last_became_peered":"2024-05-17T21:58:16.183529+0000","last_unstale":"2024-05-17T21:58:18.215345+0000","last_undegraded":"2024-05-17T21:58:18.215345+0000","last_fullsized":"2024-05-17T21:58:18.215345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:37.886606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.216405+0000","last_change":"2024-05-17T21:58:17.205479+0000","last_active":"2024-05-17T21:58:18.216405+0000","last_peered":"2024-05-17T21:58:18.216405+0000","last_clean":"2024-05-17T21:58:18.216405+0000","last_became_active":"2024-05-17T21:58:17.204901+0000","last_became_peered":"2024-05-17T21:58:17.204901+0000","last_unstale":"2024-05-17T21:58:18.216405+0000","last_undegraded":"2024-05-17T21:58:18.216405+0000","last_fullsized":"2024-05-17T21:58:18.216405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:44.569768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214351+0000","last_change":"2024-05-17T21:58:16.183756+0000","last_active":"2024-05-17T21:58:18.214351+0000","last_peered":"2024-05-17T21:58:18.214351+0000","last_clean":"2024-05-17T21:58:18.214351+0000","last_became_active":"2024-05-17T21:58:16.181527+0000","last_became_peered":"2024-05-17T21:58:16.181527+0000","last_unstale":"2024-05-17T21:58:18.214351+0000","last_undegraded":"2024-05-17T21:58:18.214351+0000","last_fullsized":"2024-05-17T21:58:18.214351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:20.967111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.217422+0000","last_change":"2024-05-17T21:58:17.219478+0000","last_active":"2024-05-17T21:58:18.217422+0000","last_peered":"2024-05-17T21:58:18.217422+0000","last_clean":"2024-05-17T21:58:18.217422+0000","last_became_active":"2024-05-17T21:58:17.218451+0000","last_became_peered":"2024-05-17T21:58:17.218451+0000","last_unstale":"2024-05-17T21:58:18.217422+0000","last_undegraded":"2024-05-17T21:58:18.217422+0000","last_fullsized":"2024-05-17T21:58:18.217422+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:25:20.341116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.208352+0000","last_change":"2024-05-17T21:58:16.179380+0000","last_active":"2024-05-17T21:58:18.208352+0000","last_peered":"2024-05-17T21:58:18.208352+0000","last_clean":"2024-05-17T21:58:18.208352+0000","last_became_active":"2024-05-17T21:58:16.178966+0000","last_became_peered":"2024-05-17T21:58:16.178966+0000","last_unstale":"2024-05-17T21:58:18.208352+0000","last_undegraded":"2024-05-17T21:58:18.208352+0000","last_fullsized":"2024-05-17T21:58:18.208352+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:57.087708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.226127+0000","last_change":"2024-05-17T21:58:17.224753+0000","last_active":"2024-05-17T21:58:17.226127+0000","last_peered":"2024-05-17T21:58:17.226127+0000","last_clean":"2024-05-17T21:58:17.226127+0000","last_became_active":"2024-05-17T21:58:17.224411+0000","last_became_peered":"2024-05-17T21:58:17.224411+0000","last_unstale":"2024-05-17T21:58:17.226127+0000","last_undegraded":"2024-05-17T21:58:17.226127+0000","last_fullsized":"2024-05-17T21:58:17.226127+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:21.978105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.214143+0000","last_change":"2024-05-17T21:58:16.160805+0000","last_active":"2024-05-17T21:58:18.214143+0000","last_peered":"2024-05-17T21:58:18.214143+0000","last_clean":"2024-05-17T21:58:18.214143+0000","last_became_active":"2024-05-17T21:58:16.160333+0000","last_became_peered":"2024-05-17T21:58:16.160333+0000","last_unstale":"2024-05-17T21:58:18.214143+0000","last_undegraded":"2024-05-17T21:58:18.214143+0000","last_fullsized":"2024-05-17T21:58:18.214143+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:20:26.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.220416+0000","last_change":"2024-05-17T21:58:17.214929+0000","last_active":"2024-05-17T21:58:17.220416+0000","last_peered":"2024-05-17T21:58:17.220416+0000","last_clean":"2024-05-17T21:58:17.220416+0000","last_became_active":"2024-05-17T21:58:17.213949+0000","last_became_peered":"2024-05-17T21:58:17.213949+0000","last_unstale":"2024-05-17T21:58:17.220416+0000","last_undegraded":"2024-05-17T21:58:17.220416+0000","last_fullsized":"2024-05-17T21:58:17.220416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:10:58.098702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.209253+0000","last_change":"2024-05-17T21:58:16.179937+0000","last_active":"2024-05-17T21:58:17.209253+0000","last_peered":"2024-05-17T21:58:17.209253+0000","last_clean":"2024-05-17T21:58:17.209253+0000","last_became_active":"2024-05-17T21:58:16.179484+0000","last_became_peered":"2024-05-17T21:58:16.179484+0000","last_unstale":"2024-05-17T21:58:17.209253+0000","last_undegraded":"2024-05-17T21:58:17.209253+0000","last_fullsized":"2024-05-17T21:58:17.209253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:56.605637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.219863+0000","last_change":"2024-05-17T21:58:17.217200+0000","last_active":"2024-05-17T21:58:17.219863+0000","last_peered":"2024-05-17T21:58:17.219863+0000","last_clean":"2024-05-17T21:58:17.219863+0000","last_became_active":"2024-05-17T21:58:17.216861+0000","last_became_peered":"2024-05-17T21:58:17.216861+0000","last_unstale":"2024-05-17T21:58:17.219863+0000","last_undegraded":"2024-05-17T21:58:17.219863+0000","last_fullsized":"2024-05-17T21:58:17.219863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:48.018626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.213543+0000","last_change":"2024-05-17T21:58:16.164394+0000","last_active":"2024-05-17T21:58:18.213543+0000","last_peered":"2024-05-17T21:58:18.213543+0000","last_clean":"2024-05-17T21:58:18.213543+0000","last_became_active":"2024-05-17T21:58:16.163934+0000","last_became_peered":"2024-05-17T21:58:16.163934+0000","last_unstale":"2024-05-17T21:58:18.213543+0000","last_undegraded":"2024-05-17T21:58:18.213543+0000","last_fullsized":"2024-05-17T21:58:18.213543+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:15:56.605637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.229708+0000","last_change":"2024-05-17T21:58:17.225182+0000","last_active":"2024-05-17T21:58:17.229708+0000","last_peered":"2024-05-17T21:58:17.229708+0000","last_clean":"2024-05-17T21:58:17.229708+0000","last_became_active":"2024-05-17T21:58:17.224735+0000","last_became_peered":"2024-05-17T21:58:17.224735+0000","last_unstale":"2024-05-17T21:58:17.229708+0000","last_undegraded":"2024-05-17T21:58:17.229708+0000","last_fullsized":"2024-05-17T21:58:17.229708+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:24:55.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.711044+0000","last_change":"2024-05-17T21:58:16.196829+0000","last_active":"2024-05-17T21:58:18.711044+0000","last_peered":"2024-05-17T21:58:18.711044+0000","last_clean":"2024-05-17T21:58:18.711044+0000","last_became_active":"2024-05-17T21:58:16.195797+0000","last_became_peered":"2024-05-17T21:58:16.195797+0000","last_unstale":"2024-05-17T21:58:18.711044+0000","last_undegraded":"2024-05-17T21:58:18.711044+0000","last_fullsized":"2024-05-17T21:58:18.711044+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:53:43.102799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-17T21:58:18.710389+0000","last_change":"2024-05-17T21:58:17.224316+0000","last_active":"2024-05-17T21:58:18.710389+0000","last_peered":"2024-05-17T21:58:18.710389+0000","last_clean":"2024-05-17T21:58:18.710389+0000","last_became_active":"2024-05-17T21:58:17.221751+0000","last_became_peered":"2024-05-17T21:58:17.221751+0000","last_unstale":"2024-05-17T21:58:18.710389+0000","last_undegraded":"2024-05-17T21:58:18.710389+0000","last_fullsized":"2024-05-17T21:58:18.710389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:08.532377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.214833+0000","last_change":"2024-05-17T21:58:16.190454+0000","last_active":"2024-05-17T21:58:17.214833+0000","last_peered":"2024-05-17T21:58:17.214833+0000","last_clean":"2024-05-17T21:58:17.214833+0000","last_became_active":"2024-05-17T21:58:16.190016+0000","last_became_peered":"2024-05-17T21:58:16.190016+0000","last_unstale":"2024-05-17T21:58:17.214833+0000","last_undegraded":"2024-05-17T21:58:17.214833+0000","last_fullsized":"2024-05-17T21:58:17.214833+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:47.725441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.253736+0000","last_change":"2024-05-17T21:58:17.235801+0000","last_active":"2024-05-17T21:58:17.253736+0000","last_peered":"2024-05-17T21:58:17.253736+0000","last_clean":"2024-05-17T21:58:17.253736+0000","last_became_active":"2024-05-17T21:58:17.217044+0000","last_became_peered":"2024-05-17T21:58:17.217044+0000","last_unstale":"2024-05-17T21:58:17.253736+0000","last_undegraded":"2024-05-17T21:58:17.253736+0000","last_fullsized":"2024-05-17T21:58:17.253736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:16.143281+0000","last_clean_scrub_stamp":"2024-05-17T21:58:16.143281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:46:48.736435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.213183+0000","last_change":"2024-05-17T21:58:16.203141+0000","last_active":"2024-05-17T21:58:17.213183+0000","last_peered":"2024-05-17T21:58:17.213183+0000","last_clean":"2024-05-17T21:58:17.213183+0000","last_became_active":"2024-05-17T21:58:16.202480+0000","last_became_peered":"2024-05-17T21:58:16.202480+0000","last_unstale":"2024-05-17T21:58:17.213183+0000","last_undegraded":"2024-05-17T21:58:17.213183+0000","last_fullsized":"2024-05-17T21:58:17.213183+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:15.132287+0000","last_clean_scrub_stamp":"2024-05-17T21:58:15.132287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:18:43.558773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":128,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-17T21:58:17.205693+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:58:17.205693+0000","last_peered":"2024-05-17T21:58:17.205693+0000","last_clean":"2024-05-17T21:58:17.205693+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:58:17.205693+0000","last_undegraded":"2024-05-17T21:58:17.205693+0000","last_fullsized":"2024-05-17T21:58:17.205693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":718284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":134798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":127930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":129460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":134196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":135760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":718194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:58:21.701 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T21:58:21.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T21:58:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.894+0000 7fc00cc62640 1 Processor -- start 2024-05-17T21:58:21.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 -- start start 2024-05-17T21:58:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0080b08a0 0x7fc00815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 0x7fc0081605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00806f2d0 con 0x7fc00806eec0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc0080b3fe0 con 0x7fc0080b08a0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc00cc62640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc00815c3b0 con 0x7fc0081601b0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0080b08a0 0x7fc00815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0080b08a0 0x7fc00815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51786/0 (socket says 172.21.15.83:51786) 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 -- 172.21.15.83:0/431253664 learned_addr learned my addr 172.21.15.83:0/431253664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc006d9d640 1 --2- 172.21.15.83:0/431253664 >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 0x7fc0081605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 -- 172.21.15.83:0/431253664 >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 msgr2=0x7fc0081605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 --2- 172.21.15.83:0/431253664 >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 0x7fc0081605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 -- 172.21.15.83:0/431253664 >> v1:172.21.15.155:6789/0 conn(0x7fc00806eec0 legacy=0x7fc0080b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fbffdd9b640 1 -- 172.21.15.83:0/431253664 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00806f5a0 con 0x7fc0080b08a0 2024-05-17T21:58:21.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.898+0000 7fc006d9d640 1 --2- 172.21.15.83:0/431253664 >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 0x7fc0081605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fbffdd9b640 1 --2- 172.21.15.83:0/431253664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc00815be70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc0080ce2b0 tx=0x7fbff001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40cc59b74d36d82 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc005d9b640 1 -- 172.21.15.83:0/431253664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00ab620 con 0x7fc0080b08a0 2024-05-17T21:58:21.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc005d9b640 1 -- 172.21.15.83:0/431253664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff00b0070 con 0x7fc0080b08a0 2024-05-17T21:58:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc005d9b640 1 -- 172.21.15.83:0/431253664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00ab9e0 con 0x7fc0080b08a0 2024-05-17T21:58:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- 172.21.15.83:0/431253664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 msgr2=0x7fc00815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 --2- 172.21.15.83:0/431253664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc00815be70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc0080ce2b0 tx=0x7fbff001c460 comp rx=0 tx=0).stop 2024-05-17T21:58:21.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- 172.21.15.83:0/431253664 shutdown_connections 2024-05-17T21:58:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 --2- 172.21.15.83:0/431253664 >> v2:172.21.15.155:3300/0 conn(0x7fc0081601b0 0x7fc0081605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 --2- 172.21.15.83:0/431253664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc00815be70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- 172.21.15.83:0/431253664 >> 172.21.15.83:0/431253664 conn(0x7fc00806cd90 msgr2=0x7fc0080b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- 172.21.15.83:0/431253664 shutdown_connections 2024-05-17T21:58:21.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- 172.21.15.83:0/431253664 wait complete. 2024-05-17T21:58:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 Processor -- start 2024-05-17T21:58:21.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.902+0000 7fc00cc62640 1 -- start start 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806eec0 0x7fc008154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0081601b0 0x7fc00815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc008072e80 con 0x7fc0080b08a0 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0080b0cb0 con 0x7fc00806eec0 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc008073000 con 0x7fc0081601b0 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51796/0 (socket says 172.21.15.83:51796) 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 -- 172.21.15.83:0/3103696589 learned_addr learned my addr 172.21.15.83:0/3103696589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00659c640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806eec0 0x7fc008154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 -- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0081601b0 msgr2=0x7fc00815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0081601b0 0x7fc00815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 -- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806eec0 msgr2=0x7fc008154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806eec0 0x7fc008154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 -- 172.21.15.83:0/3103696589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0081538c0 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbffdd9b640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ad040 tx=0x7fbff001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00abd80 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff00be3f0 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00be7e0 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc008153ba0 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc008154160 con 0x7fc0080b08a0 2024-05-17T21:58:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00815be70 con 0x7fc0080b08a0 2024-05-17T21:58:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbff00b43f0 con 0x7fc0080b08a0 2024-05-17T21:58:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 0x7fbfe8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbff01293d0 con 0x7fc0080b08a0 2024-05-17T21:58:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.906+0000 7fc00659c640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 0x7fbfe8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:21.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.910+0000 7fc00659c640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 0x7fbfe8071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbff800b810 tx=0x7fbff8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:21.910+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbff00c4060 con 0x7fc0080b08a0 2024-05-17T21:58:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.122+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc0080b0f10 con 0x7fc0080b08a0 2024-05-17T21:58:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.126+0000 7fbfff7fe640 1 -- 172.21.15.83:0/3103696589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fbff0016110 con 0x7fc0080b08a0 2024-05-17T21:58:22.122 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T21:58:22.122 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T21:58:22.123 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T21:58:22.123 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T21:58:22.123 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T21:58:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 msgr2=0x7fbfe8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 0x7fbfe8071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbff800b810 tx=0x7fbff8008040 comp rx=0 tx=0).stop 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 msgr2=0x7fc008155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ad040 tx=0x7fbff001aca0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 shutdown_connections 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0081601b0 0x7fc00815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0080b08a0 0x7fc008155520 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbfe806f400 0x7fbfe8071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 --2- 172.21.15.83:0/3103696589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc00806eec0 0x7fc008154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 >> 172.21.15.83:0/3103696589 conn(0x7fc00806cd90 msgr2=0x7fc0080b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 shutdown_connections 2024-05-17T21:58:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.130+0000 7fc00cc62640 1 -- 172.21.15.83:0/3103696589 wait complete. 2024-05-17T21:58:22.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 Processor -- start 2024-05-17T21:58:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 -- start start 2024-05-17T21:58:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 0x7fadb8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadb8073370 0x7fadb806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8073780 con 0x7fadb80d1040 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fadb815e790 con 0x7fadb8073370 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadbda9b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fadb815ea40 con 0x7fadb8144080 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadb8073370 0x7fadb806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadb8073370 0x7fadb806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51806/0 (socket says 172.21.15.83:51806) 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 -- 172.21.15.83:0/705348063 learned_addr learned my addr 172.21.15.83:0/705348063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadaffff640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 0x7fadb8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 -- 172.21.15.83:0/705348063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 msgr2=0x7fadb8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 0x7fadb8072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 -- 172.21.15.83:0/705348063 >> v1:172.21.15.155:6789/0 conn(0x7fadb80d1040 legacy=0x7fadb815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadb7fff640 1 -- 172.21.15.83:0/705348063 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb815ed20 con 0x7fadb8073370 2024-05-17T21:58:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.282+0000 7fadaffff640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 0x7fadb8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadb7fff640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8073370 0x7fadb806f730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e50 tx=0x7fadb001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49caf3582f4530 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadb6ffd640 1 -- 172.21.15.83:0/705348063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00a5660 con 0x7fadb8073370 2024-05-17T21:58:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadb6ffd640 1 -- 172.21.15.83:0/705348063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00a5800 con 0x7fadb8073370 2024-05-17T21:58:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadb6ffd640 1 -- 172.21.15.83:0/705348063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00a5bd0 con 0x7fadb8073370 2024-05-17T21:58:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 -- 172.21.15.83:0/705348063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8073370 msgr2=0x7fadb806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8073370 0x7fadb806f730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e50 tx=0x7fadb001c820 comp rx=0 tx=0).stop 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 -- 172.21.15.83:0/705348063 shutdown_connections 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8073370 0x7fadb806f730 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 --2- 172.21.15.83:0/705348063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb8144080 0x7fadb8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 -- 172.21.15.83:0/705348063 >> 172.21.15.83:0/705348063 conn(0x7fadb806cd90 msgr2=0x7fadb80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 -- 172.21.15.83:0/705348063 shutdown_connections 2024-05-17T21:58:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.286+0000 7fadbda9b640 1 -- 172.21.15.83:0/705348063 wait complete. 2024-05-17T21:58:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 Processor -- start 2024-05-17T21:58:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- start start 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb8073370 0x7fadb8163840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb80d1040 0x7fadb8142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8070480 con 0x7fadb8144080 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8144490 con 0x7fadb8073370 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fadb8070600 con 0x7fadb80d1040 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51812/0 (socket says 172.21.15.83:51812) 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 -- 172.21.15.83:0/3489083235 learned_addr learned my addr 172.21.15.83:0/3489083235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadaffff640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb80d1040 0x7fadb8142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb77fe640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb8073370 0x7fadb8163840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 -- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb80d1040 msgr2=0x7fadb8142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb80d1040 0x7fadb8142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 -- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb8073370 msgr2=0x7fadb8163840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb8073370 0x7fadb8163840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 -- 172.21.15.83:0/3489083235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb8164070 con 0x7fadb8144080 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb7fff640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fadb00af040 tx=0x7fadb00195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00b8670 con 0x7fadb8144080 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00be070 con 0x7fadb8144080 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb8164350 con 0x7fadb8144080 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00b8a50 con 0x7fadb8144080 2024-05-17T21:58:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb8140890 con 0x7fadb8144080 2024-05-17T21:58:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.290+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8069c10 con 0x7fadb8144080 2024-05-17T21:58:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.294+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fadb00b8bf0 con 0x7fadb8144080 2024-05-17T21:58:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.294+0000 7fadb4ff9640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 0x7fad98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.294+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fadb012a4b0 con 0x7fadb8144080 2024-05-17T21:58:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.294+0000 7fadb77fe640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 0x7fad98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.294+0000 7fadb77fe640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 0x7fad98071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fada800b810 tx=0x7fada8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:22.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.298+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fadb8073780 con 0x7fadb8144080 2024-05-17T21:58:22.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.482+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadb8163e00 con 0x7fadb8144080 2024-05-17T21:58:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.482+0000 7fadb4ff9640 1 -- 172.21.15.83:0/3489083235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fadb00f73a0 con 0x7fadb8144080 2024-05-17T21:58:22.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:22.478 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:22.479 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:22.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 msgr2=0x7fad98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 0x7fad98071910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fada800b810 tx=0x7fada8008040 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 msgr2=0x7fadb81430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fadb00af040 tx=0x7fadb00195c0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 shutdown_connections 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb8144080 0x7fadb81430d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadb80d1040 0x7fadb8142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad9806f400 0x7fad98071910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 --2- 172.21.15.83:0/3489083235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadb8073370 0x7fadb8163840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 >> 172.21.15.83:0/3489083235 conn(0x7fadb806cd90 msgr2=0x7fadb80b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 shutdown_connections 2024-05-17T21:58:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:22.486+0000 7fadbda9b640 1 -- 172.21.15.83:0/3489083235 wait complete. 2024-05-17T21:58:22.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T21:58:16.566175+0000', 'modified': '2024-05-17T21:58:19.619567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4776': {'gid': 4776, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/3630234827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 3630234827}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3630234827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4776, 'qdb_cluster': [4776]} 2024-05-17T21:58:22.901 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T21:58:22.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T21:58:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 Processor -- start 2024-05-17T21:58:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 -- start start 2024-05-17T21:58:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f861c15ea50 0x7f861c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f861c06ccd0 0x7f861c15f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f861c15bae0 con 0x7f861c15ea50 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f861c06d0e0 con 0x7f861c06ccd0 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f862219e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f861c0af470 con 0x7f861c0b4e50 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f861c15ea50 0x7f861c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f861c15ea50 0x7f861c0b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51828/0 (socket says 172.21.15.83:51828) 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 -- 172.21.15.83:0/3863475142 learned_addr learned my addr 172.21.15.83:0/3863475142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861affd640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861bfff640 1 --2- 172.21.15.83:0/3863475142 >> v2:172.21.15.155:3300/0 conn(0x7f861c06ccd0 0x7f861c15f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 -- 172.21.15.83:0/3863475142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 msgr2=0x7f861c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c06c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 -- 172.21.15.83:0/3863475142 >> v2:172.21.15.155:3300/0 conn(0x7f861c06ccd0 msgr2=0x7f861c15f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 --2- 172.21.15.83:0/3863475142 >> v2:172.21.15.155:3300/0 conn(0x7f861c06ccd0 0x7f861c15f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.066+0000 7f861b7fe640 1 -- 172.21.15.83:0/3863475142 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c15e290 con 0x7f861c15ea50 2024-05-17T21:58:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f861b7fe640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c15ea50 0x7f861c0b4910 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f861c072ea0 tx=0x7f860c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929541683f9fc39c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f861a7fc640 1 -- 172.21.15.83:0/3863475142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b0070 con 0x7f861c15ea50 2024-05-17T21:58:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f861a7fc640 1 -- 172.21.15.83:0/3863475142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c0ab600 con 0x7f861c15ea50 2024-05-17T21:58:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f861a7fc640 1 -- 172.21.15.83:0/3863475142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0ab9d0 con 0x7f861c15ea50 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 -- 172.21.15.83:0/3863475142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c15ea50 msgr2=0x7f861c0b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c15ea50 0x7f861c0b4910 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f861c072ea0 tx=0x7f860c01c490 comp rx=0 tx=0).stop 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 -- 172.21.15.83:0/3863475142 shutdown_connections 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 --2- 172.21.15.83:0/3863475142 >> v2:172.21.15.155:3300/0 conn(0x7f861c06ccd0 0x7f861c15f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c06c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 --2- 172.21.15.83:0/3863475142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c15ea50 0x7f861c0b4910 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 -- 172.21.15.83:0/3863475142 >> 172.21.15.83:0/3863475142 conn(0x7f861c09b8b0 msgr2=0x7f861c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 -- 172.21.15.83:0/3863475142 shutdown_connections 2024-05-17T21:58:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.070+0000 7f862219e640 1 -- 172.21.15.83:0/3863475142 wait complete. 2024-05-17T21:58:23.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 Processor -- start 2024-05-17T21:58:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- start start 2024-05-17T21:58:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 0x7f861c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f861c15ee60 con 0x7f861c06ccd0 2024-05-17T21:58:23.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f861c0b3f10 con 0x7f861c15ea50 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f861c15efe0 con 0x7f861c0b4e50 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51844/0 (socket says 172.21.15.83:51844) 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 -- 172.21.15.83:0/422186242 learned_addr learned my addr 172.21.15.83:0/422186242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861affd640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861bfff640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 0x7f861c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 -- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 msgr2=0x7f861c150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 -- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 msgr2=0x7f861c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 0x7f861c1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 -- 172.21.15.83:0/422186242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c1578e0 con 0x7f861c06ccd0 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861bfff640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 0x7f861c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f861b7fe640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ae040 tx=0x7f860c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f8603fff640 1 -- 172.21.15.83:0/422186242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0ba020 con 0x7f861c06ccd0 2024-05-17T21:58:23.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c153be0 con 0x7f861c06ccd0 2024-05-17T21:58:23.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c1541a0 con 0x7f861c06ccd0 2024-05-17T21:58:23.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f8603fff640 1 -- 172.21.15.83:0/422186242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c0ab600 con 0x7f861c06ccd0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.074+0000 7f8603fff640 1 -- 172.21.15.83:0/422186242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b0040 con 0x7f861c06ccd0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8603fff640 1 -- 172.21.15.83:0/422186242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f860c0bd020 con 0x7f861c06ccd0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8603fff640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 0x7f8608071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8603fff640 1 -- 172.21.15.83:0/422186242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f860c129280 con 0x7f861c06ccd0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f861affd640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 0x7f8608071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f861affd640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 0x7f8608071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f861c0770f0 tx=0x7f8610008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 Processor -- start 2024-05-17T21:58:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 -- start start 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 0x55fe365c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe36473600 0x55fe365c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe36461ca0 con 0x55fe3645cd30 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe363d4110 con 0x55fe3647dfa0 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.078+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55fe363c3cb0 con 0x55fe36473600 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51848/0 (socket says 172.21.15.83:51848) 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 -- 172.21.15.83:0/4020670813 learned_addr learned my addr 172.21.15.83:0/4020670813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861affd640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe36473600 0x55fe365c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861b7fe640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 0x55fe365c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 -- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe36473600 msgr2=0x55fe365c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe36473600 0x55fe365c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 -- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 msgr2=0x55fe365c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 0x55fe365c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 -- 172.21.15.83:0/4020670813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe36388370 con 0x55fe3645cd30 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861b7fe640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 0x55fe365c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f861bfff640 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8614009f20 tx=0x7f86140077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7100c90a1145c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f86017fa640 1 -- 172.21.15.83:0/4020670813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861401a070 con 0x55fe3645cd30 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f86017fa640 1 -- 172.21.15.83:0/4020670813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8614003db0 con 0x55fe3645cd30 2024-05-17T21:58:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f86017fa640 1 -- 172.21.15.83:0/4020670813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614004230 con 0x55fe3645cd30 2024-05-17T21:58:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- 172.21.15.83:0/4020670813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 msgr2=0x55fe365c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8614009f20 tx=0x7f86140077c0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- 172.21.15.83:0/4020670813 shutdown_connections 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe3645cd30 0x55fe365c4330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe36473600 0x55fe365c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 --2- 172.21.15.83:0/4020670813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3647dfa0 0x55fe365c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- 172.21.15.83:0/4020670813 >> 172.21.15.83:0/4020670813 conn(0x55fe36085460 msgr2=0x55fe36578e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- 172.21.15.83:0/4020670813 shutdown_connections 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- 172.21.15.83:0/4020670813 wait complete. 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 -- start start 2024-05-17T21:58:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.082+0000 7f8629418000 1 Processor -- start 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 0x55fe36580290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3647dfa0 0x55fe36585460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe3637fd40 con 0x55fe36473600 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55fe363c7cf0 con 0x55fe3645cd30 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55fe3636ce90 con 0x55fe3647dfa0 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 10 client.0 _ll_register_callbacks cb 0x55fe363a1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51856/0 (socket says 172.21.15.83:51856) 2024-05-17T21:58:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 -- 172.21.15.83:0/1258040044 learned_addr learned my addr 172.21.15.83:0/1258040044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861b7fe640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 0x55fe36580290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861bfff640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3647dfa0 0x55fe36585460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 -- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3647dfa0 msgr2=0x55fe36585460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3647dfa0 0x55fe36585460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 -- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 msgr2=0x55fe36580290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 0x55fe36580290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 -- 172.21.15.83:0/1258040044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe363926e0 con 0x55fe36473600 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861b7fe640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 0x55fe36580290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861affd640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f861000f7a0 tx=0x7f8610004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf935c9bc915fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8610025020 con 0x55fe36473600 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8610021830 con 0x55fe36473600 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8614003db0 con 0x55fe36473600 2024-05-17T21:58:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f861002f050 con 0x55fe36473600 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 10 client.4944 fetch_fsmap learned FSMap version 16 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fe36382690 con 0x55fe36473600 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8614003db0 con 0x55fe36473600 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f8610032020 con 0x55fe36473600 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 10 client.4944 fetch_fsmap finished waiting for FSMap version 16 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 10 client.4944 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 20 client.4944 populate_metadata read hostname 'smithi083' 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe3606a7d0 0x55fe365ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 4 client.4944 mds_command: new command op to 4776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 --> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe3635fa00 con 0x55fe3606a7d0 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861b7fe640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe3606a7d0 0x55fe365ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 10 client.4944 ms_handle_connect on v2:172.21.15.83:6835/3630234827 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f861b7fe640 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe3606a7d0 0x55fe365ca920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f1943540ae9e57 server_cookie=3cd81924ab55f568 in_seq=0 out_seq=0 2024-05-17T21:58:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 1 -- 172.21.15.83:0/1258040044 <== mds.0 v2:172.21.15.83:6835/3630234827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe3635fa00 con 0x55fe3606a7d0 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 10 client.4944 handle_command_reply: tid=0 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 client.4944 shutdown 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 msgr2=0x55fe365807d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f861000f7a0 tx=0x7f8610004740 comp rx=0 tx=0).stop 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 shutdown_connections 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe3606a7d0 0x55fe365ca920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3647dfa0 0x55fe36585460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f85feff5640 0 client.4944 ms_handle_reset on v2:172.21.15.83:6835/3630234827 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe365807d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 --2- 172.21.15.83:0/1258040044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe3645cd30 0x55fe36580290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 >> 172.21.15.83:0/1258040044 conn(0x55fe36085460 msgr2=0x55fe36578e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 shutdown_connections 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 1 -- 172.21.15.83:0/1258040044 wait complete. 2024-05-17T21:58:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.086+0000 7f8629418000 20 client.4944 trim_cache size 0 max 16384 2024-05-17T21:58:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 Processor -- start 2024-05-17T21:58:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- start start 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36085460 0x55fe36632220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36632760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85f800c7c0 con 0x55fe36473600 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85f800b7a0 con 0x55fe36085460 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85f800aa80 con 0x55fe3645cd30 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51858/0 (socket says 172.21.15.83:51858) 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 -- 172.21.15.83:0/2726580827 learned_addr learned my addr 172.21.15.83:0/2726580827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861b7fe640 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36085460 0x55fe36632220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861affd640 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36632760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 -- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 msgr2=0x55fe36632760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36632760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 -- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36085460 msgr2=0x55fe36632220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36085460 0x55fe36632220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 -- 172.21.15.83:0/2726580827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3644acb0 con 0x55fe36473600 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f861bfff640 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8614018040 tx=0x7f8614013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d09963bacdfa00 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8600ff9640 1 -- 172.21.15.83:0/2726580827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614025020 con 0x55fe36473600 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8600ff9640 1 -- 172.21.15.83:0/2726580827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861401b040 con 0x55fe36473600 2024-05-17T21:58:23.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8600ff9640 1 -- 172.21.15.83:0/2726580827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861401a040 con 0x55fe36473600 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- 172.21.15.83:0/2726580827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 msgr2=0x55fe36632ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8614018040 tx=0x7f8614013000 comp rx=0 tx=0).stop 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- 172.21.15.83:0/2726580827 shutdown_connections 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36473600 0x55fe36632ca0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36632760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 --2- 172.21.15.83:0/2726580827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36085460 0x55fe36632220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- 172.21.15.83:0/2726580827 >> 172.21.15.83:0/2726580827 conn(0x55fe3657cc60 msgr2=0x55fe3657f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- 172.21.15.83:0/2726580827 shutdown_connections 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.106+0000 7f8629418000 1 -- 172.21.15.83:0/2726580827 wait complete. 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- start start 2024-05-17T21:58:23.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 Processor -- start 2024-05-17T21:58:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36636590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36473600 0x55fe36636ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe3644a820 con 0x55fe36085460 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85f800b0d0 con 0x55fe36473600 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55fe36392c10 con 0x55fe3645cd30 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 10 client.0 _ll_register_callbacks cb 0x7f861000b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51864/0 (socket says 172.21.15.83:51864) 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 -- 172.21.15.83:0/2071197749 learned_addr learned my addr 172.21.15.83:0/2071197749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 -- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 msgr2=0x55fe36636590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36636590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 -- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36473600 msgr2=0x55fe36636ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36473600 0x55fe36636ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 -- 172.21.15.83:0/2071197749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe363834f0 con 0x55fe36085460 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861b7fe640 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f860c0f5960 tx=0x7f860c0f6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb57443f4f2a0cce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c127dd0 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f860c0fe5a0 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f861401b040 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f860c1290b0 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 10 client.4950 fetch_fsmap learned FSMap version 16 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fe3638d4f0 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861401b040 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f860c0b3030 con 0x55fe36085460 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 10 client.4950 fetch_fsmap finished waiting for FSMap version 16 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 10 client.4950 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 20 client.4950 populate_metadata read hostname 'smithi083' 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe36083e80 0x55fe3657a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 4 client.4950 mds_command: new command op to 4776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 --> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] -- command(tid 0: {"prefix": "status"}) -- 0x55fe363d6060 con 0x55fe36083e80 2024-05-17T21:58:23.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861affd640 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe36083e80 0x55fe3657a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 10 client.4950 ms_handle_connect on v2:172.21.15.83:6835/3630234827 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f861affd640 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe36083e80 0x55fe3657a8a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccd8258bd1e0795 server_cookie=ec363b2907f6d948 in_seq=0 out_seq=0 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 1 -- 172.21.15.83:0/2071197749 <== mds.0 v2:172.21.15.83:6835/3630234827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fe363d6060 con 0x55fe36083e80 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f85feff5640 10 client.4950 handle_command_reply: tid=0 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.110+0000 7f8629418000 1 client.4950 shutdown 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 msgr2=0x55fe36636050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f860c0f5960 tx=0x7f860c0f6f90 comp rx=0 tx=0).stop 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 shutdown_connections 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:6835/3630234827,v1:172.21.15.83:6837/3630234827] conn(0x55fe36083e80 0x55fe3657a8a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe36473600 0x55fe36636ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55fe3645cd30 0x55fe36636590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f85feff5640 0 client.4950 ms_handle_reset on v2:172.21.15.83:6835/3630234827 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 --2- 172.21.15.83:0/2071197749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe36085460 0x55fe36636050 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 >> 172.21.15.83:0/2071197749 conn(0x55fe3657cc60 msgr2=0x55fe365c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 shutdown_connections 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 1 -- 172.21.15.83:0/2071197749 wait complete. 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f8629418000 20 client.4950 trim_cache size 0 max 16384 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4776, 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T21:58:23.113 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.9159564170000003, 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 16, 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 31, 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 31, 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 12.837464861000001 2024-05-17T21:58:23.114 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 msgr2=0x7f8608071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 0x7f8608071910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f861c0770f0 tx=0x7f8610008040 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 msgr2=0x7f861c14e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f860c0ae040 tx=0x7f860c01acd0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 shutdown_connections 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f860806f400 0x7f8608071910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f861c15ea50 0x7f861c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f861c0b4e50 0x7f861c150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 --2- 172.21.15.83:0/422186242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f861c06ccd0 0x7f861c14e710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 >> 172.21.15.83:0/422186242 conn(0x7f861c09b8b0 msgr2=0x7f861c160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 shutdown_connections 2024-05-17T21:58:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.114+0000 7f862219e640 1 -- 172.21.15.83:0/422186242 wait complete. 2024-05-17T21:58:23.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:58:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 Processor -- start 2024-05-17T21:58:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 -- start start 2024-05-17T21:58:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39b8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b815c830 con 0x7f39b815fe20 2024-05-17T21:58:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b81608a0 con 0x7f39b8073540 2024-05-17T21:58:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39bccc7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39b8160b80 con 0x7f39b815d2f0 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f39b815fe20 0x7f39b8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41628/0 (socket says 172.21.15.83:41628) 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b5d9b640 1 -- 172.21.15.83:0/2184684145 learned_addr learned my addr 172.21.15.83:0/2184684145 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b659c640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39b8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1773290526 0 0) 0x7f39b815c830 con 0x7f39b815fe20 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39a4003530 con 0x7f39b815fe20 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3576319781 0 0) 0x7f39a4003530 con 0x7f39b815fe20 2024-05-17T21:58:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.266+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 msgr2=0x7f39b8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39b8159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 >> v1:172.21.15.155:6789/0 conn(0x7f39b8073540 legacy=0x7f39b80b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b815caf0 con 0x7f39b815fe20 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f39a0001700 con 0x7f39b815fe20 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 >> v1:172.21.15.83:6789/0 conn(0x7f39b815fe20 legacy=0x7f39b8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39a40039f0 0x7f39a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8160b80 con 0x7f39a4004d50 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39a400dd70 con 0x7f39a40039f0 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39a400df20 con 0x7f39b815d2f0 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b5d9b640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b659c640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39a40039f0 0x7f39a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 -- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 msgr2=0x7f39a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b815d2f0 0x7f39a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 -- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39a40039f0 msgr2=0x7f39a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39a40039f0 0x7f39a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 -- 172.21.15.83:0/2184684145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a4003530 con 0x7f39a4004d50 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39a4005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39b0001f90 tx=0x7f39b001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d53800b34aa4c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0001be0 con 0x7f39a4004d50 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b00a66d0 con 0x7f39a4004d50 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b559a640 1 -- 172.21.15.83:0/2184684145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00a6ac0 con 0x7f39a4004d50 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 -- 172.21.15.83:0/2184684145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 msgr2=0x7f39a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 --2- 172.21.15.83:0/2184684145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39a4005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39b0001f90 tx=0x7f39b001c8c0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39b6d9d640 1 -- 172.21.15.83:0/2184684145 reap_dead start 2024-05-17T21:58:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 -- 172.21.15.83:0/2184684145 shutdown_connections 2024-05-17T21:58:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 -- 172.21.15.83:0/2184684145 >> 172.21.15.83:0/2184684145 conn(0x7f39b806cd90 msgr2=0x7f39b80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 -- 172.21.15.83:0/2184684145 shutdown_connections 2024-05-17T21:58:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.270+0000 7f39bccc7640 1 -- 172.21.15.83:0/2184684145 wait complete. 2024-05-17T21:58:23.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 Processor -- start 2024-05-17T21:58:23.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 -- start start 2024-05-17T21:58:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b814a000 0x7f39b815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b815b870 0x7f39b81581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8160230 con 0x7f39a4004d50 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d10b0 con 0x7f39b815b870 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39bccc7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39b81603b0 con 0x7f39b814a000 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51880/0 (socket says 172.21.15.83:51880) 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.274+0000 7f39b659c640 1 -- 172.21.15.83:0/1500299038 learned_addr learned my addr 172.21.15.83:0/1500299038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b5d9b640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b814a000 0x7f39b815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b6d9d640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b815b870 0x7f39b81581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 -- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b814a000 msgr2=0x7f39b815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b814a000 0x7f39b815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 -- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b815b870 msgr2=0x7f39b81581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b815b870 0x7f39b81581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 -- 172.21.15.83:0/1500299038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b815c040 con 0x7f39a4004d50 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b659c640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f39a80106e0 tx=0x7f39a8010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8019070 con 0x7f39a4004d50 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a800ec60 con 0x7f39a4004d50 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b80c6540 con 0x7f39a4004d50 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a800b040 con 0x7f39a4004d50 2024-05-17T21:58:23.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b80c6b00 con 0x7f39a4004d50 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b80b6860 con 0x7f39a4004d50 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39a801c070 con 0x7f39a4004d50 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 0x7f3998071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f39a808f1e0 con 0x7f39a4004d50 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b5d9b640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 0x7f3998071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:23.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.278+0000 7f39b5d9b640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 0x7f3998071910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39b80b6550 tx=0x7f39a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:23.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.282+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f39a805c160 con 0x7f39a4004d50 2024-05-17T21:58:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.466+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39b80b6580 con 0x7f39a4004d50 2024-05-17T21:58:23.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.466+0000 7f39aeffd640 1 -- 172.21.15.83:0/1500299038 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f39a805ba60 con 0x7f39a4004d50 2024-05-17T21:58:23.463 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 msgr2=0x7f3998071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 0x7f3998071910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39b80b6550 tx=0x7f39a0009040 comp rx=0 tx=0).stop 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 msgr2=0x7f39b80c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f39a80106e0 tx=0x7f39a8010ba0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 shutdown_connections 2024-05-17T21:58:23.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f399806f400 0x7f3998071910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39b815b870 0x7f39b81581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39b814a000 0x7f39b815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 --2- 172.21.15.83:0/1500299038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39a4004d50 0x7f39b80c7300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 >> 172.21.15.83:0/1500299038 conn(0x7f39b806cd90 msgr2=0x7f39b80b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 shutdown_connections 2024-05-17T21:58:23.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23.470+0000 7f39bccc7640 1 -- 172.21.15.83:0/1500299038 wait complete. 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:58:23.483 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:58:23.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.483 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:58:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42441sec preferred_lft 42441sec 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:58:23.491 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:58:23.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.493 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:58:23.540 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T21:58:23.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.542 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:58:23.593 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:58:23.594 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:58:23.594 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:58:23.594 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:58:23.594 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:58:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:58:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:23.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.714 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T21:58:23.720 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T21:58:23.720 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T21:58:23.720 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:58:23.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T21:58:23.775 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T21:58:23.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:58:23.775 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:23.820 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:58:23.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:58:23.820 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:23.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:58:23.926 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:58:23.973 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:58:23.973 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:58:23.974 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:23.975 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:23.976 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:23.976 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:23.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:23.976 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:58:24.025 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:58:24.025 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T21:58:24.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:58:24.085 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:58:24.133 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:58:24.134 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:24.135 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:24.136 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:24.136 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:24.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:24.136 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:58:24.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T21:58:24.186+0000 7f6c584bc580 -1 init, newargv = 0x7f6c44005ec0 newargc=15 2024-05-17T21:58:24.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[36634]: starting ceph client 2024-05-17T21:58:24.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[36634]: starting fuse 2024-05-17T21:58:25.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:58:25.302 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:58:25.350 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:58:25.351 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:58:25.352 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:25.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.353 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:58:25.400 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:58:25.401 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T21:58:25.401 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T21:58:25.402 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:58:25.403 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:25.403 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T21:58:25.403 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:25.525 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.36634.asok 2024-05-17T21:58:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:25.537 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36634.asok 2024-05-17T21:58:25.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36634.asok status 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T21:58:25.720 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "36634", 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4959, 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 4959 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:58:25.721 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1678926094 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1678926094 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.4959 192.168.0.1:0/1678926094", 2024-05-17T21:58:25.722 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1678926094", 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 16, 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 31, 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T21:58:25.723 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:58:25.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.737 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T21:58:25.737 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:25.743 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T21:58:25.743 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:25.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:25.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.798 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-17T21:58:25.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.853 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-17T21:58:25.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.909 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-17T21:58:25.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:25.961 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-17T21:58:26.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:26.017 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-17T21:58:26.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:26.070 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./rfile 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/pfile 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/childdir 2024-05-17T21:58:26.137 INFO:teuthology.orchestra.run.smithi083.stdout:./parentdir/childdir/cfile 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-17T21:58:26.138 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.241 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T21:58:26.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.251 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.251 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.251 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.251 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.251 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-17T21:58:26.252 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.351 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2024-05-17T21:58:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-17T21:58:26.361 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.468 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T21:58:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-17T21:58:26.478 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.583 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628279 2024-05-17T21:58:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-17T21:58:26.593 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.698 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T21:58:26.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-17T21:58:26.708 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:58:26.812 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628280 2024-05-17T21:58:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:58:26.822 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:26.822 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:26.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:27.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 Processor -- start 2024-05-17T21:58:27.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 -- start start 2024-05-17T21:58:27.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b140d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140b5020 con 0x7f5b140cf380 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1409cde0 con 0x7f5b140d2120 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1b4c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140cdee0 con 0x7f5b140d17d0 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b19a5d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5b140cf380 0x7f5b1406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53766/0 (socket says 172.21.15.83:53766) 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1813803132 learned_addr learned my addr 172.21.15.83:0/1813803132 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b140d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2668286354 0 0) 0x7f5b0c0014a0 con 0x7f5b140cf380 2024-05-17T21:58:27.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.074+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b00003530 con 0x7f5b140cf380 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2403835179 0 0) 0x7f5b00003530 con 0x7f5b140cf380 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 msgr2=0x7f5b140d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b140d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 >> v1:172.21.15.155:6789/0 conn(0x7f5b140d2120 legacy=0x7f5b1406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1409c540 con 0x7f5b140cf380 2024-05-17T21:58:27.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f5b0c001c50 con 0x7f5b140cf380 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 >> v1:172.21.15.83:6789/0 conn(0x7f5b140cf380 legacy=0x7f5b1406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b00003a40 0x7f5b00003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00004e70 0x7f5b000052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140cdee0 con 0x7f5b00004e70 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0000bcd0 con 0x7f5b00003a40 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0000bed0 con 0x7f5b140d17d0 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00004e70 0x7f5b000052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b00003a40 0x7f5b00003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b18a5b640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 msgr2=0x7f5b00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b00003a40 msgr2=0x7f5b00003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b00003a40 0x7f5b00003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1813803132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b00003530 con 0x7f5b00004e70 2024-05-17T21:58:27.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b18a5b640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b140d17d0 0x7f5b00004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00004e70 0x7f5b000052a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b1408cef0 tx=0x7f5b0c01d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36da85222d4bdf0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00a650 con 0x7f5b00004e70 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0ad630 con 0x7f5b00004e70 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b0bfff640 1 -- 172.21.15.83:0/1813803132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0ada20 con 0x7f5b00004e70 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1813803132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00004e70 msgr2=0x7f5b000052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1813803132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b00004e70 0x7f5b000052a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b1408cef0 tx=0x7f5b0c01d280 comp rx=0 tx=0).stop 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1813803132 reap_dead start 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1813803132 shutdown_connections 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1813803132 >> 172.21.15.83:0/1813803132 conn(0x7f5b140d36d0 msgr2=0x7f5b140d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1813803132 shutdown_connections 2024-05-17T21:58:27.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.078+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1813803132 wait complete. 2024-05-17T21:58:27.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 Processor -- start 2024-05-17T21:58:27.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 -- start start 2024-05-17T21:58:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 0x7f5b1416a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 0x7f5b14154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140d2530 con 0x7f5b1415d270 2024-05-17T21:58:27.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140b8dc0 con 0x7f5b140b7e70 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1b4c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140d26b0 con 0x7f5b14166950 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60230/0 (socket says 172.21.15.83:60230) 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1377743566 learned_addr learned my addr 172.21.15.83:0/1377743566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b18a5b640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 0x7f5b14154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 0x7f5b1416a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 msgr2=0x7f5b14154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 0x7f5b14154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 msgr2=0x7f5b1416a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 0x7f5b1416a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 -- 172.21.15.83:0/1377743566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1415dbc0 con 0x7f5b1415d270 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 0x7f5b1416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b18a5b640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 0x7f5b14154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b19a5d640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01d2c0 tx=0x7f5b00005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b2040 con 0x7f5b1415d270 2024-05-17T21:58:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.082+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c001410 con 0x7f5b1415d270 2024-05-17T21:58:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b140b84e0 con 0x7f5b1415d270 2024-05-17T21:58:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00a650 con 0x7f5b1415d270 2024-05-17T21:58:27.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b14166d60 con 0x7f5b1415d270 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1406d8a0 con 0x7f5b1415d270 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b0c004b70 con 0x7f5b1415d270 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b09ffb640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 0x7f5af8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 0x7f5af8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f5b0c128f70 con 0x7f5b1415d270 2024-05-17T21:58:27.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.086+0000 7f5b1925c640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 0x7f5af8071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1416adf0 tx=0x7f5b04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.090+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5b1415dbc0 con 0x7f5b1415d270 2024-05-17T21:58:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b1415b220 con 0x7f5b1415d270 2024-05-17T21:58:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b09ffb640 1 -- 172.21.15.83:0/1377743566 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) ==== 80+0+0 (secure 0 0 0) 0x7f5b140b5020 con 0x7f5b1415d270 2024-05-17T21:58:27.261 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 msgr2=0x7f5af8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 0x7f5af8071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1416adf0 tx=0x7f5b04008040 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 msgr2=0x7f5b1415d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c01d2c0 tx=0x7f5b00005840 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 shutdown_connections 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b1415d270 0x7f5b1415d680 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b14166950 0x7f5b14154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5af806f400 0x7f5af8071910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 --2- 172.21.15.83:0/1377743566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b140b7e70 0x7f5b1416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 >> 172.21.15.83:0/1377743566 conn(0x7f5b140d36d0 msgr2=0x7f5b140711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 shutdown_connections 2024-05-17T21:58:27.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.266+0000 7f5b1b4c0640 1 -- 172.21.15.83:0/1377743566 wait complete. 2024-05-17T21:58:27.282 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:27.282 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:27.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:58:27.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 Processor -- start 2024-05-17T21:58:27.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 -- start start 2024-05-17T21:58:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0b3d00 0x7f22ec0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22ec143f20 0x7f22ec072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec0737d0 con 0x7f22ec0733c0 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec0b4110 con 0x7f22ec143f20 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22f11ae640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec070770 con 0x7f22ec0b3d00 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ea59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22ec143f20 0x7f22ec072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22eb59e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f22ec0733c0 0x7f22ec06eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53778/0 (socket says 172.21.15.83:53778) 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22eb59e640 1 -- 172.21.15.83:0/1751072818 learned_addr learned my addr 172.21.15.83:0/1751072818 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ead9d640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0b3d00 0x7f22ec0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ea59c640 1 -- 172.21.15.83:0/1751072818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0b3d00 msgr2=0x7f22ec0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ea59c640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0b3d00 0x7f22ec0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ea59c640 1 -- 172.21.15.83:0/1751072818 >> v1:172.21.15.83:6789/0 conn(0x7f22ec0733c0 legacy=0x7f22ec06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22ea59c640 1 -- 172.21.15.83:0/1751072818 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0d10d0 con 0x7f22ec143f20 2024-05-17T21:58:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.494+0000 7f22e9d9b640 1 -- 172.21.15.83:0/1751072818 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106105511 0 0) 0x7f22ec0737d0 con 0x7f22ec0733c0 2024-05-17T21:58:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22ea59c640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec143f20 0x7f22ec072e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097c0 tx=0x7f22e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8ac6cdc1d9959d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22e9d9b640 1 -- 172.21.15.83:0/1751072818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00b2070 con 0x7f22ec143f20 2024-05-17T21:58:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22e9d9b640 1 -- 172.21.15.83:0/1751072818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00ad640 con 0x7f22ec143f20 2024-05-17T21:58:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22e9d9b640 1 -- 172.21.15.83:0/1751072818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00ada10 con 0x7f22ec143f20 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 -- 172.21.15.83:0/1751072818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec143f20 msgr2=0x7f22ec072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec143f20 0x7f22ec072e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097c0 tx=0x7f22e001c4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 -- 172.21.15.83:0/1751072818 shutdown_connections 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec143f20 0x7f22ec072e80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 --2- 172.21.15.83:0/1751072818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0b3d00 0x7f22ec0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 -- 172.21.15.83:0/1751072818 >> 172.21.15.83:0/1751072818 conn(0x7f22ec06cd90 msgr2=0x7f22ec0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 -- 172.21.15.83:0/1751072818 shutdown_connections 2024-05-17T21:58:27.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.498+0000 7f22f11ae640 1 -- 172.21.15.83:0/1751072818 wait complete. 2024-05-17T21:58:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 Processor -- start 2024-05-17T21:58:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- start start 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0733c0 0x7f22ec15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 0x7f22ec13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec070c00 con 0x7f22ec0b3d00 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec144330 con 0x7f22ec143f20 2024-05-17T21:58:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec140740 con 0x7f22ec0733c0 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60238/0 (socket says 172.21.15.83:60238) 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 -- 172.21.15.83:0/3216447211 learned_addr learned my addr 172.21.15.83:0/3216447211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ead9d640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0733c0 0x7f22ec15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22eb59e640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 0x7f22ec13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 -- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0733c0 msgr2=0x7f22ec15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0733c0 0x7f22ec15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 -- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 msgr2=0x7f22ec13a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 0x7f22ec13a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 -- 172.21.15.83:0/3216447211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0b0230 con 0x7f22ec0b3d00 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22eb59e640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 0x7f22ec13a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22ea59c640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22e001ab50 tx=0x7f22e0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00b2040 con 0x7f22ec0b3d00 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00be070 con 0x7f22ec0b3d00 2024-05-17T21:58:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec0b0510 con 0x7f22ec0b3d00 2024-05-17T21:58:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00b8660 con 0x7f22ec0b3d00 2024-05-17T21:58:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.502+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec140480 con 0x7f22ec0b3d00 2024-05-17T21:58:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec0d0a10 con 0x7f22ec0b3d00 2024-05-17T21:58:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22e00b8800 con 0x7f22ec0b3d00 2024-05-17T21:58:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22d37fe640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 0x7f22d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22ead9d640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 0x7f22d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f22e0129970 con 0x7f22ec0b3d00 2024-05-17T21:58:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.506+0000 7f22ead9d640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 0x7f22d8071910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00b930 tx=0x7f22dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:27.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.510+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f22e0016090 con 0x7f22ec0b3d00 2024-05-17T21:58:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.682+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f22ec0b10a0 con 0x7f22ec0b3d00 2024-05-17T21:58:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.682+0000 7f22d37fe640 1 -- 172.21.15.83:0/3216447211 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) ==== 80+0+0 (secure 0 0 0) 0x7f22e00f6870 con 0x7f22ec0b3d00 2024-05-17T21:58:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 msgr2=0x7f22d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 0x7f22d8071910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00b930 tx=0x7f22dc008040 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 msgr2=0x7f22ec139f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f22e001ab50 tx=0x7f22e0019240 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 shutdown_connections 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f22d806f400 0x7f22d8071910 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22ec143f20 0x7f22ec13a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ec0b3d00 0x7f22ec139f70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 --2- 172.21.15.83:0/3216447211 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22ec0733c0 0x7f22ec15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 >> 172.21.15.83:0/3216447211 conn(0x7f22ec06cd90 msgr2=0x7f22ec0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 shutdown_connections 2024-05-17T21:58:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:27.686+0000 7f22f11ae640 1 -- 172.21.15.83:0/3216447211 wait complete. 2024-05-17T21:58:27.700 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T21:58:27.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:27.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T21:58:27.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[36634]: fuse finished with error 0 and tester_r 0 2024-05-17T21:58:27.711 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:29.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:58:29.654 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:58:33.813 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:58:33.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:58:33.813 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T21:58:33.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.030+0000 7f064e3e4640 1 Processor -- start 2024-05-17T21:58:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.030+0000 7f064e3e4640 1 -- start start 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064e3e4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0648144080 0x7f0648072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064e3e4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0648073370 0x7f064806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064e3e4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0648073780 con 0x7f06480d1040 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064e3e4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f064815e790 con 0x7f0648073370 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064e3e4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f064815ea40 con 0x7f0648144080 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0648073370 0x7f064806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0648073370 0x7f064806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60242/0 (socket says 172.21.15.83:60242) 2024-05-17T21:58:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 -- 172.21.15.83:0/1712211012 learned_addr learned my addr 172.21.15.83:0/1712211012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f06477fe640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0648144080 0x7f0648072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 -- 172.21.15.83:0/1712211012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0648144080 msgr2=0x7f0648072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0648144080 0x7f0648072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 -- 172.21.15.83:0/1712211012 >> v1:172.21.15.155:6789/0 conn(0x7f06480d1040 legacy=0x7f064815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.034+0000 7f064c981640 1 -- 172.21.15.83:0/1712211012 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064815ed20 con 0x7f0648073370 2024-05-17T21:58:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064c981640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f064806f730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0640009e50 tx=0x7f064001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ffcd68d8113444 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f0646ffd640 1 -- 172.21.15.83:0/1712211012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400a5660 con 0x7f0648073370 2024-05-17T21:58:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f0646ffd640 1 -- 172.21.15.83:0/1712211012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06400a5800 con 0x7f0648073370 2024-05-17T21:58:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f0646ffd640 1 -- 172.21.15.83:0/1712211012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400a5bd0 con 0x7f0648073370 2024-05-17T21:58:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- 172.21.15.83:0/1712211012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 msgr2=0x7f064806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f064806f730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0640009e50 tx=0x7f064001c820 comp rx=0 tx=0).stop 2024-05-17T21:58:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- 172.21.15.83:0/1712211012 shutdown_connections 2024-05-17T21:58:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f064806f730 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 --2- 172.21.15.83:0/1712211012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0648144080 0x7f0648072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- 172.21.15.83:0/1712211012 >> 172.21.15.83:0/1712211012 conn(0x7f064806cd90 msgr2=0x7f06480b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- 172.21.15.83:0/1712211012 shutdown_connections 2024-05-17T21:58:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- 172.21.15.83:0/1712211012 wait complete. 2024-05-17T21:58:34.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 Processor -- start 2024-05-17T21:58:34.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.038+0000 7f064e3e4640 1 -- start start 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06480d1040 0x7f0648164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 0x7f0648165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0648070480 con 0x7f0648073370 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0648144490 con 0x7f0648144080 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0648070600 con 0x7f06480d1040 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60256/0 (socket says 172.21.15.83:60256) 2024-05-17T21:58:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 -- 172.21.15.83:0/3011335047 learned_addr learned my addr 172.21.15.83:0/3011335047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f06477fe640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06480d1040 0x7f0648164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064c981640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 0x7f0648165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 -- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06480d1040 msgr2=0x7f0648164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06480d1040 0x7f0648164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 -- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 msgr2=0x7f0648165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 0x7f0648165320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 -- 172.21.15.83:0/3011335047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0648164280 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064c981640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 0x7f0648165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0647fff640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f063800b410 tx=0x7f063800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638015030 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0638004020 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06481634c0 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0638004450 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0648163a80 con 0x7f0648073370 2024-05-17T21:58:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0648069c10 con 0x7f0648073370 2024-05-17T21:58:34.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0638018070 con 0x7f0648073370 2024-05-17T21:58:34.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 0x7f0634071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f063808f500 con 0x7f0648073370 2024-05-17T21:58:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.042+0000 7f06477fe640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 0x7f0634071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.046+0000 7f06477fe640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 0x7f0634071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06481fa180 tx=0x7f063c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.046+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f063805c5f0 con 0x7f0648073370 2024-05-17T21:58:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.246+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06480709e0 con 0x7f0648073370 2024-05-17T21:58:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.246+0000 7f0644ff9640 1 -- 172.21.15.83:0/3011335047 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f063805bef0 con 0x7f0648073370 2024-05-17T21:58:34.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:34.239 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 msgr2=0x7f0634071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 0x7f0634071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f06481fa180 tx=0x7f063c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 msgr2=0x7f06480b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f063800b410 tx=0x7f063800b8d0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 shutdown_connections 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063406f400 0x7f0634071910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0648144080 0x7f0648165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06480d1040 0x7f0648164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 --2- 172.21.15.83:0/3011335047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648073370 0x7f06480b0260 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 >> 172.21.15.83:0/3011335047 conn(0x7f064806cd90 msgr2=0x7f06480b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 shutdown_connections 2024-05-17T21:58:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.250+0000 7f064e3e4640 1 -- 172.21.15.83:0/3011335047 wait complete. 2024-05-17T21:58:34.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T21:58:34.444 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T21:58:34.444 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T21:58:34.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:34.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 Processor -- start 2024-05-17T21:58:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 -- start start 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64fc144080 0x7f64fc072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64fc06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc073780 con 0x7f64fc0d1040 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc15e790 con 0x7f64fc144080 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6502e9b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc15ea40 con 0x7f64fc073370 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6500c37640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f64fc0d1040 0x7f64fc15e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53784/0 (socket says 172.21.15.83:53784) 2024-05-17T21:58:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6500c37640 1 -- 172.21.15.83:0/771729593 learned_addr learned my addr 172.21.15.83:0/771729593 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fbfff640 1 --2- 172.21.15.83:0/771729593 >> v2:172.21.15.155:3300/0 conn(0x7f64fc144080 0x7f64fc072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f6501438640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64fc06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903935932 0 0) 0x7f64f4001120 con 0x7f64fc0d1040 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e4003530 con 0x7f64fc0d1040 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3916375480 0 0) 0x7f64e4003530 con 0x7f64fc0d1040 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 msgr2=0x7f64fc06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64fc06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 >> v2:172.21.15.155:3300/0 conn(0x7f64fc144080 msgr2=0x7f64fc072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 --2- 172.21.15.83:0/771729593 >> v2:172.21.15.155:3300/0 conn(0x7f64fc144080 0x7f64fc072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.598+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc15ed20 con 0x7f64fc0d1040 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f64f40018d0 con 0x7f64fc0d1040 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 >> v1:172.21.15.83:6789/0 conn(0x7f64fc0d1040 legacy=0x7f64fc15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64e4003a10 0x7f64e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4004d50 0x7f64e4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc15e790 con 0x7f64e4003a10 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc15ea40 con 0x7f64e4004d50 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64e400ba80 con 0x7f64fc073370 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64e4003a10 0x7f64e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fbfff640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6501438640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4004d50 0x7f64e4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 -- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 msgr2=0x7f64e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc073370 0x7f64e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 -- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4004d50 msgr2=0x7f64e4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4004d50 0x7f64e4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 -- 172.21.15.83:0/771729593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e4003530 con 0x7f64e4003a10 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6500c37640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64e4003a10 0x7f64e4003e80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c4c0 tx=0x7f64f401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb06cbc3ea3ff5df server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400c0e0 con 0x7f64e4003a10 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f40a9070 con 0x7f64e4003a10 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f64fb7fe640 1 -- 172.21.15.83:0/771729593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40a5db0 con 0x7f64e4003a10 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 -- 172.21.15.83:0/771729593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64e4003a10 msgr2=0x7f64e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 --2- 172.21.15.83:0/771729593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64e4003a10 0x7f64e4003e80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64f400c4c0 tx=0x7f64f401cbf0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6501438640 1 -- 172.21.15.83:0/771729593 reap_dead start 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 -- 172.21.15.83:0/771729593 shutdown_connections 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 -- 172.21.15.83:0/771729593 >> 172.21.15.83:0/771729593 conn(0x7f64fc06cd90 msgr2=0x7f64fc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 -- 172.21.15.83:0/771729593 shutdown_connections 2024-05-17T21:58:34.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.602+0000 7f6502e9b640 1 -- 172.21.15.83:0/771729593 wait complete. 2024-05-17T21:58:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 Processor -- start 2024-05-17T21:58:34.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- start start 2024-05-17T21:58:34.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4003a10 0x7f64fc13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc163a40 0x7f64fc140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc06ff40 con 0x7f64fc163e50 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc144490 con 0x7f64e4003a10 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0700c0 con 0x7f64fc163a40 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60272/0 (socket says 172.21.15.83:60272) 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 -- 172.21.15.83:0/403152149 learned_addr learned my addr 172.21.15.83:0/403152149 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f64fbfff640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc163a40 0x7f64fc140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6500c37640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4003a10 0x7f64fc13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 -- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc163a40 msgr2=0x7f64fc140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc163a40 0x7f64fc140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 -- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4003a10 msgr2=0x7f64fc13d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4003a10 0x7f64fc13d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 -- 172.21.15.83:0/403152149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc13d700 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6501438640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b3ca0 tx=0x7f64ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011e40 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec01b070 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0039a0 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc159720 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.606+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc164890 con 0x7f64fc163e50 2024-05-17T21:58:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc072e30 con 0x7f64fc163e50 2024-05-17T21:58:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64ec003b40 con 0x7f64fc163e50 2024-05-17T21:58:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f64f97fa640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f64ec08fa40 con 0x7f64fc163e50 2024-05-17T21:58:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f6500c37640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.610+0000 7f6500c37640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 0x7f64e8071910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64f40ab000 tx=0x7f64f40b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.614+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f64ec05c930 con 0x7f64fc163e50 2024-05-17T21:58:34.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64fc164340 con 0x7f64fc163e50 2024-05-17T21:58:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f64f97fa640 1 -- 172.21.15.83:0/403152149 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f64ec08fe20 con 0x7f64fc163e50 2024-05-17T21:58:34.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:34.790 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:34.790 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:34.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 msgr2=0x7f64e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 0x7f64e8071910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f64f40ab000 tx=0x7f64f40b1040 comp rx=0 tx=0).stop 2024-05-17T21:58:34.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 msgr2=0x7f64fc1405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0b3ca0 tx=0x7f64ec0115a0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 shutdown_connections 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64fc163e50 0x7f64fc1405a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64fc163a40 0x7f64fc140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f64e806f400 0x7f64e8071910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 --2- 172.21.15.83:0/403152149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64e4003a10 0x7f64fc13d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 >> 172.21.15.83:0/403152149 conn(0x7f64fc06cd90 msgr2=0x7f64fc0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 shutdown_connections 2024-05-17T21:58:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:34.798+0000 7f6502e9b640 1 -- 172.21.15.83:0/403152149 wait complete. 2024-05-17T21:58:34.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-05-17T21:58:34.981 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-17T21:58:34.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-17T21:58:44.982 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-17T21:58:45.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 Processor -- start 2024-05-17T21:58:45.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 -- start start 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecade7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eeca083b0 0x556eeca087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eec9b1ef0 con 0x556eecac3df0 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556eec9be310 con 0x556eeca083b0 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.061+0000 7f9705536080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556eec9561b0 con 0x7f96ec003ba0 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49970/0 (socket says 172.21.15.83:49970) 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 -- 172.21.15.83:0/695266240 learned_addr learned my addr 172.21.15.83:0/695266240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96fe817640 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eeca083b0 0x556eeca087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff018640 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecade7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 -- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 msgr2=0x556eecade7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecade7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 -- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eeca083b0 msgr2=0x556eeca087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eeca083b0 0x556eeca087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 -- 172.21.15.83:0/695266240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eecadca40 con 0x556eecac3df0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff819640 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96f801b490 tx=0x7f96f8001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0a4297db0d738a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96fd815640 1 -- 172.21.15.83:0/695266240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80ab5f0 con 0x556eecac3df0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96fd815640 1 -- 172.21.15.83:0/695266240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f80ab790 con 0x556eecac3df0 2024-05-17T21:58:45.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96fd815640 1 -- 172.21.15.83:0/695266240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80abb40 con 0x556eecac3df0 2024-05-17T21:58:45.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- 172.21.15.83:0/695266240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 msgr2=0x556eecac4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96f801b490 tx=0x7f96f8001130 comp rx=0 tx=0).stop 2024-05-17T21:58:45.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- 172.21.15.83:0/695266240 shutdown_connections 2024-05-17T21:58:45.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eecac3df0 0x556eecac4200 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eeca083b0 0x556eeca087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- 172.21.15.83:0/695266240 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecade7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- 172.21.15.83:0/695266240 >> 172.21.15.83:0/695266240 conn(0x556eec9f2e40 msgr2=0x556eeca379e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- 172.21.15.83:0/695266240 shutdown_connections 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- 172.21.15.83:0/695266240 wait complete. 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 Processor -- start 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- start start 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 0x556eecac2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eecac3df0 0x556eecac5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecadb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eecadecf0 con 0x556eeca083b0 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556eecadcce0 con 0x556eecac3df0 2024-05-17T21:58:45.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f9705536080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556eecac6790 con 0x7f96ec003ba0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff018640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 0x556eecac2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff018640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 0x556eecac2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49984/0 (socket says 172.21.15.83:49984) 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.065+0000 7f96ff018640 1 -- 172.21.15.83:0/3668019737 learned_addr learned my addr 172.21.15.83:0/3668019737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff819640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecadb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fe817640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eecac3df0 0x556eecac5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 -- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 msgr2=0x556eecadb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96ec003ba0 0x556eecadb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 -- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eecac3df0 msgr2=0x556eecac5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eecac3df0 0x556eecac5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 -- 172.21.15.83:0/3668019737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eecac1850 con 0x556eeca083b0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fe817640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556eecac3df0 0x556eecac5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff018640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 0x556eecac2770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f96f000fed0 tx=0x7f96f000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 -- 172.21.15.83:0/3668019737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000b7a0 con 0x556eeca083b0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 -- 172.21.15.83:0/3668019737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f000b940 con 0x556eeca083b0 2024-05-17T21:58:45.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556eecac1af0 con 0x556eeca083b0 2024-05-17T21:58:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 -- 172.21.15.83:0/3668019737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000bcf0 con 0x556eeca083b0 2024-05-17T21:58:45.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eecac2030 con 0x556eeca083b0 2024-05-17T21:58:45.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 -- 172.21.15.83:0/3668019737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96f0009370 con 0x556eeca083b0 2024-05-17T21:58:45.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96e006dde0 0x7f96e00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fe817640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96e006dde0 0x7f96e00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fc813640 1 -- 172.21.15.83:0/3668019737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f96f008e2c0 con 0x556eeca083b0 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f9705536080 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556eecac6120 0x556eeca2d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556eecac2cb0 con 0x556eecac6120 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96ff819640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556eecac6120 0x556eeca2d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.069+0000 7f96fe817640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96e006dde0 0x7f96e00702f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96e8010720 tx=0x7f96e8007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f96ff819640 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556eecac6120 0x556eeca2d5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f96ff819640 1 -- 172.21.15.83:0/3668019737 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f96f80ab940 con 0x556eecac6120 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556eecac6120 msgr2=0x556eeca2d5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556eecac6120 0x556eeca2d5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96e006dde0 msgr2=0x7f96e00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96e006dde0 0x7f96e00702f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96e8010720 tx=0x7f96e8007040 comp rx=0 tx=0).stop 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 msgr2=0x556eecac2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 --2- 172.21.15.83:0/3668019737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eeca083b0 0x556eecac2770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f96f000fed0 tx=0x7f96f000d750 comp rx=0 tx=0).stop 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f96ff819640 1 -- 172.21.15.83:0/3668019737 reap_dead start 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 shutdown_connections 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 >> 172.21.15.83:0/3668019737 conn(0x556eec9f2e40 msgr2=0x556eecac4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 shutdown_connections 2024-05-17T21:58:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.073+0000 7f9705536080 1 -- 172.21.15.83:0/3668019737 wait complete. 2024-05-17T21:58:45.071 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2024-05-17T21:58:45.071 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-17T21:58:45.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 Processor -- start 2024-05-17T21:58:45.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- start start 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c639a01710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6399081b0 con 0x55c639a8dd00 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c639963ef0 con 0x55c6399bc140 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c639a8e320 con 0x55c6399bc6a0 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50004/0 (socket says 172.21.15.83:50004) 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 -- 172.21.15.83:0/1080747494 learned_addr learned my addr 172.21.15.83:0/1080747494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 -- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 msgr2=0x55c639a01710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebe3c3640 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c639a01710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c639a01710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebebc4640 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 -- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 msgr2=0x55c6399e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 -- 172.21.15.83:0/1080747494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6398e2c10 con 0x55c639a8dd00 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebf3c5640 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097d0 tx=0x7efeb0005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188a6adeecb35c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebd3c1640 1 -- 172.21.15.83:0/1080747494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0019070 con 0x55c639a8dd00 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebd3c1640 1 -- 172.21.15.83:0/1080747494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb0008d40 con 0x55c639a8dd00 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efebd3c1640 1 -- 172.21.15.83:0/1080747494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb00146c0 con 0x55c639a8dd00 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- 172.21.15.83:0/1080747494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 msgr2=0x55c639b16ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097d0 tx=0x7efeb0005a10 comp rx=0 tx=0).stop 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- 172.21.15.83:0/1080747494 shutdown_connections 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639b16ac0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c639a01710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 --2- 172.21.15.83:0/1080747494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- 172.21.15.83:0/1080747494 >> 172.21.15.83:0/1080747494 conn(0x55c6399a4e40 msgr2=0x55c639a02180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- 172.21.15.83:0/1080747494 shutdown_connections 2024-05-17T21:58:45.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.141+0000 7efec50e2080 1 -- 172.21.15.83:0/1080747494 wait complete. 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 Processor -- start 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- start start 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c6399b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639a8d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6399e6a60 con 0x55c639a8dd00 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c639a70850 con 0x55c6399bc140 2024-05-17T21:58:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c6399e6d00 con 0x55c6399bc6a0 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639a8d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639a8d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50016/0 (socket says 172.21.15.83:50016) 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 -- 172.21.15.83:0/3992798145 learned_addr learned my addr 172.21.15.83:0/3992798145 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebebc4640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebe3c3640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c6399b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 -- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 msgr2=0x55c6399b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c6399bc6a0 0x55c6399b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 -- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 msgr2=0x55c6399fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6399bc140 0x55c6399fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 -- 172.21.15.83:0/3992798145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6399ff480 con 0x55c639a8dd00 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efebf3c5640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639a8d4f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efeb00149e0 tx=0x7efeb0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efea7fff640 1 -- 172.21.15.83:0/3992798145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0023020 con 0x55c639a8dd00 2024-05-17T21:58:45.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efea7fff640 1 -- 172.21.15.83:0/3992798145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb001a040 con 0x55c639a8dd00 2024-05-17T21:58:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6399ff720 con 0x55c639a8dd00 2024-05-17T21:58:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efea7fff640 1 -- 172.21.15.83:0/3992798145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0019070 con 0x55c639a8dd00 2024-05-17T21:58:45.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.145+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6399fbd70 con 0x55c639a8dd00 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efea7fff640 1 -- 172.21.15.83:0/3992798145 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efeb0026020 con 0x55c639a8dd00 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efea7fff640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efeac06dde0 0x7efeac0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efea7fff640 1 -- 172.21.15.83:0/3992798145 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7efeb008fab0 con 0x55c639a8dd00 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebebc4640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efeac06dde0 0x7efeac0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55c6399fc960 0x55c6399bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c639b23520 con 0x55c6399fc960 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebe3c3640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55c6399fc960 0x55c6399bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebebc4640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efeac06dde0 0x7efeac0702f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efeb8001230 tx=0x7efeb80014c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebe3c3640 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55c6399fc960 0x55c6399bd180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebe3c3640 1 -- 172.21.15.83:0/3992798145 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7efeb40aa9b0 con 0x55c6399fc960 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55c6399fc960 msgr2=0x55c6399bd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55c6399fc960 0x55c6399bd180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efeac06dde0 msgr2=0x7efeac0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efeac06dde0 0x7efeac0702f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efeb8001230 tx=0x7efeb80014c0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 msgr2=0x55c639a8d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 --2- 172.21.15.83:0/3992798145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c639a8dd00 0x55c639a8d4f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efeb00149e0 tx=0x7efeb0003030 comp rx=0 tx=0).stop 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efebf3c5640 1 -- 172.21.15.83:0/3992798145 reap_dead start 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 shutdown_connections 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 >> 172.21.15.83:0/3992798145 conn(0x55c6399a4e40 msgr2=0x55c639a01710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 shutdown_connections 2024-05-17T21:58:45.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.149+0000 7efec50e2080 1 -- 172.21.15.83:0/3992798145 wait complete. 2024-05-17T21:58:45.149 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2024-05-17T21:58:45.149 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-17T21:58:45.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 Processor -- start 2024-05-17T21:58:45.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- start start 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dcec100 0x55907dceabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dd60460 0x55907dd763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55907dc4cef0 con 0x55907dd77df0 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55907dc59310 con 0x55907dcec100 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55907dbf11b0 con 0x55907dd60460 2024-05-17T21:58:45.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50034/0 (socket says 172.21.15.83:50034) 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/1700017059 learned_addr learned my addr 172.21.15.83:0/1700017059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bec8c640 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dd60460 0x55907dd763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dd60460 msgr2=0x55907dd763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dd60460 0x55907dd763d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dcec100 msgr2=0x55907dceabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bf48d640 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dcec100 0x55907dceabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dcec100 0x55907dceabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/1700017059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907dcd2680 con 0x55907dd77df0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b2040 tx=0x7fd7b8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c944f7a378ab9eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bdc8a640 1 -- 172.21.15.83:0/1700017059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80c0070 con 0x55907dd77df0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bdc8a640 1 -- 172.21.15.83:0/1700017059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b80b5040 con 0x55907dd77df0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7bdc8a640 1 -- 172.21.15.83:0/1700017059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80b4050 con 0x55907dd77df0 2024-05-17T21:58:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- 172.21.15.83:0/1700017059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 msgr2=0x55907dd79210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b2040 tx=0x7fd7b8003030 comp rx=0 tx=0).stop 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- 172.21.15.83:0/1700017059 shutdown_connections 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd79210 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dd60460 0x55907dd763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/1700017059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dcec100 0x55907dceabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- 172.21.15.83:0/1700017059 >> 172.21.15.83:0/1700017059 conn(0x55907dc8de40 msgr2=0x55907dd5cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- 172.21.15.83:0/1700017059 shutdown_connections 2024-05-17T21:58:45.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.217+0000 7fd7c59ab080 1 -- 172.21.15.83:0/1700017059 wait complete. 2024-05-17T21:58:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 Processor -- start 2024-05-17T21:58:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- start start 2024-05-17T21:58:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dcec100 0x55907dd5e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dd60460 0x55907dd5c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd74a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55907dcea470 con 0x55907dd77df0 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55907dcea710 con 0x55907dd60460 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55907dcd2920 con 0x55907dcec100 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd74a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd74a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50040/0 (socket says 172.21.15.83:50040) 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/721408157 learned_addr learned my addr 172.21.15.83:0/721408157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bf48d640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dcec100 0x55907dd5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bec8c640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dd60460 0x55907dd5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dcec100 msgr2=0x55907dd5e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55907dcec100 0x55907dd5e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dd60460 msgr2=0x55907dd5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dd60460 0x55907dd5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/721408157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907dd5ed20 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bec8c640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55907dd60460 0x55907dd5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bfc8e640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd74a60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800a450 tx=0x7fd7b801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bcc88640 1 -- 172.21.15.83:0/721408157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80c9070 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bcc88640 1 -- 172.21.15.83:0/721408157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b80bb710 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55907dd5efc0 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7bcc88640 1 -- 172.21.15.83:0/721408157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80b4050 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.221+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55907dd5cf10 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bcc88640 1 -- 172.21.15.83:0/721408157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7b80cc070 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bcc88640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7a406de10 0x7fd7a4070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bcc88640 1 -- 172.21.15.83:0/721408157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b812cb80 con 0x55907dd77df0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bf48d640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7a406de10 0x7fd7a4070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55907dcd0a60 0x55907dd5d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55907dd5d8f0 con 0x55907dcd0a60 2024-05-17T21:58:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bec8c640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55907dcd0a60 0x55907dd5d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bf48d640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7a406de10 0x7fd7a4070320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000e3a0 tx=0x7fd7b00035a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bec8c640 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55907dcd0a60 0x55907dd5d3b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bec8c640 1 -- 172.21.15.83:0/721408157 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fd7a80017c0 con 0x55907dcd0a60 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55907dcd0a60 msgr2=0x55907dd5d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55907dcd0a60 0x55907dd5d3b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7a406de10 msgr2=0x7fd7a4070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd7a406de10 0x7fd7a4070320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000e3a0 tx=0x7fd7b00035a0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 msgr2=0x55907dd74a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 --2- 172.21.15.83:0/721408157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55907dd77df0 0x55907dd74a60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800a450 tx=0x7fd7b801c3e0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7bfc8e640 1 -- 172.21.15.83:0/721408157 reap_dead start 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 shutdown_connections 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 >> 172.21.15.83:0/721408157 conn(0x55907dc8de40 msgr2=0x55907dca3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 shutdown_connections 2024-05-17T21:58:45.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.225+0000 7fd7c59ab080 1 -- 172.21.15.83:0/721408157 wait complete. 2024-05-17T21:58:45.225 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2024-05-17T21:58:45.225 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-17T21:58:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 Processor -- start 2024-05-17T21:58:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 -- start start 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 0x556ca6ea0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6f2e020 0x556ca6e97e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ca6dffef0 con 0x556ca6ea0f70 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ca6e0c310 con 0x556ca6f2e020 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.293+0000 7f6d4392c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556ca6da41b0 con 0x7f6d28002330 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50054/0 (socket says 172.21.15.83:50054) 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 -- 172.21.15.83:0/919126004 learned_addr learned my addr 172.21.15.83:0/919126004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3d40e640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 0x556ca6ea0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3dc0f640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6f2e020 0x556ca6e97e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 -- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 msgr2=0x556ca6ea0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 0x556ca6ea0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 -- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6f2e020 msgr2=0x556ca6e97e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6f2e020 0x556ca6e97e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 -- 172.21.15.83:0/919126004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca6e88720 con 0x556ca6ea0f70 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3d40e640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 0x556ca6ea0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400d490 tx=0x7f6d3400d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ae8a5ec14f624 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d2f7fe640 1 -- 172.21.15.83:0/919126004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400a040 con 0x556ca6ea0f70 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d2f7fe640 1 -- 172.21.15.83:0/919126004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34003310 con 0x556ca6ea0f70 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d2f7fe640 1 -- 172.21.15.83:0/919126004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34001870 con 0x556ca6ea0f70 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- 172.21.15.83:0/919126004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 msgr2=0x556ca6e97950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400d490 tx=0x7f6d3400d950 comp rx=0 tx=0).stop 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- 172.21.15.83:0/919126004 shutdown_connections 2024-05-17T21:58:45.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6f2e020 0x556ca6e97e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ca6ea0f70 0x556ca6e97950 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- 172.21.15.83:0/919126004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d28002330 0x556ca6ea0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- 172.21.15.83:0/919126004 >> 172.21.15.83:0/919126004 conn(0x556ca6e40e40 msgr2=0x556ca6e859e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- 172.21.15.83:0/919126004 shutdown_connections 2024-05-17T21:58:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- 172.21.15.83:0/919126004 wait complete. 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 Processor -- start 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- start start 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6ea0f70 0x556ca6ea1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556ca6f2e020 0x556ca6f29280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 0x556ca6f297c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ca6e87390 con 0x7f6d28002330 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556ca6e870f0 con 0x556ca6ea0f70 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.297+0000 7f6d4392c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556ca6f2e9e0 con 0x556ca6f2e020 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 0x556ca6f297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 0x556ca6f297c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50070/0 (socket says 172.21.15.83:50070) 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 -- 172.21.15.83:0/1743281893 learned_addr learned my addr 172.21.15.83:0/1743281893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556ca6f2e020 0x556ca6f29280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3d40e640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6ea0f70 0x556ca6ea1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 -- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556ca6f2e020 msgr2=0x556ca6f29280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556ca6f2e020 0x556ca6f29280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 -- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6ea0f70 msgr2=0x556ca6ea1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556ca6ea0f70 0x556ca6ea1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 -- 172.21.15.83:0/1743281893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca6fc2fe0 con 0x7f6d28002330 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3dc0f640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 0x556ca6f297c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x556ca6e3f8c0 tx=0x7f6d3800e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 -- 172.21.15.83:0/1743281893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38018020 con 0x7f6d28002330 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 -- 172.21.15.83:0/1743281893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d38014820 con 0x7f6d28002330 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ca6fc2360 con 0x7f6d28002330 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 -- 172.21.15.83:0/1743281893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3800f040 con 0x7f6d28002330 2024-05-17T21:58:45.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca6fc28a0 con 0x7f6d28002330 2024-05-17T21:58:45.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 -- 172.21.15.83:0/1743281893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d3801b020 con 0x7f6d28002330 2024-05-17T21:58:45.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d2006de10 0x7f6d20070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3d40e640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d2006de10 0x7f6d20070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d2e7fc640 1 -- 172.21.15.83:0/1743281893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f6d38090cd0 con 0x7f6d28002330 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d4392c080 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x556ca6ea1fb0 0x556ca6e80f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556ca6f17e20 con 0x556ca6ea1fb0 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.301+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x556ca6ea1fb0 0x556ca6e80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d3d40e640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d2006de10 0x7f6d20070320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d300ae000 tx=0x7f6d30003030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d3cc0d640 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x556ca6ea1fb0 0x556ca6e80f80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d3cc0d640 1 -- 172.21.15.83:0/1743281893 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f6d340014f0 con 0x556ca6ea1fb0 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x556ca6ea1fb0 msgr2=0x556ca6e80f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x556ca6ea1fb0 0x556ca6e80f80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d2006de10 msgr2=0x7f6d20070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d2006de10 0x7f6d20070320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d300ae000 tx=0x7f6d30003030 comp rx=0 tx=0).stop 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 msgr2=0x556ca6f297c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 --2- 172.21.15.83:0/1743281893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d28002330 0x556ca6f297c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x556ca6e3f8c0 tx=0x7f6d3800e680 comp rx=0 tx=0).stop 2024-05-17T21:58:45.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d3dc0f640 1 -- 172.21.15.83:0/1743281893 reap_dead start 2024-05-17T21:58:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 shutdown_connections 2024-05-17T21:58:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 >> 172.21.15.83:0/1743281893 conn(0x556ca6e40e40 msgr2=0x556ca6e865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 shutdown_connections 2024-05-17T21:58:45.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.305+0000 7f6d4392c080 1 -- 172.21.15.83:0/1743281893 wait complete. 2024-05-17T21:58:45.303 INFO:teuthology.orchestra.run.smithi083.stdout:mytag 2024-05-17T21:58:45.303 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-17T21:58:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.369+0000 7fb1b3ece080 1 Processor -- start 2024-05-17T21:58:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.369+0000 7fb1b3ece080 1 -- start start 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x5561556d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556155749c60 0x5561556d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561555dc1b0 con 0x55615568fc00 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556155637ef0 con 0x556155749c60 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556155761f50 con 0x5561556906a0 2024-05-17T21:58:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50076/0 (socket says 172.21.15.83:50076) 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 -- 172.21.15.83:0/1480218238 learned_addr learned my addr 172.21.15.83:0/1480218238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 -- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 msgr2=0x5561556d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad1af640 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x5561556d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x5561556d6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 -- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556155749c60 msgr2=0x5561556d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556155749c60 0x5561556d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 -- 172.21.15.83:0/1480218238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561555b6c10 con 0x55615568fc00 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b2040 tx=0x7fb1a8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2400ed51e1b5f649 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1a7fff640 1 -- 172.21.15.83:0/1480218238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80be070 con 0x55615568fc00 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1a7fff640 1 -- 172.21.15.83:0/1480218238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a80b5040 con 0x55615568fc00 2024-05-17T21:58:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1a7fff640 1 -- 172.21.15.83:0/1480218238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80b4070 con 0x55615568fc00 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- 172.21.15.83:0/1480218238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 msgr2=0x5561556d5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b2040 tx=0x7fb1a8003030 comp rx=0 tx=0).stop 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- 172.21.15.83:0/1480218238 shutdown_connections 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556155749c60 0x5561556d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x5561556d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/1480218238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55615568fc00 0x5561556d5fd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- 172.21.15.83:0/1480218238 >> 172.21.15.83:0/1480218238 conn(0x556155678e40 msgr2=0x556155748d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- 172.21.15.83:0/1480218238 shutdown_connections 2024-05-17T21:58:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- 172.21.15.83:0/1480218238 wait complete. 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 Processor -- start 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.373+0000 7fb1b3ece080 1 -- start start 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55615568fc00 0x556155760c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x55615572ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 0x556155760550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561557e9cc0 con 0x556155749c60 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5561557e9f60 con 0x55615568fc00 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5561557ea4f0 con 0x5561556906a0 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 0x556155760550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 0x556155760550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50078/0 (socket says 172.21.15.83:50078) 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 -- 172.21.15.83:0/759601733 learned_addr learned my addr 172.21.15.83:0/759601733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 -- 172.21.15.83:0/759601733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 msgr2=0x55615572ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5561556906a0 0x55615572ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 -- 172.21.15.83:0/759601733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55615568fc00 msgr2=0x556155760c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55615568fc00 0x556155760c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55615568fc00 0x556155760c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 -- 172.21.15.83:0/759601733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556155730630 con 0x556155749c60 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1ae1b1640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 0x556155760550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006e10 tx=0x7fb1a000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1a6ffd640 1 -- 172.21.15.83:0/759601733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0016070 con 0x556155749c60 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1a6ffd640 1 -- 172.21.15.83:0/759601733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a00062d0 con 0x556155749c60 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5561557308d0 con 0x556155749c60 2024-05-17T21:58:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1a6ffd640 1 -- 172.21.15.83:0/759601733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0004710 con 0x556155749c60 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556155730e10 con 0x556155749c60 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1a6ffd640 1 -- 172.21.15.83:0/759601733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1a0019070 con 0x556155749c60 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.377+0000 7fb1a6ffd640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb19006de10 0x7fb190070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1a6ffd640 1 -- 172.21.15.83:0/759601733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb1a008d060 con 0x556155749c60 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb19006de10 0x7fb190070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5561557f77e0 0x5561557f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5561557f8130 con 0x5561557f77e0 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ad1af640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5561557f77e0 0x5561557f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ad9b0640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb19006de10 0x7fb190070320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80bb000 tx=0x7fb1a80a9000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ad1af640 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5561557f77e0 0x5561557f7bf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ad1af640 1 -- 172.21.15.83:0/759601733 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fb19800b6e0 con 0x5561557f77e0 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5561557f77e0 msgr2=0x5561557f7bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5561557f77e0 0x5561557f7bf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb19006de10 msgr2=0x7fb190070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb19006de10 0x7fb190070320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80bb000 tx=0x7fb1a80a9000 comp rx=0 tx=0).stop 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 msgr2=0x556155760550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 --2- 172.21.15.83:0/759601733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556155749c60 0x556155760550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0006e10 tx=0x7fb1a000b4b0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1ae1b1640 1 -- 172.21.15.83:0/759601733 reap_dead start 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 shutdown_connections 2024-05-17T21:58:45.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 >> 172.21.15.83:0/759601733 conn(0x556155678e40 msgr2=0x55615568d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 shutdown_connections 2024-05-17T21:58:45.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.381+0000 7fb1b3ece080 1 -- 172.21.15.83:0/759601733 wait complete. 2024-05-17T21:58:45.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T21:58:45.380 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T21:58:45.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:45.426 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:45.467 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T21:58:45.467 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:58:45.468 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:45.468 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:45.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:58:45.473 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:58:45.473 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:45.473 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:45.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:58:45.515 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:58:45.515 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:45.515 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:45.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:58:45.521 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:58:45.521 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:45.521 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:45.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:58:45.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 Processor -- start 2024-05-17T21:58:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 -- start start 2024-05-17T21:58:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcb900d1040 0x7fcb90144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb9015e080 0x7fcb90072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9015ea40 con 0x7fcb9015e490 2024-05-17T21:58:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9015ed20 con 0x7fcb9015e080 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9798e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9015f000 con 0x7fcb900d1040 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb94f29640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb9015e080 0x7fcb90072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb95f2b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb9015e490 0x7fcb901616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39684/0 (socket says 172.21.15.83:39684) 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb95f2b640 1 -- 172.21.15.83:0/2677889022 learned_addr learned my addr 172.21.15.83:0/2677889022 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.729+0000 7fcb9572a640 1 --2- 172.21.15.83:0/2677889022 >> v2:172.21.15.155:3300/0 conn(0x7fcb900d1040 0x7fcb90144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb94f29640 1 -- 172.21.15.83:0/2677889022 >> v2:172.21.15.155:3300/0 conn(0x7fcb900d1040 msgr2=0x7fcb90144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb94f29640 1 --2- 172.21.15.83:0/2677889022 >> v2:172.21.15.155:3300/0 conn(0x7fcb900d1040 0x7fcb90144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb94f29640 1 -- 172.21.15.83:0/2677889022 >> v1:172.21.15.83:6789/0 conn(0x7fcb9015e490 legacy=0x7fcb901616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb94f29640 1 -- 172.21.15.83:0/2677889022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9006ef30 con 0x7fcb9015e080 2024-05-17T21:58:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb87fff640 1 -- 172.21.15.83:0/2677889022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579492871 0 0) 0x7fcb9015ea40 con 0x7fcb9015e490 2024-05-17T21:58:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb94f29640 1 --2- 172.21.15.83:0/2677889022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb9015e080 0x7fcb90072e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb90071d60 tx=0x7fcb8c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8794b730cabd1c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb87fff640 1 -- 172.21.15.83:0/2677889022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0b1070 con 0x7fcb9015e080 2024-05-17T21:58:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb87fff640 1 -- 172.21.15.83:0/2677889022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c0ac5f0 con 0x7fcb9015e080 2024-05-17T21:58:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb87fff640 1 -- 172.21.15.83:0/2677889022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0ac9c0 con 0x7fcb9015e080 2024-05-17T21:58:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 -- 172.21.15.83:0/2677889022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb9015e080 msgr2=0x7fcb90072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 --2- 172.21.15.83:0/2677889022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb9015e080 0x7fcb90072e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb90071d60 tx=0x7fcb8c01c480 comp rx=0 tx=0).stop 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 -- 172.21.15.83:0/2677889022 shutdown_connections 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 --2- 172.21.15.83:0/2677889022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb9015e080 0x7fcb90072e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 --2- 172.21.15.83:0/2677889022 >> v2:172.21.15.155:3300/0 conn(0x7fcb900d1040 0x7fcb90144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 -- 172.21.15.83:0/2677889022 >> 172.21.15.83:0/2677889022 conn(0x7fcb9006cd90 msgr2=0x7fcb900b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 -- 172.21.15.83:0/2677889022 shutdown_connections 2024-05-17T21:58:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.733+0000 7fcb9798e640 1 -- 172.21.15.83:0/2677889022 wait complete. 2024-05-17T21:58:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 Processor -- start 2024-05-17T21:58:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- start start 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb9015e080 0x7fcb900b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb9015e490 0x7fcb9015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9016abe0 con 0x7fcb900d1040 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9016aa60 con 0x7fcb9015e080 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9016ad60 con 0x7fcb9015e490 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50104/0 (socket says 172.21.15.83:50104) 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 -- 172.21.15.83:0/4055682475 learned_addr learned my addr 172.21.15.83:0/4055682475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb94f29640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb9015e080 0x7fcb900b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb95f2b640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb9015e490 0x7fcb9015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 -- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb9015e490 msgr2=0x7fcb9015b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb9015e490 0x7fcb9015b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 -- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb9015e080 msgr2=0x7fcb900b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb9015e080 0x7fcb900b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 -- 172.21.15.83:0/4055682475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9013c110 con 0x7fcb900d1040 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9572a640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000d620 tx=0x7fcb8000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800125f0 con 0x7fcb900d1040 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb80019070 con 0x7fcb900d1040 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9013c3f0 con 0x7fcb900d1040 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800129b0 con 0x7fcb900d1040 2024-05-17T21:58:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.737+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb90140040 con 0x7fcb900d1040 2024-05-17T21:58:45.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb901402a0 con 0x7fcb900d1040 2024-05-17T21:58:45.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb80012b50 con 0x7fcb900d1040 2024-05-17T21:58:45.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb85ffb640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 0x7fcb74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:45.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fcb800900f0 con 0x7fcb900d1040 2024-05-17T21:58:45.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb94f29640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 0x7fcb74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:45.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.741+0000 7fcb94f29640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 0x7fcb74071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0ae000 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.745+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcb8005d150 con 0x7fcb900d1040 2024-05-17T21:58:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.941+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb90144080 con 0x7fcb900d1040 2024-05-17T21:58:45.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.941+0000 7fcb85ffb640 1 -- 172.21.15.83:0/4055682475 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7fcb8005ca50 con 0x7fcb900d1040 2024-05-17T21:58:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:45.936 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:45.936 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 msgr2=0x7fcb74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 0x7fcb74071910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0ae000 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 msgr2=0x7fcb9013fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000d620 tx=0x7fcb8000dae0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 shutdown_connections 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcb9015e490 0x7fcb9015b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fcb7406f400 0x7fcb74071910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb9015e080 0x7fcb900b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 --2- 172.21.15.83:0/4055682475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb900d1040 0x7fcb9013fa70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 >> 172.21.15.83:0/4055682475 conn(0x7fcb9006cd90 msgr2=0x7fcb9015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 shutdown_connections 2024-05-17T21:58:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:45.945+0000 7fcb9798e640 1 -- 172.21.15.83:0/4055682475 wait complete. 2024-05-17T21:58:45.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 Processor -- start 2024-05-17T21:58:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 -- start start 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6db80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db80d34e0 con 0x7f6db80d1150 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db80d3780 con 0x7f6db806e3d0 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dc05d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6db80d3a60 con 0x7f6db815ffa0 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbdb6d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6db80d1150 0x7f6db806dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39688/0 (socket says 172.21.15.83:39688) 2024-05-17T21:58:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbdb6d640 1 -- 172.21.15.83:0/2400664557 learned_addr learned my addr 172.21.15.83:0/2400664557 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbe36e640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6db80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793201164 0 0) 0x7f6db80d34e0 con 0x7f6db80d1150 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6da4003530 con 0x7f6db80d1150 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 650355419 0 0) 0x7f6da4003530 con 0x7f6db80d1150 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 msgr2=0x7f6db80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6db80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 >> v1:172.21.15.155:6789/0 conn(0x7f6db806e3d0 legacy=0x7f6db80d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db80d3d40 con 0x7f6db80d1150 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f6dac001700 con 0x7f6db80d1150 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 >> v1:172.21.15.83:6789/0 conn(0x7f6db80d1150 legacy=0x7f6db806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da4003a40 0x7f6da4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6da4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.101+0000 7f6dbd36c640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6da40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80d3a60 con 0x7f6da4004810 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6da400bb90 con 0x7f6da4003a40 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6da400bd90 con 0x7f6db815ffa0 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6da4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbe36e640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da4003a40 0x7f6da4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbeb6f640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6da40052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 -- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 msgr2=0x7f6da40052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6da40052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 -- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da4003a40 msgr2=0x7f6da4003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da4003a40 0x7f6da4003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 -- 172.21.15.83:0/2400664557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da4003530 con 0x7f6da4004810 2024-05-17T21:58:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbeb6f640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db815ffa0 0x7f6da40052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6da4004c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c240 tx=0x7f6dac01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf08d832bf390d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac007cf0 con 0x7f6da4004810 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac005ae0 con 0x7f6da4004810 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbd36c640 1 -- 172.21.15.83:0/2400664557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac007070 con 0x7f6da4004810 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 -- 172.21.15.83:0/2400664557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 msgr2=0x7f6da4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/2400664557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6da4004c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c240 tx=0x7f6dac01f330 comp rx=0 tx=0).stop 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dbeb6f640 1 -- 172.21.15.83:0/2400664557 reap_dead start 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 -- 172.21.15.83:0/2400664557 shutdown_connections 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 -- 172.21.15.83:0/2400664557 >> 172.21.15.83:0/2400664557 conn(0x7f6db80d28f0 msgr2=0x7f6db80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 -- 172.21.15.83:0/2400664557 shutdown_connections 2024-05-17T21:58:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.105+0000 7f6dc05d2640 1 -- 172.21.15.83:0/2400664557 wait complete. 2024-05-17T21:58:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 Processor -- start 2024-05-17T21:58:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- start start 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6db8155610 0x7f6db8157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db8155a20 0x7f6db8157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80d17e0 con 0x7f6da4004810 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6db80d1660 con 0x7f6db8155610 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6db80d0c10 con 0x7f6db8155a20 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50120/0 (socket says 172.21.15.83:50120) 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 -- 172.21.15.83:0/1685655394 learned_addr learned my addr 172.21.15.83:0/1685655394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 -- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db8155a20 msgr2=0x7f6db8157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db8155a20 0x7f6db8157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 -- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6db8155610 msgr2=0x7f6db8157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6db8155610 0x7f6db8157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 -- 172.21.15.83:0/1685655394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db81626c0 con 0x7f6da4004810 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dbe36e640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db40115e0 tx=0x7f6db4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4015870 con 0x7f6da4004810 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db4015a10 con 0x7f6da4004810 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db81629a0 con 0x7f6da4004810 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4015dc0 con 0x7f6da4004810 2024-05-17T21:58:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.109+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db8162f60 con 0x7f6da4004810 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db806dcc0 con 0x7f6da4004810 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6db400f480 con 0x7f6da4004810 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6daaffd640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f6db408fc70 con 0x7f6da4004810 2024-05-17T21:58:46.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.113+0000 7f6dbdb6d640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 0x7f6d9c071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0ae000 tx=0x7f6db80d3c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.117+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6db4019030 con 0x7f6da4004810 2024-05-17T21:58:46.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.305+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6db8158060 con 0x7f6da4004810 2024-05-17T21:58:46.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6daaffd640 1 -- 172.21.15.83:0/1685655394 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f6db405cb60 con 0x7f6da4004810 2024-05-17T21:58:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:46.302 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:46.302 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 msgr2=0x7f6d9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 0x7f6d9c071910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0ae000 tx=0x7f6db80d3c30 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 msgr2=0x7f6db815f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db40115e0 tx=0x7f6db4011aa0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 shutdown_connections 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6db8155a20 0x7f6db8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6db8155610 0x7f6db8157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 --2- 172.21.15.83:0/1685655394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da4004810 0x7f6db815f610 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 >> 172.21.15.83:0/1685655394 conn(0x7f6db80d28f0 msgr2=0x7f6db80760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 shutdown_connections 2024-05-17T21:58:46.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.309+0000 7f6dc05d2640 1 -- 172.21.15.83:0/1685655394 wait complete. 2024-05-17T21:58:46.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:46.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 Processor -- start 2024-05-17T21:58:46.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 -- start start 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65f8073540 0x7f65f8168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f815e790 con 0x7f65f80d1040 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f815ea40 con 0x7f65f8144080 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65fd70a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f65f815ed20 con 0x7f65f8073540 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f65f80d1040 0x7f65f815e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:39702/0 (socket says 172.21.15.83:39702) 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f6ffd640 1 -- 172.21.15.83:0/2426662126 learned_addr learned my addr 172.21.15.83:0/2426662126 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146859035 0 0) 0x7f65f815e790 con 0x7f65f80d1040 2024-05-17T21:58:46.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f77fe640 1 --2- 172.21.15.83:0/2426662126 >> v2:172.21.15.155:3300/0 conn(0x7f65f8073540 0x7f65f8168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65e0003530 con 0x7f65f80d1040 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494203009 0 0) 0x7f65e0003530 con 0x7f65f80d1040 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 >> v2:172.21.15.155:3300/0 conn(0x7f65f8073540 msgr2=0x7f65f8168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 --2- 172.21.15.83:0/2426662126 >> v2:172.21.15.155:3300/0 conn(0x7f65f8073540 0x7f65f8168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 >> v1:172.21.15.155:6789/0 conn(0x7f65f8144080 legacy=0x7f65f8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f815f000 con 0x7f65f80d1040 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f65e8001a80 con 0x7f65f80d1040 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 >> v1:172.21.15.83:6789/0 conn(0x7f65f80d1040 legacy=0x7f65f815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0003a60 0x7f65e0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 0x7f65e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e0005260 0x7f65e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65f815ed20 con 0x7f65e0004810 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65e000c1f0 con 0x7f65e0005260 2024-05-17T21:58:46.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65e000c3f0 con 0x7f65e0003a60 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 0x7f65e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f77fe640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e0005260 0x7f65e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f6ffd640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0003a60 0x7f65e0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 -- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0003a60 msgr2=0x7f65e0003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0003a60 0x7f65e0003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 -- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e0005260 msgr2=0x7f65e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65e0005260 0x7f65e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f77fe640 1 -- 172.21.15.83:0/2426662126 reap_dead start 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f67fc640 1 -- 172.21.15.83:0/2426662126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e0003530 con 0x7f65e0004810 2024-05-17T21:58:46.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.461+0000 7f65f6ffd640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0003a60 0x7f65e0003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:46.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65f67fc640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 0x7f65e0004c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65f80ceb70 tx=0x7f65ec01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40526dac45f4004b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0013e0 con 0x7f65e0004810 2024-05-17T21:58:46.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec0052d0 con 0x7f65e0004810 2024-05-17T21:58:46.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65f5ffb640 1 -- 172.21.15.83:0/2426662126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec003840 con 0x7f65e0004810 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 -- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 msgr2=0x7f65e0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 0x7f65e0004c20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f65f80ceb70 tx=0x7f65ec01c330 comp rx=0 tx=0).stop 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 -- 172.21.15.83:0/2426662126 shutdown_connections 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 --2- 172.21.15.83:0/2426662126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65e0004810 0x7f65e0004c20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 -- 172.21.15.83:0/2426662126 >> 172.21.15.83:0/2426662126 conn(0x7f65f806cd90 msgr2=0x7f65f80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 -- 172.21.15.83:0/2426662126 shutdown_connections 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 -- 172.21.15.83:0/2426662126 wait complete. 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.465+0000 7f65fd70a640 1 Processor -- start 2024-05-17T21:58:46.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- start start 2024-05-17T21:58:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0004810 0x7f65f815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f813d9e0 0x7f65f813ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8144490 con 0x7f65f80b22c0 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65f806f210 con 0x7f65f813d9e0 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65f815e080 con 0x7f65e0004810 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50136/0 (socket says 172.21.15.83:50136) 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 -- 172.21.15.83:0/3189075846 learned_addr learned my addr 172.21.15.83:0/3189075846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f6ffd640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0004810 0x7f65f815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f67fc640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f813d9e0 0x7f65f813ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 -- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0004810 msgr2=0x7f65f815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0004810 0x7f65f815d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 -- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f813d9e0 msgr2=0x7f65f813ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f813d9e0 0x7f65f813ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 -- 172.21.15.83:0/3189075846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f80b2c10 con 0x7f65f80b22c0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65f77fe640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65f80cfe60 tx=0x7f65f0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0003620 con 0x7f65f80b22c0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f001b070 con 0x7f65f80b22c0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0003a20 con 0x7f65f80b22c0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f81615b0 con 0x7f65f80b22c0 2024-05-17T21:58:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f8161b70 con 0x7f65f80b22c0 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f815e3b0 con 0x7f65f80b22c0 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.469+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f65f0003bc0 con 0x7f65f80b22c0 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.473+0000 7f65df7fe640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 0x7f65e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.473+0000 7f65f6ffd640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 0x7f65e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.473+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f65f008f9e0 con 0x7f65f80b22c0 2024-05-17T21:58:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.473+0000 7f65f6ffd640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 0x7f65e4071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65f806b180 tx=0x7f65e8001280 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.473+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f65f0024020 con 0x7f65f80b22c0 2024-05-17T21:58:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.625+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65f80b1300 con 0x7f65f80b22c0 2024-05-17T21:58:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.625+0000 7f65df7fe640 1 -- 172.21.15.83:0/3189075846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16645 (secure 0 0 0) 0x7f65f80b2c10 con 0x7f65f80b22c0 2024-05-17T21:58:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:46.620 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:18.200289+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:14.593534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:15.504714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 msgr2=0x7f65e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 0x7f65e4071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65f806b180 tx=0x7f65e8001280 comp rx=0 tx=0).stop 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 msgr2=0x7f65f80b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f65f80cfe60 tx=0x7f65f0011920 comp rx=0 tx=0).stop 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 shutdown_connections 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65f80b22c0 0x7f65f80b26d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65e406f400 0x7f65e4071910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f813d9e0 0x7f65f813ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 --2- 172.21.15.83:0/3189075846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65e0004810 0x7f65f815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 >> 172.21.15.83:0/3189075846 conn(0x7f65f806cd90 msgr2=0x7f65f80cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 shutdown_connections 2024-05-17T21:58:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.629+0000 7f65fd70a640 1 -- 172.21.15.83:0/3189075846 wait complete. 2024-05-17T21:58:46.640 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T21:58:46.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:46.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 Processor -- start 2024-05-17T21:58:46.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 -- start start 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d3c0af3c0 0x7f1d3c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d3c0afd10 0x7f1d3c06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3c0b16f0 con 0x7f1d3c15df90 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c0b0120 con 0x7f1d3c0afd10 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d42635640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c0b6190 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d3c0afd10 0x7f1d3c06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1d3c0afd10 0x7f1d3c06c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50140/0 (socket says 172.21.15.83:50140) 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 -- 172.21.15.83:0/1412555343 learned_addr learned my addr 172.21.15.83:0/1412555343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d3b7fe640 1 --2- 172.21.15.83:0/1412555343 >> v2:172.21.15.155:3300/0 conn(0x7f1d3c0af3c0 0x7f1d3c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 -- 172.21.15.83:0/1412555343 >> v2:172.21.15.155:3300/0 conn(0x7f1d3c0af3c0 msgr2=0x7f1d3c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 --2- 172.21.15.83:0/1412555343 >> v2:172.21.15.155:3300/0 conn(0x7f1d3c0af3c0 0x7f1d3c0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 -- 172.21.15.83:0/1412555343 >> v1:172.21.15.155:6789/0 conn(0x7f1d3c15df90 legacy=0x7f1d3c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:46.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.785+0000 7f1d40bd2640 1 -- 172.21.15.83:0/1412555343 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c0b6470 con 0x7f1d3c0afd10 2024-05-17T21:58:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d40bd2640 1 --2- 172.21.15.83:0/1412555343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0afd10 0x7f1d3c06c790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b10a0 tx=0x7f1d3401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc58cdc522fafe1a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d3affd640 1 -- 172.21.15.83:0/1412555343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340a55e0 con 0x7f1d3c0afd10 2024-05-17T21:58:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d3affd640 1 -- 172.21.15.83:0/1412555343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340a5780 con 0x7f1d3c0afd10 2024-05-17T21:58:46.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d3affd640 1 -- 172.21.15.83:0/1412555343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340a5b50 con 0x7f1d3c0afd10 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 -- 172.21.15.83:0/1412555343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0afd10 msgr2=0x7f1d3c06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 --2- 172.21.15.83:0/1412555343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0afd10 0x7f1d3c06c790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b10a0 tx=0x7f1d3401c960 comp rx=0 tx=0).stop 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 -- 172.21.15.83:0/1412555343 shutdown_connections 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 --2- 172.21.15.83:0/1412555343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0afd10 0x7f1d3c06c790 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 --2- 172.21.15.83:0/1412555343 >> v2:172.21.15.155:3300/0 conn(0x7f1d3c0af3c0 0x7f1d3c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 -- 172.21.15.83:0/1412555343 >> 172.21.15.83:0/1412555343 conn(0x7f1d3c0d20b0 msgr2=0x7f1d3c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 -- 172.21.15.83:0/1412555343 shutdown_connections 2024-05-17T21:58:46.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.789+0000 7f1d42635640 1 -- 172.21.15.83:0/1412555343 wait complete. 2024-05-17T21:58:46.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 Processor -- start 2024-05-17T21:58:46.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- start start 2024-05-17T21:58:46.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d3c0afd10 0x7f1d3c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d3c15df90 0x7f1d3c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c06d4e0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c0b6da0 con 0x7f1d3c15df90 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c06d660 con 0x7f1d3c0afd10 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50148/0 (socket says 172.21.15.83:50148) 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 -- 172.21.15.83:0/3815205326 learned_addr learned my addr 172.21.15.83:0/3815205326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3b7fe640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d3c0afd10 0x7f1d3c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d40bd2640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d3c15df90 0x7f1d3c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 -- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d3c0afd10 msgr2=0x7f1d3c14c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d3c0afd10 0x7f1d3c14c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 -- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d3c15df90 msgr2=0x7f1d3c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d3c15df90 0x7f1d3c15c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 -- 172.21.15.83:0/3815205326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c1571d0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d3bfff640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00b410 tx=0x7f1d2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c015030 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c004020 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004410 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c1501a0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c150760 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c0af7d0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.793+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d2c018070 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.797+0000 7f1d38ff9640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 0x7f1d28071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.797+0000 7f1d3b7fe640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 0x7f1d28071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.797+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f1d2c08ede0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.797+0000 7f1d3b7fe640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 0x7f1d28071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0d1650 tx=0x7f1d30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:46.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.801+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1d3c1571d0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.981+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d3c0b56f0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.981+0000 7f1d38ff9640 1 -- 172.21.15.83:0/3815205326 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f1d2c05bcd0 con 0x7f1d3c0af3c0 2024-05-17T21:58:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:46.975 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 msgr2=0x7f1d28071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 0x7f1d28071940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0d1650 tx=0x7f1d30008040 comp rx=0 tx=0).stop 2024-05-17T21:58:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 msgr2=0x7f1d3c14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00b410 tx=0x7f1d2c00b8d0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 shutdown_connections 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2806f430 0x7f1d28071940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d3c15df90 0x7f1d3c15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d3c0afd10 0x7f1d3c14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 --2- 172.21.15.83:0/3815205326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d3c0af3c0 0x7f1d3c14fb00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 >> 172.21.15.83:0/3815205326 conn(0x7f1d3c0d20b0 msgr2=0x7f1d3c0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 shutdown_connections 2024-05-17T21:58:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:46.985+0000 7f1d42635640 1 -- 172.21.15.83:0/3815205326 wait complete. 2024-05-17T21:58:46.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:47.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 Processor -- start 2024-05-17T21:58:47.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 -- start start 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b80d1040 0x7f23b8144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b815e080 0x7f23b8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 0x7f23b8073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f23b815ea40 con 0x7f23b815e080 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f23b815ed20 con 0x7f23b815e490 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23c0422640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f23b815f000 con 0x7f23b80d1040 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b815e080 0x7f23b8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23b815e080 0x7f23b8072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55572/0 (socket says 172.21.15.83:55572) 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 -- 172.21.15.83:0/597113235 learned_addr learned my addr 172.21.15.83:0/597113235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23be9bf640 1 --2- 172.21.15.83:0/597113235 >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 0x7f23b8073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23be1be640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b80d1040 0x7f23b8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 -- 172.21.15.83:0/597113235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b80d1040 msgr2=0x7f23b8144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b80d1040 0x7f23b8144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 -- 172.21.15.83:0/597113235 >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 msgr2=0x7f23b8073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/597113235 >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 0x7f23b8073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23bd9bd640 1 -- 172.21.15.83:0/597113235 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b806ef30 con 0x7f23b815e080 2024-05-17T21:58:47.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.145+0000 7f23be9bf640 1 --2- 172.21.15.83:0/597113235 >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 0x7f23b8073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b815e080 0x7f23b8072e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23b806d730 tx=0x7f23a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e101e6d720807 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23bd1bc640 1 -- 172.21.15.83:0/597113235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80ab5f0 con 0x7f23b815e080 2024-05-17T21:58:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23bd1bc640 1 -- 172.21.15.83:0/597113235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a80b0070 con 0x7f23b815e080 2024-05-17T21:58:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23bd1bc640 1 -- 172.21.15.83:0/597113235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80ab9b0 con 0x7f23b815e080 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- 172.21.15.83:0/597113235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b815e080 msgr2=0x7f23b8072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b815e080 0x7f23b8072e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23b806d730 tx=0x7f23a801c520 comp rx=0 tx=0).stop 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- 172.21.15.83:0/597113235 shutdown_connections 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 --2- 172.21.15.83:0/597113235 >> v2:172.21.15.155:3300/0 conn(0x7f23b815e490 0x7f23b8073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b815e080 0x7f23b8072e30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 --2- 172.21.15.83:0/597113235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b80d1040 0x7f23b8144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- 172.21.15.83:0/597113235 >> 172.21.15.83:0/597113235 conn(0x7f23b806cd90 msgr2=0x7f23b80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- 172.21.15.83:0/597113235 shutdown_connections 2024-05-17T21:58:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- 172.21.15.83:0/597113235 wait complete. 2024-05-17T21:58:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 Processor -- start 2024-05-17T21:58:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.149+0000 7f23c0422640 1 -- start start 2024-05-17T21:58:47.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b815e080 0x7f23b813eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 0x7f23b813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81622a0 con 0x7f23b80d1040 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8162120 con 0x7f23b815e490 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f23b8162420 con 0x7f23b815e080 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55576/0 (socket says 172.21.15.83:55576) 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 -- 172.21.15.83:0/1161932505 learned_addr learned my addr 172.21.15.83:0/1161932505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be9bf640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 0x7f23b813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 -- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b815e080 msgr2=0x7f23b813eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b815e080 0x7f23b813eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b815e080 0x7f23b813eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 -- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 msgr2=0x7f23b813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 0x7f23b813f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 -- 172.21.15.83:0/1161932505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b815da50 con 0x7f23b80d1040 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be9bf640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 0x7f23b813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23be1be640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23b400d830 tx=0x7f23b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4018030 con 0x7f23b80d1040 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b400be10 con 0x7f23b80d1040 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b8142620 con 0x7f23b80d1040 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40098b0 con 0x7f23b80d1040 2024-05-17T21:58:47.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b80b0760 con 0x7f23b80d1040 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b8144080 con 0x7f23b80d1040 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23b401b070 con 0x7f23b80d1040 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 0x7f239c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f23b408f430 con 0x7f23b80d1040 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.153+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 0x7f239c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.157+0000 7f23bd9bd640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 0x7f239c071910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23a80af000 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.161+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f23b405c3b0 con 0x7f23b80d1040 2024-05-17T21:58:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.353+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23b806fa20 con 0x7f23b80d1040 2024-05-17T21:58:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.353+0000 7f23aeffd640 1 -- 172.21.15.83:0/1161932505 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5187 (secure 0 0 0) 0x7f23b405bcb0 con 0x7f23b80d1040 2024-05-17T21:58:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:47.348 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:58:16.566175+0000","modified":"2024-05-17T21:58:19.619567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4776":{"gid":4776,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3630234827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3630234827},{"type":"v1","addr":"172.21.15.83:6837","nonce":3630234827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4776,"qdb_cluster":[4776]},"id":2}]} 2024-05-17T21:58:47.348 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 16 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 msgr2=0x7f239c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 0x7f239c071910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23a80af000 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 msgr2=0x7f23b81420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23b400d830 tx=0x7f23b400dcf0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 shutdown_connections 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f239c06f400 0x7f239c071910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b815e490 0x7f23b813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23b815e080 0x7f23b813eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 --2- 172.21.15.83:0/1161932505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b80d1040 0x7f23b81420e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 >> 172.21.15.83:0/1161932505 conn(0x7f23b806cd90 msgr2=0x7f23b8070220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 shutdown_connections 2024-05-17T21:58:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.357+0000 7f23c0422640 1 -- 172.21.15.83:0/1161932505 wait complete. 2024-05-17T21:58:47.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:47.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 Processor -- start 2024-05-17T21:58:47.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 -- start start 2024-05-17T21:58:47.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cbc073780 con 0x7f3cbc0d1040 2024-05-17T21:58:47.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3cbc15e790 con 0x7f3cbc144080 2024-05-17T21:58:47.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc2f97640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cbc15ea40 con 0x7f3cbc073370 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cbbfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cbbfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:53992/0 (socket says 172.21.15.83:53992) 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:50214/0 (socket says 172.21.15.83:50214) 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cbbfff640 1 -- 172.21.15.83:0/1320878825 learned_addr learned my addr 172.21.15.83:0/1320878825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 -- 172.21.15.83:0/1320878825 >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 msgr2=0x7f3cbc072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 --2- 172.21.15.83:0/1320878825 >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 -- 172.21.15.83:0/1320878825 >> v1:172.21.15.155:6789/0 conn(0x7f3cbc0d1040 legacy=0x7f3cbc15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cc1534640 1 -- 172.21.15.83:0/1320878825 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc15ed20 con 0x7f3cbc073370 2024-05-17T21:58:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.509+0000 7f3cbbfff640 1 --2- 172.21.15.83:0/1320878825 >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:58:47.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc1534640 1 --2- 172.21.15.83:0/1320878825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009d90 tx=0x7f3cb401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e080f28ce4c55e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cbb7fe640 1 -- 172.21.15.83:0/1320878825 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40b2070 con 0x7f3cbc073370 2024-05-17T21:58:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cbb7fe640 1 -- 172.21.15.83:0/1320878825 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb40a5a00 con 0x7f3cbc073370 2024-05-17T21:58:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cbb7fe640 1 -- 172.21.15.83:0/1320878825 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40a5dd0 con 0x7f3cbc073370 2024-05-17T21:58:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1320878825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 msgr2=0x7f3cbc06f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1320878825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009d90 tx=0x7f3cb401c940 comp rx=0 tx=0).stop 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1320878825 shutdown_connections 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1320878825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc073370 0x7f3cbc06f730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1320878825 >> v2:172.21.15.155:3300/0 conn(0x7f3cbc144080 0x7f3cbc072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1320878825 >> 172.21.15.83:0/1320878825 conn(0x7f3cbc06cd90 msgr2=0x7f3cbc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1320878825 shutdown_connections 2024-05-17T21:58:47.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.513+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1320878825 wait complete. 2024-05-17T21:58:47.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 Processor -- start 2024-05-17T21:58:47.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- start start 2024-05-17T21:58:47.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cbc0d1040 0x7f3cbc140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc144080 0x7f3cbc131a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc070480 con 0x7f3cbc073370 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc144490 con 0x7f3cbc0d1040 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cbc070600 con 0x7f3cbc144080 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55580/0 (socket says 172.21.15.83:55580) 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 -- 172.21.15.83:0/1043564638 learned_addr learned my addr 172.21.15.83:0/1043564638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc1534640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc144080 0x7f3cbc131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cbbfff640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cbc0d1040 0x7f3cbc140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 -- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc144080 msgr2=0x7f3cbc131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc144080 0x7f3cbc131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 -- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cbc0d1040 msgr2=0x7f3cbc140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cbc0d1040 0x7f3cbc140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 -- 172.21.15.83:0/1043564638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc163f70 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc0d33640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c3c0 tx=0x7f3cac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac017070 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac0052d0 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0054c0 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc164250 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.517+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc164cc0 con 0x7f3cbc073370 2024-05-17T21:58:47.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cbc0709c0 con 0x7f3cbc073370 2024-05-17T21:58:47.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3cac01a070 con 0x7f3cbc073370 2024-05-17T21:58:47.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cb97fa640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 0x7f3ca8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cbbfff640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 0x7f3ca8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f3cac08e970 con 0x7f3cbc073370 2024-05-17T21:58:47.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.521+0000 7f3cbbfff640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 0x7f3ca8071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b2130 tx=0x7f3cb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.525+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3cbc073780 con 0x7f3cbc073370 2024-05-17T21:58:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.673+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cbc168600 con 0x7f3cbc073370 2024-05-17T21:58:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cb97fa640 1 -- 172.21.15.83:0/1043564638 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16645 (secure 0 0 0) 0x7f3cac05b7c0 con 0x7f3cbc073370 2024-05-17T21:58:47.668 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:47.669 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:18.200289+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:14.593534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:15.504714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 msgr2=0x7f3ca8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 0x7f3ca8071940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b2130 tx=0x7f3cb0008040 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 msgr2=0x7f3cbc140f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c3c0 tx=0x7f3cac00c880 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 shutdown_connections 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cbc144080 0x7f3cbc131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ca806f430 0x7f3ca8071940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cbc0d1040 0x7f3cbc140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 --2- 172.21.15.83:0/1043564638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc073370 0x7f3cbc140f60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 >> 172.21.15.83:0/1043564638 conn(0x7f3cbc06cd90 msgr2=0x7f3cbc142a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 shutdown_connections 2024-05-17T21:58:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.677+0000 7f3cc2f97640 1 -- 172.21.15.83:0/1043564638 wait complete. 2024-05-17T21:58:47.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T21:58:47.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 Processor -- start 2024-05-17T21:58:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 -- start start 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b6c09ba00 0x7f8b6c09be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8b6c071400 0x7f8b6c071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b6c09c350 con 0x7f8b6c0b88b0 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6c0d1a40 con 0x7f8b6c09ba00 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b71921640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6c09d040 con 0x7f8b6c071400 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b6c09ba00 0x7f8b6c09be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b6c09ba00 0x7f8b6c09be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55586/0 (socket says 172.21.15.83:55586) 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 -- 172.21.15.83:0/2894907482 learned_addr learned my addr 172.21.15.83:0/2894907482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b6b7fe640 1 --2- 172.21.15.83:0/2894907482 >> v2:172.21.15.155:3300/0 conn(0x7f8b6c071400 0x7f8b6c071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 -- 172.21.15.83:0/2894907482 >> v2:172.21.15.155:3300/0 conn(0x7f8b6c071400 msgr2=0x7f8b6c071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 --2- 172.21.15.83:0/2894907482 >> v2:172.21.15.155:3300/0 conn(0x7f8b6c071400 0x7f8b6c071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 -- 172.21.15.83:0/2894907482 >> v1:172.21.15.155:6789/0 conn(0x7f8b6c0b88b0 legacy=0x7f8b6c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.833+0000 7f8b63fff640 1 -- 172.21.15.83:0/2894907482 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c09c710 con 0x7f8b6c09ba00 2024-05-17T21:58:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b63fff640 1 --2- 172.21.15.83:0/2894907482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c09be10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c09cfb0 tx=0x7f8b5401c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b94278c0910d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b6a7fc640 1 -- 172.21.15.83:0/2894907482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540b0070 con 0x7f8b6c09ba00 2024-05-17T21:58:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b6a7fc640 1 -- 172.21.15.83:0/2894907482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b540ab600 con 0x7f8b6c09ba00 2024-05-17T21:58:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b6a7fc640 1 -- 172.21.15.83:0/2894907482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540ab9d0 con 0x7f8b6c09ba00 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- 172.21.15.83:0/2894907482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c09ba00 msgr2=0x7f8b6c09be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 --2- 172.21.15.83:0/2894907482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c09be10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c09cfb0 tx=0x7f8b5401c220 comp rx=0 tx=0).stop 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- 172.21.15.83:0/2894907482 shutdown_connections 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 --2- 172.21.15.83:0/2894907482 >> v2:172.21.15.155:3300/0 conn(0x7f8b6c071400 0x7f8b6c071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 --2- 172.21.15.83:0/2894907482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c09be10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- 172.21.15.83:0/2894907482 >> 172.21.15.83:0/2894907482 conn(0x7f8b6c0d3620 msgr2=0x7f8b6c0d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- 172.21.15.83:0/2894907482 shutdown_connections 2024-05-17T21:58:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- 172.21.15.83:0/2894907482 wait complete. 2024-05-17T21:58:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 Processor -- start 2024-05-17T21:58:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.837+0000 7f8b71921640 1 -- start start 2024-05-17T21:58:47.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 0x7f8b6c0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c16a870 con 0x7f8b6c071400 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c16a6f0 con 0x7f8b6c09ba00 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c16a9f0 con 0x7f8b6c0b88b0 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55592/0 (socket says 172.21.15.83:55592) 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 -- 172.21.15.83:0/847410774 learned_addr learned my addr 172.21.15.83:0/847410774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6b7fe640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 0x7f8b6c0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b63fff640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 -- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 msgr2=0x7f8b6c0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 0x7f8b6c0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 -- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b6c09ba00 msgr2=0x7f8b6c158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 -- 172.21.15.83:0/847410774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c159090 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6b7fe640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 0x7f8b6c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b6affd640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b440 tx=0x7f8b5c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c015020 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5c011830 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c1595a0 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c011c20 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c159b60 con 0x7f8b6c071400 2024-05-17T21:58:47.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c071810 con 0x7f8b6c071400 2024-05-17T21:58:47.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b5c018020 con 0x7f8b6c071400 2024-05-17T21:58:47.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 0x7f8b4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:47.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8b5c08eef0 con 0x7f8b6c071400 2024-05-17T21:58:47.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.841+0000 7f8b63fff640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 0x7f8b4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:47.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.845+0000 7f8b63fff640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 0x7f8b4c071940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ad000 tx=0x7f8b54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:47.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:47.845+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8b5c01f030 con 0x7f8b6c071400 2024-05-17T21:58:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.029+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b6c0d04f0 con 0x7f8b6c071400 2024-05-17T21:58:48.022 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T21:58:48.029+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:58:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.033+0000 7f8b68ff9640 1 -- 172.21.15.83:0/847410774 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f8b5c05be70 con 0x7f8b6c071400 2024-05-17T21:58:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.033+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 msgr2=0x7f8b4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.033+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 0x7f8b4c071940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ad000 tx=0x7f8b54008040 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.033+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 msgr2=0x7f8b6c152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.033+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b440 tx=0x7f8b5c00b900 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 shutdown_connections 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b6c0b88b0 0x7f8b6c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b4c06f430 0x7f8b4c071940 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ad000 tx=0x7f8b54008040 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b6c09ba00 0x7f8b6c158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 --2- 172.21.15.83:0/847410774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b6c071400 0x7f8b6c152cc0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00b440 tx=0x7f8b5c00b900 comp rx=0 tx=0).stop 2024-05-17T21:58:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 >> 172.21.15.83:0/847410774 conn(0x7f8b6c0d3620 msgr2=0x7f8b6c157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 shutdown_connections 2024-05-17T21:58:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.037+0000 7f8b71921640 1 -- 172.21.15.83:0/847410774 wait complete. 2024-05-17T21:58:48.039 INFO:tasks.ceph.mds.a.smithi083.stderr: -5782> 2024-05-17T21:58:21.127+0000 7f3579108640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:58:48.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T21:58:48.105 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:58:48.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 Processor -- start 2024-05-17T21:58:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 -- start start 2024-05-17T21:58:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5c300b5390 0x7f5c3015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5c300af370 0x7f5c300b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c300b58c0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5c300b5ba0 con 0x7f5c300af370 2024-05-17T21:58:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c364f0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5c300b5e80 con 0x7f5c300b5390 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5c300af370 0x7f5c300b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c34a8d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5c300b6ea0 0x7f5c30165720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53258/0 (socket says 172.21.15.83:53258) 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c34a8d640 1 -- 172.21.15.83:0/3155273097 learned_addr learned my addr 172.21.15.83:0/3155273097 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2ffff640 1 --2- 172.21.15.83:0/3155273097 >> v2:172.21.15.155:3300/0 conn(0x7f5c300b5390 0x7f5c3015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2f7fe640 1 -- 172.21.15.83:0/3155273097 >> v2:172.21.15.155:3300/0 conn(0x7f5c300b5390 msgr2=0x7f5c3015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2f7fe640 1 --2- 172.21.15.83:0/3155273097 >> v2:172.21.15.155:3300/0 conn(0x7f5c300b5390 0x7f5c3015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2f7fe640 1 -- 172.21.15.83:0/3155273097 >> v1:172.21.15.83:6789/0 conn(0x7f5c300b6ea0 legacy=0x7f5c30165720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2effd640 1 -- 172.21.15.83:0/3155273097 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206463656 0 0) 0x7f5c300b58c0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.189+0000 7f5c2f7fe640 1 -- 172.21.15.83:0/3155273097 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c300b6160 con 0x7f5c300af370 2024-05-17T21:58:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c2f7fe640 1 --2- 172.21.15.83:0/3155273097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300af370 0x7f5c300b6960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c300d3570 tx=0x7f5c2401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab261854f7794451 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c2effd640 1 -- 172.21.15.83:0/3155273097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240aa600 con 0x7f5c300af370 2024-05-17T21:58:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c2effd640 1 -- 172.21.15.83:0/3155273097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c240af070 con 0x7f5c300af370 2024-05-17T21:58:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c2effd640 1 -- 172.21.15.83:0/3155273097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240aa9c0 con 0x7f5c300af370 2024-05-17T21:58:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- 172.21.15.83:0/3155273097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300af370 msgr2=0x7f5c300b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 --2- 172.21.15.83:0/3155273097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300af370 0x7f5c300b6960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c300d3570 tx=0x7f5c2401c480 comp rx=0 tx=0).stop 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- 172.21.15.83:0/3155273097 shutdown_connections 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 --2- 172.21.15.83:0/3155273097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300af370 0x7f5c300b6960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 --2- 172.21.15.83:0/3155273097 >> v2:172.21.15.155:3300/0 conn(0x7f5c300b5390 0x7f5c3015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- 172.21.15.83:0/3155273097 >> 172.21.15.83:0/3155273097 conn(0x7f5c300b4090 msgr2=0x7f5c300b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- 172.21.15.83:0/3155273097 shutdown_connections 2024-05-17T21:58:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- 172.21.15.83:0/3155273097 wait complete. 2024-05-17T21:58:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 Processor -- start 2024-05-17T21:58:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.193+0000 7f5c364f0640 1 -- start start 2024-05-17T21:58:48.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c300af370 0x7f5c30155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c300b5390 0x7f5c30157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30169660 con 0x7f5c300b6ea0 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300b76c0 con 0x7f5c300b5390 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5c301697e0 con 0x7f5c300af370 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55624/0 (socket says 172.21.15.83:55624) 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 -- 172.21.15.83:0/2109516523 learned_addr learned my addr 172.21.15.83:0/2109516523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2ffff640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c300af370 0x7f5c30155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2f7fe640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c300b5390 0x7f5c30157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 -- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c300af370 msgr2=0x7f5c30155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c300af370 0x7f5c30155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 -- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c300b5390 msgr2=0x7f5c30157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c300b5390 0x7f5c30157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 -- 172.21.15.83:0/2109516523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c30151c40 con 0x7f5c300b6ea0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c34a8d640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c3c0 tx=0x7f5c2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280074e0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c280047f0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c300cf1c0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28004be0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c30151f20 con 0x7f5c300b6ea0 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c300b6960 con 0x7f5c300b6ea0 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c28011070 con 0x7f5c300b6ea0 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.197+0000 7f5c2cff9640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 0x7f5c1c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.201+0000 7f5c2ffff640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 0x7f5c1c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.201+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f5c2808eda0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.201+0000 7f5c2ffff640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 0x7f5c1c071940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009e90 tx=0x7f5c2000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:48.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.205+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5c2805bcb0 con 0x7f5c300b6ea0 2024-05-17T21:58:48.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:48.389+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5c30155ec0 con 0x7f5c300b6ea0 2024-05-17T21:58:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.033+0000 7f5c2cff9640 1 -- 172.21.15.83:0/2109516523 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f5c2805b5b0 con 0x7f5c300b6ea0 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 msgr2=0x7f5c1c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 0x7f5c1c071940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009e90 tx=0x7f5c2000b040 comp rx=0 tx=0).stop 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 msgr2=0x7f5c30158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c3c0 tx=0x7f5c2800c880 comp rx=0 tx=0).stop 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 shutdown_connections 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c300b6ea0 0x7f5c30158340 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c1c06f430 0x7f5c1c071940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c300b5390 0x7f5c30157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 --2- 172.21.15.83:0/2109516523 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c300af370 0x7f5c30155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 >> 172.21.15.83:0/2109516523 conn(0x7f5c300b4090 msgr2=0x7f5c30071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 shutdown_connections 2024-05-17T21:58:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.037+0000 7f5c364f0640 1 -- 172.21.15.83:0/2109516523 wait complete. 2024-05-17T21:58:49.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T21:58:49.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 Processor -- start 2024-05-17T21:58:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 -- start start 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feba4167850 0x7feba4169c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba40cff90 con 0x7feba40d17d0 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7feba4073710 con 0x7feba4167850 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7febaac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feba406d3b0 con 0x7feba406f6d0 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7feba3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7feba919d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feba4167850 0x7feba4169c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7feba3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:50258/0 (socket says 172.21.15.83:50258) 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7feba3fff640 1 -- 172.21.15.83:0/2800352506 learned_addr learned my addr 172.21.15.83:0/2800352506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.225+0000 7feba899c640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7feba40d17d0 0x7feba4073ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:53474/0 (socket says 172.21.15.83:53474) 2024-05-17T21:58:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba3fff640 1 -- 172.21.15.83:0/2800352506 >> v2:172.21.15.155:3300/0 conn(0x7feba4167850 msgr2=0x7feba4169c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba3fff640 1 --2- 172.21.15.83:0/2800352506 >> v2:172.21.15.155:3300/0 conn(0x7feba4167850 0x7feba4169c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba3fff640 1 -- 172.21.15.83:0/2800352506 >> v1:172.21.15.155:6789/0 conn(0x7feba40d17d0 legacy=0x7feba4073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba3fff640 1 -- 172.21.15.83:0/2800352506 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba406d690 con 0x7feba406f6d0 2024-05-17T21:58:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba3fff640 1 --2- 172.21.15.83:0/2800352506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feba4073890 tx=0x7feb9401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24441addd56fc506 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba37fe640 1 -- 172.21.15.83:0/2800352506 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940b0070 con 0x7feba406f6d0 2024-05-17T21:58:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba37fe640 1 -- 172.21.15.83:0/2800352506 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb940ab600 con 0x7feba406f6d0 2024-05-17T21:58:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7feba37fe640 1 -- 172.21.15.83:0/2800352506 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940ab9d0 con 0x7feba406f6d0 2024-05-17T21:58:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 -- 172.21.15.83:0/2800352506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 msgr2=0x7feba40742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 --2- 172.21.15.83:0/2800352506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feba4073890 tx=0x7feb9401c540 comp rx=0 tx=0).stop 2024-05-17T21:58:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 -- 172.21.15.83:0/2800352506 shutdown_connections 2024-05-17T21:58:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 --2- 172.21.15.83:0/2800352506 >> v2:172.21.15.155:3300/0 conn(0x7feba4167850 0x7feba4169c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 --2- 172.21.15.83:0/2800352506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba406f6d0 0x7feba40742b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 -- 172.21.15.83:0/2800352506 >> 172.21.15.83:0/2800352506 conn(0x7feba40b3ad0 msgr2=0x7feba40b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 -- 172.21.15.83:0/2800352506 shutdown_connections 2024-05-17T21:58:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.229+0000 7febaac00640 1 -- 172.21.15.83:0/2800352506 wait complete. 2024-05-17T21:58:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 Processor -- start 2024-05-17T21:58:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 -- start start 2024-05-17T21:58:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba40d17d0 0x7feba4169100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 0x7feba4169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feba416cb50 con 0x7feba406f6d0 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feba416c9d0 con 0x7feba40d17d0 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7febaac00640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feba416ccd0 con 0x7feba4167850 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7feba899c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7feba899c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55640/0 (socket says 172.21.15.83:55640) 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.233+0000 7feba899c640 1 -- 172.21.15.83:0/881233817 learned_addr learned my addr 172.21.15.83:0/881233817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba3fff640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba40d17d0 0x7feba4169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba919d640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 0x7feba4169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 -- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 msgr2=0x7feba4169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 0x7feba4169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 -- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba40d17d0 msgr2=0x7feba4169100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba40d17d0 0x7feba4169100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 -- 172.21.15.83:0/881233817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba4140650 con 0x7feba406f6d0 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba919d640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 0x7feba4169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba899c640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00c7c0 tx=0x7feb9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c017070 con 0x7feba406f6d0 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c00a640 con 0x7feba406f6d0 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba4140930 con 0x7feba406f6d0 2024-05-17T21:58:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00a9f0 con 0x7feba406f6d0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba4140ef0 con 0x7feba406f6d0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba4073ba0 con 0x7feba406f6d0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb9c01a070 con 0x7feba406f6d0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 0x7feb90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7feb9c08e810 con 0x7feba406f6d0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba3fff640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 0x7feb90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.237+0000 7feba3fff640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 0x7feb90071910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feba40b48c0 tx=0x7feb94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.241+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feb9c05b0f0 con 0x7feba406f6d0 2024-05-17T21:58:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:49.405+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feba413ed40 con 0x7feba406f6d0 2024-05-17T21:58:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.329+0000 7feba17fa640 1 -- 172.21.15.83:0/881233817 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v33) ==== 181+0+0 (secure 0 0 0) 0x7feb9c05f3f0 con 0x7feba406f6d0 2024-05-17T21:58:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 msgr2=0x7feb90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 0x7feb90071910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feba40b48c0 tx=0x7feb94008040 comp rx=0 tx=0).stop 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 msgr2=0x7feba415e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00c7c0 tx=0x7feb9c00cc80 comp rx=0 tx=0).stop 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 shutdown_connections 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feba4167850 0x7feba4169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feb9006f400 0x7feb90071910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feba40d17d0 0x7feba4169100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 --2- 172.21.15.83:0/881233817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feba406f6d0 0x7feba415e1e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 >> 172.21.15.83:0/881233817 conn(0x7feba40b3ad0 msgr2=0x7feba4070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 shutdown_connections 2024-05-17T21:58:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.333+0000 7febaac00640 1 -- 172.21.15.83:0/881233817 wait complete. 2024-05-17T21:58:50.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T21:58:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 Processor -- start 2024-05-17T21:58:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 -- start start 2024-05-17T21:58:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc294164190 0x7fc29415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2940b3020 con 0x7fc2940728e0 2024-05-17T21:58:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc2940d08f0 con 0x7fc294164190 2024-05-17T21:58:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc29b644640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc29415fff0 con 0x7fc2940cffa0 2024-05-17T21:58:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc298bdf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc2993e0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc294164190 0x7fc29415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc298bdf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:50262/0 (socket says 172.21.15.83:50262) 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc2993e0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc294164190 0x7fc29415fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54078/0 (socket says 172.21.15.83:54078) 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.521+0000 7fc298bdf640 1 -- 172.21.15.83:0/2256512900 learned_addr learned my addr 172.21.15.83:0/2256512900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc2993e0640 1 -- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 msgr2=0x7fc2940d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc2993e0640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc2993e0640 1 -- 172.21.15.83:0/2256512900 >> v1:172.21.15.155:6789/0 conn(0x7fc2940728e0 legacy=0x7fc294072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc2993e0640 1 -- 172.21.15.83:0/2256512900 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29406dea0 con 0x7fc294164190 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc287fff640 1 -- 172.21.15.83:0/2256512900 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626058453 0 0) 0x7fc28c0014a0 con 0x7fc2940728e0 2024-05-17T21:58:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc298bdf640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:58:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc2993e0640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc294164190 0x7fc29415fab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2900097c0 tx=0x7fc29001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912216f0e59a5514 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc287fff640 1 -- 172.21.15.83:0/2256512900 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900a7070 con 0x7fc294164190 2024-05-17T21:58:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc287fff640 1 -- 172.21.15.83:0/2256512900 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2900b0070 con 0x7fc294164190 2024-05-17T21:58:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.525+0000 7fc287fff640 1 -- 172.21.15.83:0/2256512900 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900ab890 con 0x7fc294164190 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc294164190 msgr2=0x7fc29415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc294164190 0x7fc29415fab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2900097c0 tx=0x7fc29001c470 comp rx=0 tx=0).stop 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- 172.21.15.83:0/2256512900 shutdown_connections 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2940cffa0 0x7fc2940d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- 172.21.15.83:0/2256512900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc294164190 0x7fc29415fab0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- 172.21.15.83:0/2256512900 >> 172.21.15.83:0/2256512900 conn(0x7fc2940b52c0 msgr2=0x7fc2940b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- 172.21.15.83:0/2256512900 shutdown_connections 2024-05-17T21:58:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- 172.21.15.83:0/2256512900 wait complete. 2024-05-17T21:58:50.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 Processor -- start 2024-05-17T21:58:50.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- start start 2024-05-17T21:58:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2940728e0 0x7fc29414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc294164190 0x7fc29415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2940d18d0 con 0x7fc2940cffa0 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc29406ea20 con 0x7fc2940728e0 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc29b644640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2941645a0 con 0x7fc294164190 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.529+0000 7fc298bdf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55644/0 (socket says 172.21.15.83:55644) 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 -- 172.21.15.83:0/3333993154 learned_addr learned my addr 172.21.15.83:0/3333993154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc2993e0640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2940728e0 0x7fc29414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc299be1640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc294164190 0x7fc29415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 -- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc294164190 msgr2=0x7fc29415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc294164190 0x7fc29415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 -- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2940728e0 msgr2=0x7fc29414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2940728e0 0x7fc29414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:50.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 -- 172.21.15.83:0/3333993154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2941584c0 con 0x7fc2940cffa0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc298bdf640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e8d0 tx=0x7fc28800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288004030 con 0x7fc2940cffa0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2880041d0 con 0x7fc2940cffa0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2941587a0 con 0x7fc2940cffa0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800c940 con 0x7fc2940cffa0 2024-05-17T21:58:50.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc294158d60 con 0x7fc2940cffa0 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2940ca460 con 0x7fc2940cffa0 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc28800cae0 con 0x7fc2940cffa0 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 0x7fc278071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc2993e0640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 0x7fc278071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5357+0+0 (secure 0 0 0) 0x7fc28808f7e0 con 0x7fc2940cffa0 2024-05-17T21:58:50.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.533+0000 7fc2993e0640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 0x7fc278071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2900af000 tx=0x7fc2900a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.537+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc288017020 con 0x7fc2940cffa0 2024-05-17T21:58:50.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:50.701+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc294073070 con 0x7fc2940cffa0 2024-05-17T21:58:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.357+0000 7fc285ffb640 1 -- 172.21.15.83:0/3333993154 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v34) ==== 169+0+0 (secure 0 0 0) 0x7fc28805c8a0 con 0x7fc2940cffa0 2024-05-17T21:58:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 msgr2=0x7fc278071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 0x7fc278071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2900af000 tx=0x7fc2900a7000 comp rx=0 tx=0).stop 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 msgr2=0x7fc29415aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc28800e8d0 tx=0x7fc28800ed90 comp rx=0 tx=0).stop 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 shutdown_connections 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc294164190 0x7fc29415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2940cffa0 0x7fc29415aba0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc27806f430 0x7fc278071940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 --2- 172.21.15.83:0/3333993154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2940728e0 0x7fc29414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 >> 172.21.15.83:0/3333993154 conn(0x7fc2940b52c0 msgr2=0x7fc2940d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 shutdown_connections 2024-05-17T21:58:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.361+0000 7fc29b644640 1 -- 172.21.15.83:0/3333993154 wait complete. 2024-05-17T21:58:51.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T21:58:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.537+0000 7f278858e640 1 Processor -- start 2024-05-17T21:58:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.537+0000 7f278858e640 1 -- start start 2024-05-17T21:58:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278858e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27800cf830 0x7f27800b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278858e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cdf40 0x7f278015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278858e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800b5640 con 0x7f278015d530 2024-05-17T21:58:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278858e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f27800b58e0 con 0x7f27800cf830 2024-05-17T21:58:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278858e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f278006f060 con 0x7f27800cdf40 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f2785b29640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cdf40 0x7f278015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27800cf830 0x7f27800b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27800cf830 0x7f27800b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54096/0 (socket says 172.21.15.83:54096) 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 -- 172.21.15.83:0/321932506 learned_addr learned my addr 172.21.15.83:0/321932506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 -- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cdf40 msgr2=0x7f278015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 --2- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cdf40 0x7f278015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 -- 172.21.15.83:0/321932506 >> v1:172.21.15.155:6789/0 conn(0x7f278015d530 legacy=0x7f278006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.541+0000 7f278632a640 1 -- 172.21.15.83:0/321932506 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278006f340 con 0x7f27800cf830 2024-05-17T21:58:51.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278632a640 1 --2- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27800cf830 0x7f27800b3aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f277c00bb10 tx=0x7f277c01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4fe1dcafab6d83c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f2785328640 1 -- 172.21.15.83:0/321932506 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0b4070 con 0x7f27800cf830 2024-05-17T21:58:51.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f2785328640 1 -- 172.21.15.83:0/321932506 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c0a7c10 con 0x7f27800cf830 2024-05-17T21:58:51.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f2785328640 1 -- 172.21.15.83:0/321932506 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0af440 con 0x7f27800cf830 2024-05-17T21:58:51.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27800cf830 msgr2=0x7f27800b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 --2- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27800cf830 0x7f27800b3aa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f277c00bb10 tx=0x7f277c01ec30 comp rx=0 tx=0).stop 2024-05-17T21:58:51.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- 172.21.15.83:0/321932506 shutdown_connections 2024-05-17T21:58:51.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 --2- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cdf40 0x7f278015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 --2- 172.21.15.83:0/321932506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27800cf830 0x7f27800b3aa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- 172.21.15.83:0/321932506 >> 172.21.15.83:0/321932506 conn(0x7f278006cd90 msgr2=0x7f27800b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:51.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- 172.21.15.83:0/321932506 shutdown_connections 2024-05-17T21:58:51.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- 172.21.15.83:0/321932506 wait complete. 2024-05-17T21:58:51.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 Processor -- start 2024-05-17T21:58:51.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.545+0000 7f278858e640 1 -- start start 2024-05-17T21:58:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cf830 0x7f2780151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f278015d530 0x7f2780151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2780073210 con 0x7f27800cdf40 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2780073090 con 0x7f278015d530 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2780073390 con 0x7f27800cf830 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55648/0 (socket says 172.21.15.83:55648) 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 -- 172.21.15.83:0/307429162 learned_addr learned my addr 172.21.15.83:0/307429162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2785b29640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cf830 0x7f2780151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2786b2b640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f278015d530 0x7f2780151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 -- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cf830 msgr2=0x7f2780151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cf830 0x7f2780151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 -- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f278015d530 msgr2=0x7f2780151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f278015d530 0x7f2780151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 -- 172.21.15.83:0/307429162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27801521a0 con 0x7f27800cdf40 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278632a640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b2c0 tx=0x7f277c01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0b4040 con 0x7f27800cdf40 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c0c1070 con 0x7f27800cdf40 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2780152480 con 0x7f27800cdf40 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0b9660 con 0x7f27800cdf40 2024-05-17T21:58:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2780152a40 con 0x7f27800cdf40 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27800b39b0 con 0x7f27800cdf40 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.549+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f277c0b7070 con 0x7f27800cdf40 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.553+0000 7f2776ffd640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 0x7f2764071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.553+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f277c12ca50 con 0x7f27800cdf40 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.553+0000 7f2785b29640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 0x7f2764071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.553+0000 7f2785b29640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 0x7f2764071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27800b0810 tx=0x7f2770008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.553+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f277c018820 con 0x7f27800cdf40 2024-05-17T21:58:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.721+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f278015cff0 con 0x7f27800cdf40 2024-05-17T21:58:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f2776ffd640 1 -- 172.21.15.83:0/307429162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f277c0f9ed0 con 0x7f27800cdf40 2024-05-17T21:58:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 msgr2=0x7f2764071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 0x7f2764071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27800b0810 tx=0x7f2770008040 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 msgr2=0x7f2780156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b2c0 tx=0x7f277c01b9d0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 shutdown_connections 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f276406f400 0x7f2764071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f278015d530 0x7f2780151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27800cf830 0x7f2780151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 --2- 172.21.15.83:0/307429162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27800cdf40 0x7f2780156c10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.737+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 >> 172.21.15.83:0/307429162 conn(0x7f278006cd90 msgr2=0x7f27800b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.741+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 shutdown_connections 2024-05-17T21:58:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.741+0000 7f278858e640 1 -- 172.21.15.83:0/307429162 wait complete. 2024-05-17T21:58:51.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-17T21:58:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 Processor -- start 2024-05-17T21:58:51.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 -- start start 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9fb80d0370 0x7f9fb815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9fb80d44a0 0x7f9fb806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb806e6c0 con 0x7f9fb806e2b0 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80b8d00 con 0x7f9fb80d44a0 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fbd668640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb80d55e0 con 0x7f9fb80d0370 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9fb80d44a0 0x7f9fb806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9fb80d44a0 0x7f9fb806dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55662/0 (socket says 172.21.15.83:55662) 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 -- 172.21.15.83:0/2232380571 learned_addr learned my addr 172.21.15.83:0/2232380571 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb6ffd640 1 --2- 172.21.15.83:0/2232380571 >> v2:172.21.15.155:3300/0 conn(0x7f9fb80d0370 0x7f9fb815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 -- 172.21.15.83:0/2232380571 >> v2:172.21.15.155:3300/0 conn(0x7f9fb80d0370 msgr2=0x7f9fb815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 --2- 172.21.15.83:0/2232380571 >> v2:172.21.15.155:3300/0 conn(0x7f9fb80d0370 0x7f9fb815fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 -- 172.21.15.83:0/2232380571 >> v1:172.21.15.155:6789/0 conn(0x7f9fb806e2b0 legacy=0x7f9fb80b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:51.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.909+0000 7f9fb67fc640 1 -- 172.21.15.83:0/2232380571 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80d4a60 con 0x7f9fb80d44a0 2024-05-17T21:58:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fb67fc640 1 --2- 172.21.15.83:0/2232380571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb80d44a0 0x7f9fb806dd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80b8410 tx=0x7f9fa001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b759fc600c3d53c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fb5ffb640 1 -- 172.21.15.83:0/2232380571 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00b0070 con 0x7f9fb80d44a0 2024-05-17T21:58:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fb5ffb640 1 -- 172.21.15.83:0/2232380571 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa00ab600 con 0x7f9fb80d44a0 2024-05-17T21:58:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fb5ffb640 1 -- 172.21.15.83:0/2232380571 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00ab9d0 con 0x7f9fb80d44a0 2024-05-17T21:58:51.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- 172.21.15.83:0/2232380571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb80d44a0 msgr2=0x7f9fb806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2232380571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb80d44a0 0x7f9fb806dd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80b8410 tx=0x7f9fa001c340 comp rx=0 tx=0).stop 2024-05-17T21:58:51.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- 172.21.15.83:0/2232380571 shutdown_connections 2024-05-17T21:58:51.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2232380571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb80d44a0 0x7f9fb806dd70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2232380571 >> v2:172.21.15.155:3300/0 conn(0x7f9fb80d0370 0x7f9fb815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- 172.21.15.83:0/2232380571 >> 172.21.15.83:0/2232380571 conn(0x7f9fb8074a40 msgr2=0x7f9fb80d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:51.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- 172.21.15.83:0/2232380571 shutdown_connections 2024-05-17T21:58:51.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- 172.21.15.83:0/2232380571 wait complete. 2024-05-17T21:58:51.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 Processor -- start 2024-05-17T21:58:51.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.913+0000 7f9fbd668640 1 -- start start 2024-05-17T21:58:51.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fb80d0370 0x7f9fb8158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fb80d44a0 0x7f9fb8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80bb180 con 0x7f9fb806e2b0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb80d3c80 con 0x7f9fb80d0370 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb80bb300 con 0x7f9fb80d44a0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55670/0 (socket says 172.21.15.83:55670) 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 -- 172.21.15.83:0/2588855033 learned_addr learned my addr 172.21.15.83:0/2588855033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 -- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fb80d44a0 msgr2=0x7f9fb8158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fb80d44a0 0x7f9fb8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 -- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fb80d0370 msgr2=0x7f9fb8158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fb80d0370 0x7f9fb8158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 -- 172.21.15.83:0/2588855033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb80cbf70 con 0x7f9fb806e2b0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fb6ffd640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800d5f0 tx=0x7f9fa800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80125f0 con 0x7f9fb806e2b0 2024-05-17T21:58:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa8019070 con 0x7f9fb806e2b0 2024-05-17T21:58:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb80cc250 con 0x7f9fb806e2b0 2024-05-17T21:58:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80129b0 con 0x7f9fb806e2b0 2024-05-17T21:58:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb8160e00 con 0x7f9fb806e2b0 2024-05-17T21:58:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb815fd00 con 0x7f9fb806e2b0 2024-05-17T21:58:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.917+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fa8012b50 con 0x7f9fb806e2b0 2024-05-17T21:58:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.921+0000 7f9faf7fe640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.921+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f9fa808f410 con 0x7f9fb806e2b0 2024-05-17T21:58:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.921+0000 7f9fb67fc640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.921+0000 7f9fb67fc640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ad000 tx=0x7f9fa0004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:51.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:51.925+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9fa805c690 con 0x7f9fb806e2b0 2024-05-17T21:58:52.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.137+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f9fb80cbcd0 con 0x7f9fb806e2b0 2024-05-17T21:58:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.357+0000 7f9faf7fe640 1 -- 172.21.15.83:0/2588855033 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v90) ==== 142+0+0 (secure 0 0 0) 0x7f9fb80cbcd0 con 0x7f9fb806e2b0 2024-05-17T21:58:52.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 msgr2=0x7f9f98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ad000 tx=0x7f9fa0004740 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 msgr2=0x7f9fb8156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800d5f0 tx=0x7f9fa800dab0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 shutdown_connections 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fb80d44a0 0x7f9fb8158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fb80d0370 0x7f9fb8158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 --2- 172.21.15.83:0/2588855033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb806e2b0 0x7f9fb8156240 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 >> 172.21.15.83:0/2588855033 conn(0x7f9fb8074a40 msgr2=0x7f9fb815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 shutdown_connections 2024-05-17T21:58:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.361+0000 7f9fbd668640 1 -- 172.21.15.83:0/2588855033 wait complete. 2024-05-17T21:58:52.375 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-17T21:58:52.375 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-17T21:58:52.375 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T21:58:52.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:58:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 Processor -- start 2024-05-17T21:58:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 -- start start 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89e80d1040 0x7f89e8144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 0x7f89e8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89e815ea40 con 0x7f89e815e490 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f89e815ed20 con 0x7f89e80d1040 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89ede02640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89e815f000 con 0x7f89e815e080 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89e77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89e80d1040 0x7f89e8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89e7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89e815e490 0x7f89e81616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53270/0 (socket says 172.21.15.83:53270) 2024-05-17T21:58:52.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89e7fff640 1 -- 172.21.15.83:0/4103136049 learned_addr learned my addr 172.21.15.83:0/4103136049 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.533+0000 7f89e6ffd640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 0x7f89e8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e77fe640 1 -- 172.21.15.83:0/4103136049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 msgr2=0x7f89e8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e77fe640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 0x7f89e8072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e77fe640 1 -- 172.21.15.83:0/4103136049 >> v1:172.21.15.83:6789/0 conn(0x7f89e815e490 legacy=0x7f89e81616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e77fe640 1 -- 172.21.15.83:0/4103136049 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e806ef30 con 0x7f89e80d1040 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e6ffd640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 0x7f89e8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:52.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e67fc640 1 -- 172.21.15.83:0/4103136049 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479302141 0 0) 0x7f89e815ea40 con 0x7f89e815e490 2024-05-17T21:58:52.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e77fe640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e8144080 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097c0 tx=0x7f89d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b0adf81a1a94b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e67fc640 1 -- 172.21.15.83:0/4103136049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80b1070 con 0x7f89e80d1040 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e67fc640 1 -- 172.21.15.83:0/4103136049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d80ac410 con 0x7f89e80d1040 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89e67fc640 1 -- 172.21.15.83:0/4103136049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80ac7e0 con 0x7f89e80d1040 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 -- 172.21.15.83:0/4103136049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 msgr2=0x7f89e8144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e8144080 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097c0 tx=0x7f89d801c4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 -- 172.21.15.83:0/4103136049 shutdown_connections 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e080 0x7f89e8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 --2- 172.21.15.83:0/4103136049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e8144080 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 -- 172.21.15.83:0/4103136049 >> 172.21.15.83:0/4103136049 conn(0x7f89e806cd90 msgr2=0x7f89e80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:52.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 -- 172.21.15.83:0/4103136049 shutdown_connections 2024-05-17T21:58:52.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.537+0000 7f89ede02640 1 -- 172.21.15.83:0/4103136049 wait complete. 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 Processor -- start 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- start start 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89e815e080 0x7f89e815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e490 0x7f89e8166f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89e816abe0 con 0x7f89e80d1040 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f89e816aa60 con 0x7f89e815e080 2024-05-17T21:58:52.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f89e816ad60 con 0x7f89e815e490 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55690/0 (socket says 172.21.15.83:55690) 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 -- 172.21.15.83:0/2630700738 learned_addr learned my addr 172.21.15.83:0/2630700738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 -- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e490 msgr2=0x7f89e8166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e490 0x7f89e8166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 -- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89e815e080 msgr2=0x7f89e815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89e815e080 0x7f89e815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 -- 172.21.15.83:0/2630700738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e81632b0 con 0x7f89e80d1040 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89e77fe640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89d801a0d0 tx=0x7f89d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80b1040 con 0x7f89e80d1040 2024-05-17T21:58:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d80bc070 con 0x7f89e80d1040 2024-05-17T21:58:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d80b5630 con 0x7f89e80d1040 2024-05-17T21:58:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e8163590 con 0x7f89e80d1040 2024-05-17T21:58:52.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.541+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e8163b50 con 0x7f89e80d1040 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e8144080 con 0x7f89e80d1040 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89d80b57d0 con 0x7f89e80d1040 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89cffff640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 0x7f89d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89e6ffd640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 0x7f89d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f89d8128ae0 con 0x7f89e80d1040 2024-05-17T21:58:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.545+0000 7f89e6ffd640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 0x7f89d4071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89e8074240 tx=0x7f89dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.549+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f89d8016090 con 0x7f89e80d1040 2024-05-17T21:58:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.709+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89e815de60 con 0x7f89e80d1040 2024-05-17T21:58:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89cffff640 1 -- 172.21.15.83:0/2630700738 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+12996 (secure 0 0 0) 0x7f89d80f5d60 con 0x7f89e80d1040 2024-05-17T21:58:52.705 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:52.706 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:58:51.347303+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3630234827":"2024-05-18T21:58:48.032188+0000","172.21.15.83:6835/3630234827":"2024-05-18T21:58:48.032188+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:58:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 msgr2=0x7f89d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 0x7f89d4071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89e8074240 tx=0x7f89dc008040 comp rx=0 tx=0).stop 2024-05-17T21:58:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 msgr2=0x7f89e81627f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89d801a0d0 tx=0x7f89d8019240 comp rx=0 tx=0).stop 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 shutdown_connections 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89e815e490 0x7f89e8166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f89d406f400 0x7f89d4071910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89e815e080 0x7f89e815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 --2- 172.21.15.83:0/2630700738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89e80d1040 0x7f89e81627f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 >> 172.21.15.83:0/2630700738 conn(0x7f89e806cd90 msgr2=0x7f89e8167b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 shutdown_connections 2024-05-17T21:58:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.713+0000 7f89ede02640 1 -- 172.21.15.83:0/2630700738 wait complete. 2024-05-17T21:58:52.728 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T21:58:52.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T21:58:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 Processor -- start 2024-05-17T21:58:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 -- start start 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01880b5b80 0x7f018815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 0x7f01880aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01880af4f0 con 0x7f01880b7430 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f01880afb00 con 0x7f01880b5b80 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018ee2a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01880b1710 con 0x7f018809bcc0 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018d3c7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 0x7f01880aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f0187fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01880b5b80 0x7f018815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018cbc6640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f01880b7430 0x7f01880b5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:53524/0 (socket says 172.21.15.83:53524) 2024-05-17T21:58:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.889+0000 7f018cbc6640 1 -- 172.21.15.83:0/790499281 learned_addr learned my addr 172.21.15.83:0/790499281 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f0187fff640 1 -- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 msgr2=0x7f01880aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f0187fff640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 0x7f01880aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f0187fff640 1 -- 172.21.15.83:0/790499281 >> v1:172.21.15.155:6789/0 conn(0x7f01880b7430 legacy=0x7f01880b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f0187fff640 1 -- 172.21.15.83:0/790499281 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01880af790 con 0x7f01880b5b80 2024-05-17T21:58:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f018d3c7640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 0x7f01880aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T21:58:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f0187fff640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f018815fc60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01700097c0 tx=0x7f017001cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4f04b10a3f5165 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f01877fe640 1 -- 172.21.15.83:0/790499281 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700b2070 con 0x7f01880b5b80 2024-05-17T21:58:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f01877fe640 1 -- 172.21.15.83:0/790499281 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01700a9070 con 0x7f01880b5b80 2024-05-17T21:58:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f01877fe640 1 -- 172.21.15.83:0/790499281 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700ad880 con 0x7f01880b5b80 2024-05-17T21:58:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f018ee2a640 1 -- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 msgr2=0x7f018815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.893+0000 7f018ee2a640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f018815fc60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01700097c0 tx=0x7f017001cde0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 -- 172.21.15.83:0/790499281 shutdown_connections 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f018809bcc0 0x7f01880aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 --2- 172.21.15.83:0/790499281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f018815fc60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 -- 172.21.15.83:0/790499281 >> 172.21.15.83:0/790499281 conn(0x7f018809b8b0 msgr2=0x7f01880b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 -- 172.21.15.83:0/790499281 shutdown_connections 2024-05-17T21:58:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 -- 172.21.15.83:0/790499281 wait complete. 2024-05-17T21:58:52.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 Processor -- start 2024-05-17T21:58:52.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.897+0000 7f018ee2a640 1 -- start start 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f01880ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 0x7f018814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01880b1070 con 0x7f018809bcc0 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01880b0ef0 con 0x7f01880b5b80 2024-05-17T21:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01880ce900 con 0x7f01880b7430 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55702/0 (socket says 172.21.15.83:55702) 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 -- 172.21.15.83:0/342183880 learned_addr learned my addr 172.21.15.83:0/342183880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f0187fff640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f01880ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018d3c7640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 0x7f018814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 -- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 msgr2=0x7f018814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 0x7f018814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 -- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 msgr2=0x7f01880ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f01880ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 -- 172.21.15.83:0/342183880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01880cc490 con 0x7f018809bcc0 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018d3c7640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 0x7f018814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018cbc6640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f018000cab0 tx=0x7f018000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180017070 con 0x7f018809bcc0 2024-05-17T21:58:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018000ac50 con 0x7f018809bcc0 2024-05-17T21:58:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180012400 con 0x7f018809bcc0 2024-05-17T21:58:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01881598a0 con 0x7f018809bcc0 2024-05-17T21:58:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01880cca10 con 0x7f018809bcc0 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01880aff90 con 0x7f018809bcc0 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.901+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f018001a070 con 0x7f018809bcc0 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.905+0000 7f01857fa640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 0x7f0168071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.905+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f018008ecd0 con 0x7f018809bcc0 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.905+0000 7f0187fff640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 0x7f0168071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.905+0000 7f0187fff640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 0x7f0168071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f01700af000 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:52.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:52.909+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f018005bec0 con 0x7f018809bcc0 2024-05-17T21:58:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01880707c0 con 0x7f018809bcc0 2024-05-17T21:58:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f01857fa640 1 -- 172.21.15.83:0/342183880 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f018005b7c0 con 0x7f018809bcc0 2024-05-17T21:58:53.058 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T21:58:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 msgr2=0x7f0168071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 0x7f0168071940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f01700af000 tx=0x7f0170008040 comp rx=0 tx=0).stop 2024-05-17T21:58:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 msgr2=0x7f01880cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f018000cab0 tx=0x7f018000cf70 comp rx=0 tx=0).stop 2024-05-17T21:58:53.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 shutdown_connections 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01880b7430 0x7f018814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f016806f430 0x7f0168071940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01880b5b80 0x7f01880ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 --2- 172.21.15.83:0/342183880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f018809bcc0 0x7f01880cfec0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 >> 172.21.15.83:0/342183880 conn(0x7f018809b8b0 msgr2=0x7f01880b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 shutdown_connections 2024-05-17T21:58:53.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.065+0000 7f018ee2a640 1 -- 172.21.15.83:0/342183880 wait complete. 2024-05-17T21:58:53.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-17T21:58:53.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 Processor -- start 2024-05-17T21:58:53.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 -- start start 2024-05-17T21:58:53.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9300d1040 0x7fb930144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93015e080 0x7fb930072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93015ea40 con 0x7fb93015e490 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb93015ed20 con 0x7fb9300d1040 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb937059640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb93015f000 con 0x7fb93015e080 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9300d1040 0x7fb930144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9300d1040 0x7fb930144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55716/0 (socket says 172.21.15.83:55716) 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 -- 172.21.15.83:0/1982515795 learned_addr learned my addr 172.21.15.83:0/1982515795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb92ffff640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93015e080 0x7fb930072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 -- 172.21.15.83:0/1982515795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93015e080 msgr2=0x7fb930072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93015e080 0x7fb930072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 -- 172.21.15.83:0/1982515795 >> v1:172.21.15.155:6789/0 conn(0x7fb93015e490 legacy=0x7fb9301616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:53.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.229+0000 7fb934df5640 1 -- 172.21.15.83:0/1982515795 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93006ef30 con 0x7fb9300d1040 2024-05-17T21:58:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb934df5640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9300d1040 0x7fb930144080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb928009e50 tx=0x7fb92801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd868333de8bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb92f7fe640 1 -- 172.21.15.83:0/1982515795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280a5780 con 0x7fb9300d1040 2024-05-17T21:58:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb92f7fe640 1 -- 172.21.15.83:0/1982515795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9280a5920 con 0x7fb9300d1040 2024-05-17T21:58:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb92f7fe640 1 -- 172.21.15.83:0/1982515795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9280a5cf0 con 0x7fb9300d1040 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 -- 172.21.15.83:0/1982515795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9300d1040 msgr2=0x7fb930144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9300d1040 0x7fb930144080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb928009e50 tx=0x7fb92801ca30 comp rx=0 tx=0).stop 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 -- 172.21.15.83:0/1982515795 shutdown_connections 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb93015e080 0x7fb930072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 --2- 172.21.15.83:0/1982515795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9300d1040 0x7fb930144080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 -- 172.21.15.83:0/1982515795 >> 172.21.15.83:0/1982515795 conn(0x7fb93006cd90 msgr2=0x7fb9300b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 -- 172.21.15.83:0/1982515795 shutdown_connections 2024-05-17T21:58:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.233+0000 7fb937059640 1 -- 172.21.15.83:0/1982515795 wait complete. 2024-05-17T21:58:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 Processor -- start 2024-05-17T21:58:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- start start 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9300d1040 0x7fb9301397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 0x7fb930142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb93016abe0 con 0x7fb93015e080 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb93016aa60 con 0x7fb93015e490 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb93016ad60 con 0x7fb9300d1040 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55720/0 (socket says 172.21.15.83:55720) 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 -- 172.21.15.83:0/3658533916 learned_addr learned my addr 172.21.15.83:0/3658533916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb934df5640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9300d1040 0x7fb9301397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb9355f6640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 0x7fb930142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 -- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9300d1040 msgr2=0x7fb9301397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9300d1040 0x7fb9301397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 -- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 msgr2=0x7fb930142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 0x7fb930142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 -- 172.21.15.83:0/3658533916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93013fbd0 con 0x7fb93015e080 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb9355f6640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 0x7fb930142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92ffff640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d800 tx=0x7fb92400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92400b6c0 con 0x7fb93015e080 2024-05-17T21:58:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924019070 con 0x7fb93015e080 2024-05-17T21:58:53.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb930144920 con 0x7fb93015e080 2024-05-17T21:58:53.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92400ba80 con 0x7fb93015e080 2024-05-17T21:58:53.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb930144c00 con 0x7fb93015e080 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.237+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb930144080 con 0x7fb93015e080 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.241+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb92400bc20 con 0x7fb93015e080 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.241+0000 7fb92d7fa640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 0x7fb91c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.241+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fb92408fe20 con 0x7fb93015e080 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.241+0000 7fb934df5640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 0x7fb91c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:53.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.241+0000 7fb934df5640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 0x7fb91c071960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb9280ae000 tx=0x7fb9280a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.245+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb92405d0f0 con 0x7fb93015e080 2024-05-17T21:58:53.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:53.461+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fb93013de50 con 0x7fb93015e080 2024-05-17T21:58:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.369+0000 7fb92d7fa640 1 -- 172.21.15.83:0/3658533916 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v92) ==== 152+0+0 (secure 0 0 0) 0x7fb92405c9f0 con 0x7fb93015e080 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 msgr2=0x7fb91c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 0x7fb91c071960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb9280ae000 tx=0x7fb9280a7070 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 msgr2=0x7fb930139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d800 tx=0x7fb92400dcc0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 shutdown_connections 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb91c06f450 0x7fb91c071960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb93015e490 0x7fb930142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93015e080 0x7fb930139cf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 --2- 172.21.15.83:0/3658533916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9300d1040 0x7fb9301397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 >> 172.21.15.83:0/3658533916 conn(0x7fb93006cd90 msgr2=0x7fb9300b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:54.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 shutdown_connections 2024-05-17T21:58:54.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.373+0000 7fb937059640 1 -- 172.21.15.83:0/3658533916 wait complete. 2024-05-17T21:58:54.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T21:58:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 Processor -- start 2024-05-17T21:58:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 -- start start 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f340b2f70 0x7f3f340d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f340cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3406c930 con 0x7f3f340d1d80 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3406d540 con 0x7f3f340b2f70 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f38d4b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3406cc10 con 0x7f3f340cf700 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f340b2f70 0x7f3f340d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f340b2f70 0x7f3f340d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55736/0 (socket says 172.21.15.83:55736) 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 -- 172.21.15.83:0/291376831 learned_addr learned my addr 172.21.15.83:0/291376831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f32d9d640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f340cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 -- 172.21.15.83:0/291376831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 msgr2=0x7f3f340cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f340cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 -- 172.21.15.83:0/291376831 >> v1:172.21.15.155:6789/0 conn(0x7f3f340d1d80 legacy=0x7f3f340ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.557+0000 7f3f3259c640 1 -- 172.21.15.83:0/291376831 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3406cef0 con 0x7f3f340b2f70 2024-05-17T21:58:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f3259c640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340b2f70 0x7f3f340d1840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009870 tx=0x7f3f340ce3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b54bd07c33c2730 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f31d9b640 1 -- 172.21.15.83:0/291376831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b2070 con 0x7f3f340b2f70 2024-05-17T21:58:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f31d9b640 1 -- 172.21.15.83:0/291376831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f240a9070 con 0x7f3f340b2f70 2024-05-17T21:58:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f31d9b640 1 -- 172.21.15.83:0/291376831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240ad880 con 0x7f3f340b2f70 2024-05-17T21:58:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 -- 172.21.15.83:0/291376831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340b2f70 msgr2=0x7f3f340d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340b2f70 0x7f3f340d1840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009870 tx=0x7f3f340ce3c0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 -- 172.21.15.83:0/291376831 shutdown_connections 2024-05-17T21:58:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f340cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/291376831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340b2f70 0x7f3f340d1840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 -- 172.21.15.83:0/291376831 >> 172.21.15.83:0/291376831 conn(0x7f3f340b4170 msgr2=0x7f3f340b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 -- 172.21.15.83:0/291376831 shutdown_connections 2024-05-17T21:58:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.561+0000 7f3f38d4b640 1 -- 172.21.15.83:0/291376831 wait complete. 2024-05-17T21:58:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 Processor -- start 2024-05-17T21:58:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- start start 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340b2f70 0x7f3f34150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f3415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3416cb00 con 0x7f3f340d1d80 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3416cc80 con 0x7f3f340b2f70 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3416ce00 con 0x7f3f340cf700 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55740/0 (socket says 172.21.15.83:55740) 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 -- 172.21.15.83:0/1003894602 learned_addr learned my addr 172.21.15.83:0/1003894602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f3259c640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340b2f70 0x7f3f34150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f29d9b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f3415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 -- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 msgr2=0x7f3f3415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f3415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 -- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340b2f70 msgr2=0x7f3f34150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340b2f70 0x7f3f34150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 -- 172.21.15.83:0/1003894602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f34150840 con 0x7f3f340d1d80 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f32d9d640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007d40 tx=0x7f3f2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c014420 con 0x7f3f340d1d80 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c019070 con 0x7f3f340d1d80 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f341fd060 con 0x7f3f340d1d80 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c014820 con 0x7f3f340d1d80 2024-05-17T21:58:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.565+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341fd620 con 0x7f3f340d1d80 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f341f9720 con 0x7f3f340d1d80 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f2c0149c0 con 0x7f3f340d1d80 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f2b7fe640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 0x7f3f14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f3f2c08f760 con 0x7f3f340d1d80 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f3259c640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 0x7f3f14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.569+0000 7f3f3259c640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 0x7f3f14071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f240af000 tx=0x7f3f24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.573+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3f2c05c9e0 con 0x7f3f340d1d80 2024-05-17T21:58:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.733+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3f340d1840 con 0x7f3f340d1d80 2024-05-17T21:58:54.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.737+0000 7f3f2b7fe640 1 -- 172.21.15.83:0/1003894602 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f3f2c05c2e0 con 0x7f3f340d1d80 2024-05-17T21:58:54.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 msgr2=0x7f3f14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 0x7f3f14071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f240af000 tx=0x7f3f24008040 comp rx=0 tx=0).stop 2024-05-17T21:58:54.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 msgr2=0x7f3f340b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007d40 tx=0x7f3f2c00e4b0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 shutdown_connections 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f340d1d80 0x7f3f340b6de0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f340cf700 0x7f3f3415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f1406f400 0x7f3f14071910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 --2- 172.21.15.83:0/1003894602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340b2f70 0x7f3f34150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 >> 172.21.15.83:0/1003894602 conn(0x7f3f340b4170 msgr2=0x7f3f340ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 shutdown_connections 2024-05-17T21:58:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:54.741+0000 7f3f38d4b640 1 -- 172.21.15.83:0/1003894602 wait complete. 2024-05-17T21:58:54.752 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T21:58:54.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:54.794 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:58:54.848 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T21:58:54.848 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:54.848 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:54.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:58:54.854 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:54.854 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:54.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:58:54.859 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:58:54.859 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:54.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:58:54.902 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:58:54.902 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:58:54.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:58:54.907 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T21:58:54.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:58:55.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.097+0000 7fb1cd6e5640 1 Processor -- start 2024-05-17T21:58:55.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.097+0000 7fb1cd6e5640 1 -- start start 2024-05-17T21:58:55.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1cd6e5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1c8144080 0x7fb1c8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1cd6e5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8073370 0x7fb1c806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1cd6e5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c8073780 con 0x7fb1c80d1040 2024-05-17T21:58:55.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1cd6e5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c815e790 con 0x7fb1c8144080 2024-05-17T21:58:55.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1cd6e5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c815ea40 con 0x7fb1c8073370 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1c8144080 0x7fb1c8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8073370 0x7fb1c806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1c8144080 0x7fb1c8072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54174/0 (socket says 172.21.15.83:54174) 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 -- 172.21.15.83:0/1623960236 learned_addr learned my addr 172.21.15.83:0/1623960236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 -- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8073370 msgr2=0x7fb1c806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 --2- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8073370 0x7fb1c806f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 -- 172.21.15.83:0/1623960236 >> v1:172.21.15.155:6789/0 conn(0x7fb1c80d1040 legacy=0x7fb1c815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.101+0000 7fb1c67fc640 1 -- 172.21.15.83:0/1623960236 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c815ed20 con 0x7fb1c8144080 2024-05-17T21:58:55.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1c67fc640 1 --2- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c8144080 0x7fb1c8072e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009870 tx=0x7fb1bc01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66882a99512a4ca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1c5ffb640 1 -- 172.21.15.83:0/1623960236 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0a95f0 con 0x7fb1c8144080 2024-05-17T21:58:55.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1c5ffb640 1 -- 172.21.15.83:0/1623960236 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0a9790 con 0x7fb1c8144080 2024-05-17T21:58:55.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1c5ffb640 1 -- 172.21.15.83:0/1623960236 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0a9b60 con 0x7fb1c8144080 2024-05-17T21:58:55.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c8144080 msgr2=0x7fb1c8072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c8144080 0x7fb1c8072e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009870 tx=0x7fb1bc01c530 comp rx=0 tx=0).stop 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/1623960236 shutdown_connections 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8073370 0x7fb1c806f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/1623960236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c8144080 0x7fb1c8072e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/1623960236 >> 172.21.15.83:0/1623960236 conn(0x7fb1c806cd90 msgr2=0x7fb1c80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/1623960236 shutdown_connections 2024-05-17T21:58:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.105+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/1623960236 wait complete. 2024-05-17T21:58:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 Processor -- start 2024-05-17T21:58:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- start start 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c80d1040 0x7fb1c815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8144080 0x7fb1c80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8070480 con 0x7fb1c8073370 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8144490 con 0x7fb1c80d1040 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c8070600 con 0x7fb1c8144080 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55742/0 (socket says 172.21.15.83:55742) 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 -- 172.21.15.83:0/3862569270 learned_addr learned my addr 172.21.15.83:0/3862569270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c77fe640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8144080 0x7fb1c80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 -- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8144080 msgr2=0x7fb1c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8144080 0x7fb1c80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 -- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c80d1040 msgr2=0x7fb1c815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c80d1040 0x7fb1c815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 -- 172.21.15.83:0/3862569270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c80b5180 con 0x7fb1c8073370 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1c6ffd640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c420 tx=0x7fb1b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8017070 con 0x7fb1c8073370 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b80052d0 con 0x7fb1c8073370 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c80b5460 con 0x7fb1c8073370 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80054c0 con 0x7fb1c8073370 2024-05-17T21:58:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c80b5a20 con 0x7fb1c8073370 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.109+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c806f730 con 0x7fb1c8073370 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.113+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1b801a070 con 0x7fb1c8073370 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.113+0000 7fb1af7fe640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 0x7fb1b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.113+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fb1b808e420 con 0x7fb1c8073370 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.113+0000 7fb1c67fc640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 0x7fb1b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.113+0000 7fb1c67fc640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 0x7fb1b4071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ae000 tx=0x7fb1bc004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.117+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb1c8073780 con 0x7fb1c8073370 2024-05-17T21:58:55.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c8145170 con 0x7fb1c8073370 2024-05-17T21:58:55.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1af7fe640 1 -- 172.21.15.83:0/3862569270 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3875 (secure 0 0 0) 0x7fb1b805b590 con 0x7fb1c8073370 2024-05-17T21:58:55.289 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:55.290 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3005345985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3005345985},{"type":"v1","addr":"172.21.15.155:6834","nonce":3005345985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2588641383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2588641383},{"type":"v1","addr":"172.21.15.155:6835","nonce":2588641383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2647977650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2647977650},{"type":"v1","addr":"172.21.15.83:6836","nonce":2647977650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5091,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1976770616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1976770616},{"type":"v1","addr":"172.21.15.83:6837","nonce":1976770616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-05-17T21:58:55.290 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 msgr2=0x7fb1b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 0x7fb1b4071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ae000 tx=0x7fb1bc004740 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 msgr2=0x7fb1c80ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c420 tx=0x7fb1b800c8e0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 shutdown_connections 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c8144080 0x7fb1c80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb1b406f400 0x7fb1b4071910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c80d1040 0x7fb1c815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 --2- 172.21.15.83:0/3862569270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c8073370 0x7fb1c80ad630 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 >> 172.21.15.83:0/3862569270 conn(0x7fb1c806cd90 msgr2=0x7fb1c8161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:55.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 shutdown_connections 2024-05-17T21:58:55.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.297+0000 7fb1cd6e5640 1 -- 172.21.15.83:0/3862569270 wait complete. 2024-05-17T21:58:55.310 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T21:58:55.310 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T21:58:55.311 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T21:58:55.311 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:55.311 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T21:58:55.311 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T21:58:55.311 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T21:58:55.311 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:55.311 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T21:58:55.312 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T21:58:55.312 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T21:58:55.312 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:55.312 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T21:58:55.312 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T21:58:55.312 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T21:58:55.312 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:58:55.324 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T21:58:55.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T21:58:55.325 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T21:58:55.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T21:58:55.331 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T21:58:55.331 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T21:58:55.332 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T21:58:55.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T21:58:55.369 INFO:tasks.ceph.mds.c:Started 2024-05-17T21:58:55.369 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:55.370 INFO:tasks.ceph.mds.a:Started 2024-05-17T21:58:55.370 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:55.375 INFO:tasks.ceph.mds.b:Started 2024-05-17T21:58:55.375 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:55.376 INFO:tasks.ceph.mds.d:Started 2024-05-17T21:58:55.376 DEBUG:teuthology.parallel:result is None 2024-05-17T21:58:55.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T21:58:55.455 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T21:58:55.455 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:58:55.475 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T21:58:55.475 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T21:58:55.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 Processor -- start 2024-05-17T21:58:55.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 -- start start 2024-05-17T21:58:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff2cc0b4810 0x7ff2cc0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0d2000 0x7ff2cc0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2cc0d4170 con 0x7ff2cc0d0290 2024-05-17T21:58:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff2cc0d4450 con 0x7ff2cc0b4810 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d26b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc0d4730 con 0x7ff2cc0d2000 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff2cc0b4810 0x7ff2cc0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff2cc0d0290 0x7ff2cc15fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53304/0 (socket says 172.21.15.83:53304) 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cb7fe640 1 -- 172.21.15.83:0/195505563 learned_addr learned my addr 172.21.15.83:0/195505563 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2d0c51640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0d2000 0x7ff2cc0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cbfff640 1 -- 172.21.15.83:0/195505563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0d2000 msgr2=0x7ff2cc0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cbfff640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0d2000 0x7ff2cc0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cbfff640 1 -- 172.21.15.83:0/195505563 >> v1:172.21.15.83:6789/0 conn(0x7ff2cc0d0290 legacy=0x7ff2cc15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2caffd640 1 -- 172.21.15.83:0/195505563 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3234564339 0 0) 0x7ff2cc0d4170 con 0x7ff2cc0d0290 2024-05-17T21:58:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.521+0000 7ff2cbfff640 1 -- 172.21.15.83:0/195505563 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc0d4a10 con 0x7ff2cc0b4810 2024-05-17T21:58:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2cbfff640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0b4810 0x7ff2cc0d5210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5206ac8a2039d8a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2caffd640 1 -- 172.21.15.83:0/195505563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00aa620 con 0x7ff2cc0b4810 2024-05-17T21:58:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2caffd640 1 -- 172.21.15.83:0/195505563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c00af070 con 0x7ff2cc0b4810 2024-05-17T21:58:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2caffd640 1 -- 172.21.15.83:0/195505563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00aa9e0 con 0x7ff2cc0b4810 2024-05-17T21:58:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- 172.21.15.83:0/195505563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0b4810 msgr2=0x7ff2cc0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0b4810 0x7ff2cc0d5210 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097c0 tx=0x7ff2c001c4a0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- 172.21.15.83:0/195505563 shutdown_connections 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0d2000 0x7ff2cc0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/195505563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0b4810 0x7ff2cc0d5210 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- 172.21.15.83:0/195505563 >> 172.21.15.83:0/195505563 conn(0x7ff2cc06e010 msgr2=0x7ff2cc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- 172.21.15.83:0/195505563 shutdown_connections 2024-05-17T21:58:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- 172.21.15.83:0/195505563 wait complete. 2024-05-17T21:58:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 Processor -- start 2024-05-17T21:58:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.525+0000 7ff2d26b4640 1 -- start start 2024-05-17T21:58:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0b4810 0x7ff2cc1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2cc0d0290 0x7ff2cc0b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b9670 con 0x7ff2cc0d2000 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b94f0 con 0x7ff2cc0d0290 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc0b97f0 con 0x7ff2cc0b4810 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55782/0 (socket says 172.21.15.83:55782) 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 -- 172.21.15.83:0/211886165 learned_addr learned my addr 172.21.15.83:0/211886165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2cbfff640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0b4810 0x7ff2cc1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2cb7fe640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2cc0d0290 0x7ff2cc0b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 -- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0b4810 msgr2=0x7ff2cc1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0b4810 0x7ff2cc1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 -- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2cc0d0290 msgr2=0x7ff2cc0b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2cc0d0290 0x7ff2cc0b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 -- 172.21.15.83:0/211886165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc153150 con 0x7ff2cc0d2000 2024-05-17T21:58:55.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d0c51640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4007d80 tx=0x7ff2c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4014420 con 0x7ff2cc0d2000 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c4019070 con 0x7ff2cc0d2000 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4014800 con 0x7ff2cc0d2000 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc0b77c0 con 0x7ff2cc0d2000 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc0b7d80 con 0x7ff2cc0d2000 2024-05-17T21:58:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.529+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc0b7530 con 0x7ff2cc0d2000 2024-05-17T21:58:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.533+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2c40149a0 con 0x7ff2cc0d2000 2024-05-17T21:58:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.533+0000 7ff2c8ff9640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 0x7ff2b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.533+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7ff2c40900a0 con 0x7ff2cc0d2000 2024-05-17T21:58:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.533+0000 7ff2cbfff640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 0x7ff2b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.533+0000 7ff2cbfff640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 0x7ff2b8071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00ae000 tx=0x7ff2c00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.537+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2c405cd20 con 0x7ff2cc0d2000 2024-05-17T21:58:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:55.693+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff2cc0d5210 con 0x7ff2cc0d2000 2024-05-17T21:58:56.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.765+0000 7ff2c8ff9640 1 -- 172.21.15.83:0/211886165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7ff2c4061020 con 0x7ff2cc0d2000 2024-05-17T21:58:56.759 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T21:58:56.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 msgr2=0x7ff2b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 0x7ff2b8071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00ae000 tx=0x7ff2c00a7000 comp rx=0 tx=0).stop 2024-05-17T21:58:56.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 msgr2=0x7ff2cc15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4007d80 tx=0x7ff2c400e4b0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 shutdown_connections 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2cc0d2000 0x7ff2cc15b210 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2b806f400 0x7ff2b8071910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2cc0d0290 0x7ff2cc0b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 --2- 172.21.15.83:0/211886165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2cc0b4810 0x7ff2cc1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 >> 172.21.15.83:0/211886165 conn(0x7ff2cc06e010 msgr2=0x7ff2cc15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 shutdown_connections 2024-05-17T21:58:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.769+0000 7ff2d26b4640 1 -- 172.21.15.83:0/211886165 wait complete. 2024-05-17T21:58:56.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T21:58:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.949+0000 7f00425c7640 1 Processor -- start 2024-05-17T21:58:56.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.949+0000 7f00425c7640 1 -- start start 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f00425c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f003c1461c0 0x7f003c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f00425c7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c15e6b0 0x7f003c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f00425c7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003c1465d0 con 0x7f003c070510 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f00425c7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f003c06ef30 con 0x7f003c1461c0 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f00425c7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f003c06f210 con 0x7f003c15e6b0 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f003c1461c0 0x7f003c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f0040b64640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f003c070510 0x7f003c1605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53312/0 (socket says 172.21.15.83:53312) 2024-05-17T21:58:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f003c1461c0 0x7f003c15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55790/0 (socket says 172.21.15.83:55790) 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f0040b64640 1 -- 172.21.15.83:0/3899606278 learned_addr learned my addr 172.21.15.83:0/3899606278 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003b7fe640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c15e6b0 0x7f003c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 -- 172.21.15.83:0/3899606278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c15e6b0 msgr2=0x7f003c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c15e6b0 0x7f003c06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 -- 172.21.15.83:0/3899606278 >> v1:172.21.15.83:6789/0 conn(0x7f003c070510 legacy=0x7f003c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003affd640 1 -- 172.21.15.83:0/3899606278 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593517286 0 0) 0x7f003c1465d0 con 0x7f003c070510 2024-05-17T21:58:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.953+0000 7f003bfff640 1 -- 172.21.15.83:0/3899606278 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c070920 con 0x7f003c1461c0 2024-05-17T21:58:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f003bfff640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c15e170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f002c009820 tx=0x7f002c01e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709e0095e7faa91f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f003affd640 1 -- 172.21.15.83:0/3899606278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0ae070 con 0x7f003c1461c0 2024-05-17T21:58:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f003affd640 1 -- 172.21.15.83:0/3899606278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0a8650 con 0x7f003c1461c0 2024-05-17T21:58:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f003affd640 1 -- 172.21.15.83:0/3899606278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0a8a20 con 0x7f003c1461c0 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- 172.21.15.83:0/3899606278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 msgr2=0x7f003c15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c15e170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f002c009820 tx=0x7f002c01e680 comp rx=0 tx=0).stop 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- 172.21.15.83:0/3899606278 shutdown_connections 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c15e6b0 0x7f003c06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 --2- 172.21.15.83:0/3899606278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c15e170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- 172.21.15.83:0/3899606278 >> 172.21.15.83:0/3899606278 conn(0x7f003c06cd90 msgr2=0x7f003c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- 172.21.15.83:0/3899606278 shutdown_connections 2024-05-17T21:58:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- 172.21.15.83:0/3899606278 wait complete. 2024-05-17T21:58:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 Processor -- start 2024-05-17T21:58:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.957+0000 7f00425c7640 1 -- start start 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 0x7f003c159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 0x7f003c1437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f003c160e30 con 0x7f003c1461c0 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f003c160cb0 con 0x7f003c15e6b0 2024-05-17T21:58:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f003c0d10a0 con 0x7f003c070510 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55794/0 (socket says 172.21.15.83:55794) 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 -- 172.21.15.83:0/1788267754 learned_addr learned my addr 172.21.15.83:0/1788267754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003bfff640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 0x7f003c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0040b64640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 0x7f003c1437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 -- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 msgr2=0x7f003c159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 0x7f003c159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 -- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 msgr2=0x7f003c1437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 0x7f003c1437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 -- 172.21.15.83:0/1788267754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c0b6420 con 0x7f003c1461c0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0040b64640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 0x7f003c1437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003bfff640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 0x7f003c159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f003b7fe640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f003000d5f0 tx=0x7f003000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300125f0 con 0x7f003c1461c0 2024-05-17T21:58:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0030019070 con 0x7f003c1461c0 2024-05-17T21:58:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c0b5660 con 0x7f003c1461c0 2024-05-17T21:58:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300129b0 con 0x7f003c1461c0 2024-05-17T21:58:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c0b5c20 con 0x7f003c1461c0 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003c15e170 con 0x7f003c1461c0 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0030012b50 con 0x7f003c1461c0 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.961+0000 7f0038ff9640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 0x7f0028071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.965+0000 7f003bfff640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 0x7f0028071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.965+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f003008f1d0 con 0x7f003c1461c0 2024-05-17T21:58:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.965+0000 7f003bfff640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 0x7f0028071990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f002c0b0000 tx=0x7f002c0a9000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:56.969+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f003005c580 con 0x7f003c1461c0 2024-05-17T21:58:57.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.185+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f003c0b4900 con 0x7f003c1461c0 2024-05-17T21:58:57.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f0038ff9640 1 -- 172.21.15.83:0/1788267754 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f003005be80 con 0x7f003c1461c0 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.182 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:57.183 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T21:58:57.184 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.186 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.187 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T21:58:57.188 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T21:58:57.189 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.191 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:58:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:58:57.194 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:58:57.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.194 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T21:58:57.194 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 msgr2=0x7f0028071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 0x7f0028071990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f002c0b0000 tx=0x7f002c0a9000 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 msgr2=0x7f003c159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f003000d5f0 tx=0x7f003000dab0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 shutdown_connections 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f002806f480 0x7f0028071990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003c15e6b0 0x7f003c1437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f003c1461c0 0x7f003c159d40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 --2- 172.21.15.83:0/1788267754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003c070510 0x7f003c159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 >> 172.21.15.83:0/1788267754 conn(0x7f003c06cd90 msgr2=0x7f003c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:57.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 shutdown_connections 2024-05-17T21:58:57.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.189+0000 7f00425c7640 1 -- 172.21.15.83:0/1788267754 wait complete. 2024-05-17T21:58:57.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T21:58:57.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.353+0000 7f998f97a640 1 Processor -- start 2024-05-17T21:58:57.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.353+0000 7f998f97a640 1 -- start start 2024-05-17T21:58:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998f97a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99881605a0 0x7f99881609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998f97a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99881465d0 con 0x7f99881461c0 2024-05-17T21:58:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998f97a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9988160ef0 con 0x7f998806ffd0 2024-05-17T21:58:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998f97a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f998806ef30 con 0x7f99881605a0 2024-05-17T21:58:57.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998df17640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99881605a0 0x7f99881609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998d716640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f99881461c0 0x7f998815e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41582/0 (socket says 172.21.15.83:41582) 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998df17640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99881605a0 0x7f99881609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60268/0 (socket says 172.21.15.83:60268) 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998d716640 1 -- 172.21.15.83:0/77368820 learned_addr learned my addr 172.21.15.83:0/77368820 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998df17640 1 -- 172.21.15.83:0/77368820 >> v1:172.21.15.155:6789/0 conn(0x7f998806ffd0 legacy=0x7f99880703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998df17640 1 -- 172.21.15.83:0/77368820 >> v1:172.21.15.83:6789/0 conn(0x7f99881461c0 legacy=0x7f998815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.357+0000 7f998df17640 1 -- 172.21.15.83:0/77368820 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998806f210 con 0x7f99881605a0 2024-05-17T21:58:57.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998df17640 1 --2- 172.21.15.83:0/77368820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f99881609b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9980009d90 tx=0x7f998001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65be72ba5dc4085f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f997ffff640 1 -- 172.21.15.83:0/77368820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800b2070 con 0x7f99881605a0 2024-05-17T21:58:57.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f997ffff640 1 -- 172.21.15.83:0/77368820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800a5820 con 0x7f99881605a0 2024-05-17T21:58:57.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f997ffff640 1 -- 172.21.15.83:0/77368820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800a5bf0 con 0x7f99881605a0 2024-05-17T21:58:57.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- 172.21.15.83:0/77368820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 msgr2=0x7f99881609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 --2- 172.21.15.83:0/77368820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f99881609b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9980009d90 tx=0x7f998001c760 comp rx=0 tx=0).stop 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- 172.21.15.83:0/77368820 shutdown_connections 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 --2- 172.21.15.83:0/77368820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f99881609b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- 172.21.15.83:0/77368820 >> 172.21.15.83:0/77368820 conn(0x7f998806cd90 msgr2=0x7f99880b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- 172.21.15.83:0/77368820 shutdown_connections 2024-05-17T21:58:57.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- 172.21.15.83:0/77368820 wait complete. 2024-05-17T21:58:57.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 Processor -- start 2024-05-17T21:58:57.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.361+0000 7f998f97a640 1 -- start start 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f998806ffd0 0x7f998813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 0x7f998813d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d10a0 con 0x7f99881605a0 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f998815e880 con 0x7f99881461c0 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d1220 con 0x7f998806ffd0 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60278/0 (socket says 172.21.15.83:60278) 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 -- 172.21.15.83:0/1257298044 learned_addr learned my addr 172.21.15.83:0/1257298044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998cf15640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 0x7f998813d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 -- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f998806ffd0 msgr2=0x7f998813cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f998806ffd0 0x7f998813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 -- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 msgr2=0x7f998813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 0x7f998813d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 -- 172.21.15.83:0/1257298044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880af990 con 0x7f99881605a0 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998cf15640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 0x7f998813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998df17640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99800b0040 tx=0x7f9980019500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800b2040 con 0x7f99881605a0 2024-05-17T21:58:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800bf070 con 0x7f99881605a0 2024-05-17T21:58:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800b8870 con 0x7f99881605a0 2024-05-17T21:58:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880b4750 con 0x7f99881605a0 2024-05-17T21:58:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99880b4d10 con 0x7f99881605a0 2024-05-17T21:58:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99881609b0 con 0x7f99881605a0 2024-05-17T21:58:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99800b8a10 con 0x7f99881605a0 2024-05-17T21:58:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 0x7f996c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f998012a020 con 0x7f99881605a0 2024-05-17T21:58:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.365+0000 7f998d716640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 0x7f996c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.369+0000 7f998d716640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 0x7f996c071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99880b11f0 tx=0x7f9984007a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.369+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9988160ef0 con 0x7f99881605a0 2024-05-17T21:58:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f99880b41a0 con 0x7f99881605a0 2024-05-17T21:58:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f997dffb640 1 -- 172.21.15.83:0/1257298044 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f99800f72f0 con 0x7f99881605a0 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:58:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 msgr2=0x7f996c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 0x7f996c071910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99880b11f0 tx=0x7f9984007a50 comp rx=0 tx=0).stop 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 msgr2=0x7f998813f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f99800b0040 tx=0x7f9980019500 comp rx=0 tx=0).stop 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 shutdown_connections 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99881605a0 0x7f998813f6d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f996c06f400 0x7f996c071910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99881461c0 0x7f998813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 --2- 172.21.15.83:0/1257298044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f998806ffd0 0x7f998813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 >> 172.21.15.83:0/1257298044 conn(0x7f998806cd90 msgr2=0x7f9988143c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 shutdown_connections 2024-05-17T21:58:57.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.537+0000 7f998f97a640 1 -- 172.21.15.83:0/1257298044 wait complete. 2024-05-17T21:58:57.553 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T21:58:57.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T21:58:57.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.713+0000 7f7624469640 1 Processor -- start 2024-05-17T21:58:57.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.713+0000 7f7624469640 1 -- start start 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7624469640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f761c160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7624469640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c160d70 con 0x7f761c1570e0 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7624469640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c0d0820 con 0x7f761c0b4f20 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7624469640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f761c0b4a90 con 0x7f761c160420 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f761c1570e0 0x7f761c0b3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41598/0 (socket says 172.21.15.83:41598) 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 -- 172.21.15.83:0/1428807084 learned_addr learned my addr 172.21.15.83:0/1428807084 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320943399 0 0) 0x7f7618001120 con 0x7f761c1570e0 2024-05-17T21:58:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621a04640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f761c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7608003530 con 0x7f761c1570e0 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3561741377 0 0) 0x7f7608003530 con 0x7f761c1570e0 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 msgr2=0x7f761c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f761c160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 >> v1:172.21.15.155:6789/0 conn(0x7f761c0b4f20 legacy=0x7f761c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c0d0b00 con 0x7f761c1570e0 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f76180018d0 con 0x7f761c1570e0 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 >> v1:172.21.15.83:6789/0 conn(0x7f761c1570e0 legacy=0x7f761c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f7608003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f7608004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7608004d50 0x7f7608005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f761c0b4a90 con 0x7f7608003a40 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f760800bb90 con 0x7f7608004d50 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f760800bd90 con 0x7f761c160420 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f7608003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7621a04640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f7608004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622a06640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7608004d50 0x7f7608005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 -- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 msgr2=0x7f7608004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c160420 0x7f7608004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 -- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7608004d50 msgr2=0x7f7608005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7608004d50 0x7f7608005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 -- 172.21.15.83:0/1428807084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7608003530 con 0x7f7608003a40 2024-05-17T21:58:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.717+0000 7f7622205640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f7608003eb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f761800c4c0 tx=0x7f761801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9bdde5539cacea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800c0e0 con 0x7f7608003a40 2024-05-17T21:58:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180a9020 con 0x7f7608003a40 2024-05-17T21:58:57.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7621203640 1 -- 172.21.15.83:0/1428807084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0b6250 con 0x7f7608003a40 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- 172.21.15.83:0/1428807084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 msgr2=0x7f7608003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 --2- 172.21.15.83:0/1428807084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f7608003eb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f761800c4c0 tx=0x7f761801cbe0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7622a06640 1 -- 172.21.15.83:0/1428807084 reap_dead start 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- 172.21.15.83:0/1428807084 shutdown_connections 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- 172.21.15.83:0/1428807084 >> 172.21.15.83:0/1428807084 conn(0x7f761c06cd90 msgr2=0x7f761c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- 172.21.15.83:0/1428807084 shutdown_connections 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- 172.21.15.83:0/1428807084 wait complete. 2024-05-17T21:58:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 Processor -- start 2024-05-17T21:58:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.721+0000 7f7624469640 1 -- start start 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c0b1010 0x7f761c166760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 0x7f761c072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1685b0 con 0x7f7608003a40 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f761c0d05c0 con 0x7f761c0b1010 2024-05-17T21:58:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f761c168730 con 0x7f761c143e00 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60288/0 (socket says 172.21.15.83:60288) 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 -- 172.21.15.83:0/1179301955 learned_addr learned my addr 172.21.15.83:0/1179301955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622a06640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 0x7f761c072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 -- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 msgr2=0x7f761c072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 0x7f761c072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7621a04640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c0b1010 0x7f761c166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 -- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c0b1010 msgr2=0x7f761c166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c0b1010 0x7f761c166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 -- 172.21.15.83:0/1179301955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c1656c0 con 0x7f7608003a40 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622a06640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 0x7f761c072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7622205640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7618000c10 tx=0x7f7608005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618000ee0 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618001120 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618001890 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c1659a0 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c165f60 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c0b5330 con 0x7f7608003a40 2024-05-17T21:58:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76180a5760 con 0x7f7608003a40 2024-05-17T21:58:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 0x7f7600071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7621a04640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 0x7f7600071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f76181283e0 con 0x7f7608003a40 2024-05-17T21:58:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.725+0000 7f7621a04640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 0x7f7600071ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f760c009160 tx=0x7f760c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:57.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.729+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f76180b3070 con 0x7f7608003a40 2024-05-17T21:58:57.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:57.893+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f761c163150 con 0x7f7608003a40 2024-05-17T21:58:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.769+0000 7f7612ffd640 1 -- 172.21.15.83:0/1179301955 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f761c160d70 con 0x7f7608003a40 2024-05-17T21:58:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T21:58:58.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 msgr2=0x7f7600071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:58.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 0x7f7600071ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f760c009160 tx=0x7f760c008040 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 msgr2=0x7f761c0b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7618000c10 tx=0x7f7608005840 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 shutdown_connections 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f761c143e00 0x7f761c072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f760006f5a0 0x7f7600071ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c0b1010 0x7f761c166760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 --2- 172.21.15.83:0/1179301955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608003a40 0x7f761c0b2ac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 >> 172.21.15.83:0/1179301955 conn(0x7f761c06cd90 msgr2=0x7f761c15f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 shutdown_connections 2024-05-17T21:58:58.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.773+0000 7f7624469640 1 -- 172.21.15.83:0/1179301955 wait complete. 2024-05-17T21:58:58.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T21:58:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 Processor -- start 2024-05-17T21:58:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 -- start start 2024-05-17T21:58:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb87009bcc0 0x7fb8700b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb870161520 0x7fb870161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8700b0ae0 0x7fb8700b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb8700b60c0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb8700b5de0 con 0x7fb87009bcc0 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb8761c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8700b0000 con 0x7fb870161520 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb86ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8700b0ae0 0x7fb8700b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb86ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8700b0ae0 0x7fb8700b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60296/0 (socket says 172.21.15.83:60296) 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb86ffff640 1 -- 172.21.15.83:0/1651383768 learned_addr learned my addr 172.21.15.83:0/1651383768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.961+0000 7fb86f7fe640 1 --2- 172.21.15.83:0/1651383768 >> v2:172.21.15.155:3300/0 conn(0x7fb87009bcc0 0x7fb8700b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86effd640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb870161520 0x7fb870161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 -- 172.21.15.83:0/1651383768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb870161520 msgr2=0x7fb870161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb870161520 0x7fb870161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 -- 172.21.15.83:0/1651383768 >> v2:172.21.15.155:3300/0 conn(0x7fb87009bcc0 msgr2=0x7fb8700b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 --2- 172.21.15.83:0/1651383768 >> v2:172.21.15.155:3300/0 conn(0x7fb87009bcc0 0x7fb8700b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 -- 172.21.15.83:0/1651383768 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8700b02e0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86ffff640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb8700b0ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb87006f9b0 tx=0x7fb86801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6332c504a77feecc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86e7fc640 1 -- 172.21.15.83:0/1651383768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680a5690 con 0x7fb8700b0ae0 2024-05-17T21:58:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86e7fc640 1 -- 172.21.15.83:0/1651383768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8680a5830 con 0x7fb8700b0ae0 2024-05-17T21:58:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb86e7fc640 1 -- 172.21.15.83:0/1651383768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680a5c00 con 0x7fb8700b0ae0 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 -- 172.21.15.83:0/1651383768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 msgr2=0x7fb8700b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb8700b0ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb87006f9b0 tx=0x7fb86801c9f0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 -- 172.21.15.83:0/1651383768 shutdown_connections 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb8700b0ef0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 --2- 172.21.15.83:0/1651383768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb870161520 0x7fb870161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 --2- 172.21.15.83:0/1651383768 >> v2:172.21.15.155:3300/0 conn(0x7fb87009bcc0 0x7fb8700b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 -- 172.21.15.83:0/1651383768 >> 172.21.15.83:0/1651383768 conn(0x7fb87009b8b0 msgr2=0x7fb8700b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 -- 172.21.15.83:0/1651383768 shutdown_connections 2024-05-17T21:58:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.965+0000 7fb8761c0640 1 -- 172.21.15.83:0/1651383768 wait complete. 2024-05-17T21:58:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 Processor -- start 2024-05-17T21:58:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 -- start start 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87009bcc0 0x7fb8700b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb870161520 0x7fb8701496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb870160240 con 0x7fb8700b0ae0 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701600c0 con 0x7fb870161520 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb8761c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb8701603c0 con 0x7fb87009bcc0 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb86effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb86effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60310/0 (socket says 172.21.15.83:60310) 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb86effd640 1 -- 172.21.15.83:0/407492410 learned_addr learned my addr 172.21.15.83:0/407492410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb86ffff640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb870161520 0x7fb8701496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.969+0000 7fb86f7fe640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87009bcc0 0x7fb8700b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 -- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87009bcc0 msgr2=0x7fb8700b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87009bcc0 0x7fb8700b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 -- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb870161520 msgr2=0x7fb8701496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb870161520 0x7fb8701496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 -- 172.21.15.83:0/407492410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87015a4f0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86effd640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d5f0 tx=0x7fb85800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580125f0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb858019070 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87015a7d0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580129b0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87015ad90 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8700b5500 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb858012b50 con 0x7fb8700b0ae0 2024-05-17T21:58:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 0x7fb854071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:58:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7fb85808f4d0 con 0x7fb8700b0ae0 2024-05-17T21:58:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86f7fe640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 0x7fb854071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:58:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.973+0000 7fb86f7fe640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 0x7fb854071910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b990 tx=0x7fb860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:58:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:58.977+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb85805c630 con 0x7fb8700b0ae0 2024-05-17T21:58:59.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.141+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb870159900 con 0x7fb8700b0ae0 2024-05-17T21:58:59.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:58:59.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:58:59.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb867fff640 1 -- 172.21.15.83:0/407492410 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7fb85805bf30 con 0x7fb8700b0ae0 2024-05-17T21:58:59.773 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 msgr2=0x7fb854071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 0x7fb854071910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb86000b990 tx=0x7fb860008040 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 msgr2=0x7fb870147090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb85800d5f0 tx=0x7fb85800dab0 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 shutdown_connections 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb85406f400 0x7fb854071910 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb870161520 0x7fb8701496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8700b0ae0 0x7fb870147090 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 --2- 172.21.15.83:0/407492410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87009bcc0 0x7fb8700b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:58:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 >> 172.21.15.83:0/407492410 conn(0x7fb87009b8b0 msgr2=0x7fb8700d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:58:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 shutdown_connections 2024-05-17T21:58:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:58:59.781+0000 7fb8761c0640 1 -- 172.21.15.83:0/407492410 wait complete. 2024-05-17T21:58:59.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T21:59:00.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 Processor -- start 2024-05-17T21:59:00.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 -- start start 2024-05-17T21:59:00.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e480b7430 0x7f1e4815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 0x7f1e480b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480af4f0 con 0x7f1e480b59b0 2024-05-17T21:59:00.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1e480afb00 con 0x7f1e480b7430 2024-05-17T21:59:00.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e4df21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e480b1710 con 0x7f1e4809bcc0 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e480b7430 0x7f1e4815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e480b7430 0x7f1e4815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60314/0 (socket says 172.21.15.83:60314) 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 -- 172.21.15.83:0/1891042035 learned_addr learned my addr 172.21.15.83:0/1891042035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e46ffd640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 0x7f1e480b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 -- 172.21.15.83:0/1891042035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 msgr2=0x7f1e480b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 0x7f1e480b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 -- 172.21.15.83:0/1891042035 >> v1:172.21.15.155:6789/0 conn(0x7f1e480b59b0 legacy=0x7f1e48163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e477fe640 1 -- 172.21.15.83:0/1891042035 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e480af790 con 0x7f1e480b7430 2024-05-17T21:59:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.021+0000 7f1e46ffd640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 0x7f1e480b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e477fe640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e480b7430 0x7f1e4815fc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806f990 tx=0x7f1e4001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d607027b745f7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e467fc640 1 -- 172.21.15.83:0/1891042035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b0070 con 0x7f1e480b7430 2024-05-17T21:59:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e467fc640 1 -- 172.21.15.83:0/1891042035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e400ab600 con 0x7f1e480b7430 2024-05-17T21:59:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e467fc640 1 -- 172.21.15.83:0/1891042035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ab9d0 con 0x7f1e480b7430 2024-05-17T21:59:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 -- 172.21.15.83:0/1891042035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e480b7430 msgr2=0x7f1e4815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e480b7430 0x7f1e4815fc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806f990 tx=0x7f1e4001ce80 comp rx=0 tx=0).stop 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 -- 172.21.15.83:0/1891042035 shutdown_connections 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4809bcc0 0x7f1e480b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1891042035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e480b7430 0x7f1e4815fc60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 -- 172.21.15.83:0/1891042035 >> 172.21.15.83:0/1891042035 conn(0x7f1e4809b8b0 msgr2=0x7f1e480b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 -- 172.21.15.83:0/1891042035 shutdown_connections 2024-05-17T21:59:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.025+0000 7f1e4df21640 1 -- 172.21.15.83:0/1891042035 wait complete. 2024-05-17T21:59:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 Processor -- start 2024-05-17T21:59:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- start start 2024-05-17T21:59:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e480b59b0 0x7f1e480b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e480b7430 0x7f1e480b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480b1120 con 0x7f1e4809bcc0 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480b0fa0 con 0x7f1e480b7430 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4816c970 con 0x7f1e480b59b0 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60322/0 (socket says 172.21.15.83:60322) 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 -- 172.21.15.83:0/1779979963 learned_addr learned my addr 172.21.15.83:0/1779979963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e46ffd640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e480b59b0 0x7f1e480b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 -- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e480b59b0 msgr2=0x7f1e480b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e480b59b0 0x7f1e480b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 -- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e480b7430 msgr2=0x7f1e480b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e480b7430 0x7f1e480b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 -- 172.21.15.83:0/1779979963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4815da20 con 0x7f1e4809bcc0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e477fe640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c10 tx=0x7f1e4001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ba020 con 0x7f1e4809bcc0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e400b6640 con 0x7f1e4809bcc0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e48160de0 con 0x7f1e4809bcc0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400b0040 con 0x7f1e4809bcc0 2024-05-17T21:59:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.029+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481613a0 con 0x7f1e4809bcc0 2024-05-17T21:59:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e480b5470 con 0x7f1e4809bcc0 2024-05-17T21:59:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e400bd020 con 0x7f1e4809bcc0 2024-05-17T21:59:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e3ffff640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 0x7f1e28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7f1e40129ed0 con 0x7f1e4809bcc0 2024-05-17T21:59:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e46ffd640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 0x7f1e28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.033+0000 7f1e46ffd640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 0x7f1e28071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e480d1be0 tx=0x7f1e38007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.037+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1e400c4030 con 0x7f1e4809bcc0 2024-05-17T21:59:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.237+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1e48158990 con 0x7f1e4809bcc0 2024-05-17T21:59:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.781+0000 7f1e3ffff640 1 -- 172.21.15.83:0/1779979963 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f1e40016130 con 0x7f1e4809bcc0 2024-05-17T21:59:00.777 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 6 and data pool 7 2024-05-17T21:59:00.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 msgr2=0x7f1e28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 0x7f1e28071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e480d1be0 tx=0x7f1e38007040 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 msgr2=0x7f1e48162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000c10 tx=0x7f1e4001ac60 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 shutdown_connections 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e2806f400 0x7f1e28071910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e480b7430 0x7f1e480b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e480b59b0 0x7f1e480b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 --2- 172.21.15.83:0/1779979963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e4809bcc0 0x7f1e48162800 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 >> 172.21.15.83:0/1779979963 conn(0x7f1e4809b8b0 msgr2=0x7f1e48071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.785+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 shutdown_connections 2024-05-17T21:59:00.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.789+0000 7f1e4df21640 1 -- 172.21.15.83:0/1779979963 wait complete. 2024-05-17T21:59:00.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:00.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 Processor -- start 2024-05-17T21:59:00.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 -- start start 2024-05-17T21:59:00.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8f00d0350 0x7fc8f00b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f00b6b10 0x7fc8f006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f00d1680 con 0x7fc8f015e9c0 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f00d1960 con 0x7fc8f00d0350 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f6f95640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00cf870 con 0x7fc8f00b6b10 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f4d31640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc8f015e9c0 0x7fc8f00b1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41614/0 (socket says 172.21.15.83:41614) 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f4d31640 1 -- 172.21.15.83:0/2078122617 learned_addr learned my addr 172.21.15.83:0/2078122617 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8effff640 1 --2- 172.21.15.83:0/2078122617 >> v2:172.21.15.83:3300/0 conn(0x7fc8f00d0350 0x7fc8f00b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8ef7fe640 1 -- 172.21.15.83:0/2078122617 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927290284 0 0) 0x7fc8f00d1680 con 0x7fc8f015e9c0 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8ef7fe640 1 -- 172.21.15.83:0/2078122617 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8e0003530 con 0x7fc8f015e9c0 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8f5532640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f00b6b10 0x7fc8f006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8effff640 1 -- 172.21.15.83:0/2078122617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f00b6b10 msgr2=0x7fc8f006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8effff640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f00b6b10 0x7fc8f006c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8effff640 1 -- 172.21.15.83:0/2078122617 >> v1:172.21.15.83:6789/0 conn(0x7fc8f015e9c0 legacy=0x7fc8f00b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.981+0000 7fc8effff640 1 -- 172.21.15.83:0/2078122617 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f00cfb50 con 0x7fc8f00d0350 2024-05-17T21:59:00.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8effff640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00d0350 0x7fc8f00b65d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097c0 tx=0x7fc8dc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac9ca5a80e337fa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8ef7fe640 1 -- 172.21.15.83:0/2078122617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0b0070 con 0x7fc8f00d0350 2024-05-17T21:59:00.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8ef7fe640 1 -- 172.21.15.83:0/2078122617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0ab600 con 0x7fc8f00d0350 2024-05-17T21:59:00.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8ef7fe640 1 -- 172.21.15.83:0/2078122617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0ab9d0 con 0x7fc8f00d0350 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- 172.21.15.83:0/2078122617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00d0350 msgr2=0x7fc8f00b65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00d0350 0x7fc8f00b65d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097c0 tx=0x7fc8dc01c3e0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- 172.21.15.83:0/2078122617 shutdown_connections 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f00b6b10 0x7fc8f006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/2078122617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00d0350 0x7fc8f00b65d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- 172.21.15.83:0/2078122617 >> 172.21.15.83:0/2078122617 conn(0x7fc8f00d1ca0 msgr2=0x7fc8f00d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- 172.21.15.83:0/2078122617 shutdown_connections 2024-05-17T21:59:00.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- 172.21.15.83:0/2078122617 wait complete. 2024-05-17T21:59:00.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 Processor -- start 2024-05-17T21:59:00.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.985+0000 7fc8f6f95640 1 -- start start 2024-05-17T21:59:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f00d0350 0x7fc8f0154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f015e9c0 0x7fc8f01f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00d1420 con 0x7fc8f00b6b10 2024-05-17T21:59:00.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00d12a0 con 0x7fc8f00d0350 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00764e0 con 0x7fc8f015e9c0 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60336/0 (socket says 172.21.15.83:60336) 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 -- 172.21.15.83:0/145728233 learned_addr learned my addr 172.21.15.83:0/145728233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 -- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f015e9c0 msgr2=0x7fc8f01f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f015e9c0 0x7fc8f01f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 -- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f00d0350 msgr2=0x7fc8f0154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f00d0350 0x7fc8f0154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 -- 172.21.15.83:0/145728233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f009e060 con 0x7fc8f00b6b10 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f4d31640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c460 tx=0x7fc8e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4017070 con 0x7fc8f00b6b10 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e4005cf0 con 0x7fc8f00b6b10 2024-05-17T21:59:00.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007270 con 0x7fc8f00b6b10 2024-05-17T21:59:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f009e340 con 0x7fc8f00b6b10 2024-05-17T21:59:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f009e900 con 0x7fc8f00b6b10 2024-05-17T21:59:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f00b5cf0 con 0x7fc8f00b6b10 2024-05-17T21:59:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8e401a070 con 0x7fc8f00b6b10 2024-05-17T21:59:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 0x7fc8d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8effff640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 0x7fc8d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.989+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fc8e408e3e0 con 0x7fc8f00b6b10 2024-05-17T21:59:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.993+0000 7fc8effff640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 0x7fc8d4071910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:00.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:00.997+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc8f00b5cf0 con 0x7fc8f00b6b10 2024-05-17T21:59:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.153+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8f00d0760 con 0x7fc8f00b6b10 2024-05-17T21:59:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.153+0000 7fc8ed7fa640 1 -- 172.21.15.83:0/145728233 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16602 (secure 0 0 0) 0x7fc8e405ad10 con 0x7fc8f00b6b10 2024-05-17T21:59:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:01.148 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":42,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:00.240011+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:57.908827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:59.153525+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:01.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 msgr2=0x7fc8d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 0x7fc8d4071910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0ad000 tx=0x7fc8dc008040 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 msgr2=0x7fc8f009d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c460 tx=0x7fc8e400c920 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 shutdown_connections 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f015e9c0 0x7fc8f01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8d406f400 0x7fc8d4071910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f00d0350 0x7fc8f0154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 --2- 172.21.15.83:0/145728233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f00b6b10 0x7fc8f009d760 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 >> 172.21.15.83:0/145728233 conn(0x7fc8f00d1ca0 msgr2=0x7fc8f00afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 shutdown_connections 2024-05-17T21:59:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.157+0000 7fc8f6f95640 1 -- 172.21.15.83:0/145728233 wait complete. 2024-05-17T21:59:01.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 Processor -- start 2024-05-17T21:59:01.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 -- start start 2024-05-17T21:59:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d780b8b40 0x7f4d780b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 0x7f4d7809bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7809c2e0 con 0x7f4d780b82e0 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7809d0b0 con 0x7f4d780b8b40 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d7da01640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4d780ce1e0 con 0x7f4d7809b990 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d76ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d780b8b40 0x7f4d780b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d6ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4d780b82e0 0x7f4d7809c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41624/0 (socket says 172.21.15.83:41624) 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d6ffff640 1 -- 172.21.15.83:0/52993098 learned_addr learned my addr 172.21.15.83:0/52993098 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d777fe640 1 --2- 172.21.15.83:0/52993098 >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 0x7f4d7809bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d76ffd640 1 -- 172.21.15.83:0/52993098 >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 msgr2=0x7f4d7809bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d76ffd640 1 --2- 172.21.15.83:0/52993098 >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 0x7f4d7809bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d76ffd640 1 -- 172.21.15.83:0/52993098 >> v1:172.21.15.83:6789/0 conn(0x7f4d780b82e0 legacy=0x7f4d7809c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d76ffd640 1 -- 172.21.15.83:0/52993098 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d780ce4c0 con 0x7f4d780b8b40 2024-05-17T21:59:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.305+0000 7f4d777fe640 1 --2- 172.21.15.83:0/52993098 >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 0x7f4d7809bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d76ffd640 1 --2- 172.21.15.83:0/52993098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780b64b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d78074bf0 tx=0x7f4d6801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f7dc2a8751c50e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d767fc640 1 -- 172.21.15.83:0/52993098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680b0070 con 0x7f4d780b8b40 2024-05-17T21:59:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d767fc640 1 -- 172.21.15.83:0/52993098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d680ab600 con 0x7f4d780b8b40 2024-05-17T21:59:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d767fc640 1 -- 172.21.15.83:0/52993098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680ab9d0 con 0x7f4d780b8b40 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 -- 172.21.15.83:0/52993098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 msgr2=0x7f4d780b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 --2- 172.21.15.83:0/52993098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780b64b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d78074bf0 tx=0x7f4d6801c480 comp rx=0 tx=0).stop 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 -- 172.21.15.83:0/52993098 shutdown_connections 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 --2- 172.21.15.83:0/52993098 >> v2:172.21.15.155:3300/0 conn(0x7f4d7809b990 0x7f4d7809bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 --2- 172.21.15.83:0/52993098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780b64b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 -- 172.21.15.83:0/52993098 >> 172.21.15.83:0/52993098 conn(0x7f4d780d3660 msgr2=0x7f4d780d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 -- 172.21.15.83:0/52993098 shutdown_connections 2024-05-17T21:59:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.309+0000 7f4d7da01640 1 -- 172.21.15.83:0/52993098 wait complete. 2024-05-17T21:59:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 Processor -- start 2024-05-17T21:59:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 -- start start 2024-05-17T21:59:01.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d7809b990 0x7f4d7815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 0x7f4d780b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780cfa90 con 0x7f4d780b8b40 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780cfc10 con 0x7f4d780b82e0 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d7da01640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4d780cfd90 con 0x7f4d7809b990 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60354/0 (socket says 172.21.15.83:60354) 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 -- 172.21.15.83:0/2634466217 learned_addr learned my addr 172.21.15.83:0/2634466217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d76ffd640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d7809b990 0x7f4d7815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d6ffff640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 0x7f4d780b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 -- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d7809b990 msgr2=0x7f4d7815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d7809b990 0x7f4d7815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 -- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 msgr2=0x7f4d780b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 0x7f4d780b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 -- 172.21.15.83:0/2634466217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d78162b70 con 0x7f4d780b8b40 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d6ffff640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 0x7f4d780b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d777fe640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000cdb0 tx=0x7f4d7000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000f680 con 0x7f4d780b8b40 2024-05-17T21:59:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7000f840 con 0x7f4d780b8b40 2024-05-17T21:59:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.313+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000fc30 con 0x7f4d780b8b40 2024-05-17T21:59:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d78162e50 con 0x7f4d780b8b40 2024-05-17T21:59:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7815d8d0 con 0x7f4d780b8b40 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7809bda0 con 0x7f4d780b8b40 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4d7000e3e0 con 0x7f4d780b8b40 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d74ff9640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 0x7f4d58071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f4d7008ffc0 con 0x7f4d780b8b40 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d76ffd640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 0x7f4d58071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.317+0000 7f4d76ffd640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 0x7f4d58071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ad000 tx=0x7f4d680a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.321+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4d70014030 con 0x7f4d780b8b40 2024-05-17T21:59:01.380 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T21:59:01.389+0000 7ff47d4ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:59:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.485+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d7815dbe0 con 0x7f4d780b8b40 2024-05-17T21:59:01.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.485+0000 7f4d74ff9640 1 -- 172.21.15.83:0/2634466217 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16602 (secure 0 0 0) 0x7f4d7005cf40 con 0x7f4d780b8b40 2024-05-17T21:59:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:01.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":42,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:00.240011+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:57.908827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:59.153525+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 msgr2=0x7f4d58071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 0x7f4d58071910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ad000 tx=0x7f4d680a6000 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 msgr2=0x7f4d780a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000cdb0 tx=0x7f4d7000a650 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 shutdown_connections 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d780b8b40 0x7f4d780a07e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4d5806f400 0x7f4d58071910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d780b82e0 0x7f4d780b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 --2- 172.21.15.83:0/2634466217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d7809b990 0x7f4d7815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 >> 172.21.15.83:0/2634466217 conn(0x7f4d780d3660 msgr2=0x7f4d78071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 shutdown_connections 2024-05-17T21:59:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.489+0000 7f4d7da01640 1 -- 172.21.15.83:0/2634466217 wait complete. 2024-05-17T21:59:01.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T21:59:01.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.657+0000 7f5b86426640 1 Processor -- start 2024-05-17T21:59:01.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.657+0000 7f5b86426640 1 -- start start 2024-05-17T21:59:01.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b86426640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 0x7f5b800b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b86426640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b800cdf40 0x7f5b8015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b86426640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b800b5640 con 0x7f5b8015d530 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b86426640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5b800b58e0 con 0x7f5b800cdf40 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b86426640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8006f060 con 0x7f5b800cf830 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b849c3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5b8015d530 0x7f5b8006f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41638/0 (socket says 172.21.15.83:41638) 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b849c3640 1 -- 172.21.15.83:0/1812800683 learned_addr learned my addr 172.21.15.83:0/1812800683 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1812800683 >> v2:172.21.15.83:3300/0 conn(0x7f5b800cdf40 0x7f5b8015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100208602 0 0) 0x7f5b800b5640 con 0x7f5b8015d530 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b7ffff640 1 --2- 172.21.15.83:0/1812800683 >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 0x7f5b800b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b6c005740 con 0x7f5b8015d530 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b777fe640 1 -- 172.21.15.83:0/1812800683 >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 msgr2=0x7f5b800b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1812800683 >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 0x7f5b800b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3260348532 0 0) 0x7f5b6c005740 con 0x7f5b8015d530 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b777fe640 1 -- 172.21.15.83:0/1812800683 >> v1:172.21.15.83:6789/0 conn(0x7f5b8015d530 legacy=0x7f5b8006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b777fe640 1 -- 172.21.15.83:0/1812800683 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8006f340 con 0x7f5b800cdf40 2024-05-17T21:59:01.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.661+0000 7f5b7ffff640 1 --2- 172.21.15.83:0/1812800683 >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 0x7f5b800b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:01.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1812800683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b8015cff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b8006d6e0 tx=0x7f5b6801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87a67a6f66f35b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b680b0070 con 0x7f5b800cdf40 2024-05-17T21:59:01.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b680ab400 con 0x7f5b800cdf40 2024-05-17T21:59:01.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b7f7fe640 1 -- 172.21.15.83:0/1812800683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78001aa0 con 0x7f5b800cdf40 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- 172.21.15.83:0/1812800683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 msgr2=0x7f5b8015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 --2- 172.21.15.83:0/1812800683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b8015cff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b8006d6e0 tx=0x7f5b6801c3a0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- 172.21.15.83:0/1812800683 shutdown_connections 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 --2- 172.21.15.83:0/1812800683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b8015cff0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 --2- 172.21.15.83:0/1812800683 >> v2:172.21.15.155:3300/0 conn(0x7f5b800cf830 0x7f5b800b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- 172.21.15.83:0/1812800683 >> 172.21.15.83:0/1812800683 conn(0x7f5b8006cd90 msgr2=0x7f5b800b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- 172.21.15.83:0/1812800683 shutdown_connections 2024-05-17T21:59:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- 172.21.15.83:0/1812800683 wait complete. 2024-05-17T21:59:01.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 Processor -- start 2024-05-17T21:59:01.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.665+0000 7f5b86426640 1 -- start start 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b800cf830 0x7f5b800b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b8015d530 0x7f5b800b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80073210 con 0x7f5b800cdf40 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80073090 con 0x7f5b8015d530 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5b80073390 con 0x7f5b800cf830 2024-05-17T21:59:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60368/0 (socket says 172.21.15.83:60368) 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 -- 172.21.15.83:0/1235053900 learned_addr learned my addr 172.21.15.83:0/1235053900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b800cf830 0x7f5b800b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 -- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b800cf830 msgr2=0x7f5b800b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b800cf830 0x7f5b800b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 -- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b8015d530 msgr2=0x7f5b800b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b8015d530 0x7f5b800b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 -- 172.21.15.83:0/1235053900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b80153e20 con 0x7f5b800cdf40 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7ffff640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000d830 tx=0x7f5b7000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000b6c0 con 0x7f5b800cdf40 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70019070 con 0x7f5b800cdf40 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000baa0 con 0x7f5b800cdf40 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b80158080 con 0x7f5b800cdf40 2024-05-17T21:59:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b80158640 con 0x7f5b800cdf40 2024-05-17T21:59:01.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8015cff0 con 0x7f5b800cdf40 2024-05-17T21:59:01.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b7001e460 con 0x7f5b800cdf40 2024-05-17T21:59:01.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 0x7f5b60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 0x7f5b60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.669+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f5b7008f950 con 0x7f5b800cdf40 2024-05-17T21:59:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.673+0000 7f5b777fe640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 0x7f5b60071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b680ad000 tx=0x7f5b68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.677+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5b7005c8d0 con 0x7f5b800cdf40 2024-05-17T21:59:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:01.881+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5b80073b90 con 0x7f5b800cdf40 2024-05-17T21:59:02.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.817+0000 7f5b7d7fa640 1 -- 172.21.15.83:0/1235053900 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f5b7005c1d0 con 0x7f5b800cdf40 2024-05-17T21:59:02.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 msgr2=0x7f5b60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 0x7f5b60071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b680ad000 tx=0x7f5b68008040 comp rx=0 tx=0).stop 2024-05-17T21:59:02.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 msgr2=0x7f5b801535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:02.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000d830 tx=0x7f5b7000dcf0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 shutdown_connections 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5b6006f400 0x7f5b60071910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b8015d530 0x7f5b800b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b800cf830 0x7f5b800b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 --2- 172.21.15.83:0/1235053900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b800cdf40 0x7f5b801535d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 >> 172.21.15.83:0/1235053900 conn(0x7f5b8006cd90 msgr2=0x7f5b800cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 shutdown_connections 2024-05-17T21:59:02.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:02.821+0000 7f5b86426640 1 -- 172.21.15.83:0/1235053900 wait complete. 2024-05-17T21:59:02.838 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T21:59:02.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T21:59:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.001+0000 7f3b687fc640 1 Processor -- start 2024-05-17T21:59:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.001+0000 7f3b687fc640 1 -- start start 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b687fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b600d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b687fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b600d2000 0x7f3b600d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b687fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b600d4170 con 0x7f3b600b4810 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b687fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3b600d5620 con 0x7f3b600d2000 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b687fc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b600d4450 con 0x7f3b600d5210 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66598640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3b600b4810 0x7f3b6015fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41654/0 (socket says 172.21.15.83:41654) 2024-05-17T21:59:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66598640 1 -- 172.21.15.83:0/1967428042 learned_addr learned my addr 172.21.15.83:0/1967428042 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66d99640 1 --2- 172.21.15.83:0/1967428042 >> v2:172.21.15.83:3300/0 conn(0x7f3b600d2000 0x7f3b600d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b65596640 1 -- 172.21.15.83:0/1967428042 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030661903 0 0) 0x7f3b600d4170 con 0x7f3b600b4810 2024-05-17T21:59:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b65d97640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b600d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b65596640 1 -- 172.21.15.83:0/1967428042 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b4c003530 con 0x7f3b600b4810 2024-05-17T21:59:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66d99640 1 -- 172.21.15.83:0/1967428042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 msgr2=0x7f3b600d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66d99640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b600d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66d99640 1 -- 172.21.15.83:0/1967428042 >> v1:172.21.15.83:6789/0 conn(0x7f3b600b4810 legacy=0x7f3b6015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.005+0000 7f3b66d99640 1 -- 172.21.15.83:0/1967428042 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600d4730 con 0x7f3b600d2000 2024-05-17T21:59:03.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b66d99640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600d2000 0x7f3b600d2410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3b580097c0 tx=0x7f3b5801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52b54e54afb65fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:03.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b65596640 1 -- 172.21.15.83:0/1967428042 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b580aa620 con 0x7f3b600d2000 2024-05-17T21:59:03.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b65596640 1 -- 172.21.15.83:0/1967428042 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b580af070 con 0x7f3b600d2000 2024-05-17T21:59:03.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b65596640 1 -- 172.21.15.83:0/1967428042 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b580aa9e0 con 0x7f3b600d2000 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- 172.21.15.83:0/1967428042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600d2000 msgr2=0x7f3b600d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600d2000 0x7f3b600d2410 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3b580097c0 tx=0x7f3b5801c4a0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- 172.21.15.83:0/1967428042 shutdown_connections 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600d2000 0x7f3b600d2410 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 --2- 172.21.15.83:0/1967428042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b600d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- 172.21.15.83:0/1967428042 >> 172.21.15.83:0/1967428042 conn(0x7f3b6006e010 msgr2=0x7f3b6006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- 172.21.15.83:0/1967428042 shutdown_connections 2024-05-17T21:59:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- 172.21.15.83:0/1967428042 wait complete. 2024-05-17T21:59:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 Processor -- start 2024-05-17T21:59:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.009+0000 7f3b687fc640 1 -- start start 2024-05-17T21:59:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b600d2000 0x7f3b60154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b601f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b9670 con 0x7f3b600b4810 2024-05-17T21:59:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b94f0 con 0x7f3b600d2000 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3b600b97f0 con 0x7f3b600d5210 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60380/0 (socket says 172.21.15.83:60380) 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 -- 172.21.15.83:0/3850280899 learned_addr learned my addr 172.21.15.83:0/3850280899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 -- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 msgr2=0x7f3b601f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b601f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 -- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b600d2000 msgr2=0x7f3b60154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b65d97640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b600d2000 0x7f3b60154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b600d2000 0x7f3b60154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 -- 172.21.15.83:0/3850280899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b60162250 con 0x7f3b600b4810 2024-05-17T21:59:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b66598640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00cbc0 tx=0x7f3b5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00ad40 con 0x7f3b600b4810 2024-05-17T21:59:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c012400 con 0x7f3b600b4810 2024-05-17T21:59:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b60158a50 con 0x7f3b600b4810 2024-05-17T21:59:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0127b0 con 0x7f3b600b4810 2024-05-17T21:59:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6015a2d0 con 0x7f3b600b4810 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b600d2410 con 0x7f3b600b4810 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b5c012950 con 0x7f3b600b4810 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.013+0000 7f3b56ffd640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 0x7f3b44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.017+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3b5c08f8d0 con 0x7f3b600b4810 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.017+0000 7f3b65d97640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 0x7f3b44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.017+0000 7f3b65d97640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 0x7f3b44071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b601601f0 tx=0x7f3b50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.021+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3b5c05c7c0 con 0x7f3b600b4810 2024-05-17T21:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.213+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3b60155010 con 0x7f3b600b4810 2024-05-17T21:59:03.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b56ffd640 1 -- 172.21.15.83:0/3850280899 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f3b5c05c0c0 con 0x7f3b600b4810 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 msgr2=0x7f3b44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 0x7f3b44071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b601601f0 tx=0x7f3b50008040 comp rx=0 tx=0).stop 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 msgr2=0x7f3b60158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00cbc0 tx=0x7f3b5c00a430 comp rx=0 tx=0).stop 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 shutdown_connections 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b600d5210 0x7f3b601f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3b4406f400 0x7f3b44071910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b600d2000 0x7f3b60154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 --2- 172.21.15.83:0/3850280899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b600b4810 0x7f3b60158220 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:03.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 >> 172.21.15.83:0/3850280899 conn(0x7f3b6006e010 msgr2=0x7f3b60160bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:03.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 shutdown_connections 2024-05-17T21:59:03.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:03.829+0000 7f3b687fc640 1 -- 172.21.15.83:0/3850280899 wait complete. 2024-05-17T21:59:03.850 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T21:59:03.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:04.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 Processor -- start 2024-05-17T21:59:04.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 -- start start 2024-05-17T21:59:04.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 0x7f3cd8144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e080 0x7f3cd8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd815e490 0x7f3cd8073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd815ea40 con 0x7f3cd815e490 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd815ed20 con 0x7f3cd80d1040 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cdd48b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd815f000 con 0x7f3cd815e080 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd815e490 0x7f3cd8073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd815e490 0x7f3cd8073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60386/0 (socket says 172.21.15.83:60386) 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd77fe640 1 -- 172.21.15.83:0/2582391873 learned_addr learned my addr 172.21.15.83:0/2582391873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd67fc640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e080 0x7f3cd8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd6ffd640 1 --2- 172.21.15.83:0/2582391873 >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 0x7f3cd8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.009+0000 7f3cd77fe640 1 -- 172.21.15.83:0/2582391873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e080 msgr2=0x7f3cd8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd77fe640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e080 0x7f3cd8072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd77fe640 1 -- 172.21.15.83:0/2582391873 >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 msgr2=0x7f3cd8144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd77fe640 1 --2- 172.21.15.83:0/2582391873 >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 0x7f3cd8144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd77fe640 1 -- 172.21.15.83:0/2582391873 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd806ef30 con 0x7f3cd815e490 2024-05-17T21:59:04.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd6ffd640 1 --2- 172.21.15.83:0/2582391873 >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 0x7f3cd8144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:04.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd77fe640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e490 0x7f3cd8073370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8071d60 tx=0x7f3cc801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc26ed684ef3f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd5ffb640 1 -- 172.21.15.83:0/2582391873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80ab5f0 con 0x7f3cd815e490 2024-05-17T21:59:04.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd5ffb640 1 -- 172.21.15.83:0/2582391873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc80b0070 con 0x7f3cd815e490 2024-05-17T21:59:04.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cd5ffb640 1 -- 172.21.15.83:0/2582391873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80ab9b0 con 0x7f3cd815e490 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2582391873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e490 msgr2=0x7f3cd8073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e490 0x7f3cd8073370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8071d60 tx=0x7f3cc801c520 comp rx=0 tx=0).stop 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2582391873 shutdown_connections 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e490 0x7f3cd8073370 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2582391873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e080 0x7f3cd8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2582391873 >> v2:172.21.15.155:3300/0 conn(0x7f3cd80d1040 0x7f3cd8144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2582391873 >> 172.21.15.83:0/2582391873 conn(0x7f3cd806cd90 msgr2=0x7f3cd80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2582391873 shutdown_connections 2024-05-17T21:59:04.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.013+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2582391873 wait complete. 2024-05-17T21:59:04.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 Processor -- start 2024-05-17T21:59:04.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- start start 2024-05-17T21:59:04.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd80d1040 0x7f3cd8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 0x7f3cd8145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd81622a0 con 0x7f3cd815e080 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8162120 con 0x7f3cd80d1040 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd8162420 con 0x7f3cd815e490 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60398/0 (socket says 172.21.15.83:60398) 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 -- 172.21.15.83:0/2024568294 learned_addr learned my addr 172.21.15.83:0/2024568294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd6ffd640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd80d1040 0x7f3cd8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd77fe640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 0x7f3cd8145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 -- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 msgr2=0x7f3cd8145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 0x7f3cd8145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 -- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd80d1040 msgr2=0x7f3cd8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd80d1040 0x7f3cd8140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 -- 172.21.15.83:0/2024568294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd813cad0 con 0x7f3cd815e080 2024-05-17T21:59:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd77fe640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 0x7f3cd8145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cd67fc640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000d7d0 tx=0x7f3cc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000b6c0 con 0x7f3cd815e080 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc0019070 con 0x7f3cd815e080 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd8144770 con 0x7f3cd815e080 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000ba80 con 0x7f3cd815e080 2024-05-17T21:59:04.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.017+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd8144d30 con 0x7f3cd815e080 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8147250 con 0x7f3cd815e080 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3cc000bc20 con 0x7f3cd815e080 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3ccf7fe640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 0x7f3cb8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3cd6ffd640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 0x7f3cb8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3cc008f720 con 0x7f3cd815e080 2024-05-17T21:59:04.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.021+0000 7f3cd6ffd640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 0x7f3cb8071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0004430 tx=0x7f3cd0003a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.025+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3cc005c610 con 0x7f3cd815e080 2024-05-17T21:59:04.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.217+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cd8144080 con 0x7f3cd815e080 2024-05-17T21:59:04.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.217+0000 7f3ccf7fe640 1 -- 172.21.15.83:0/2024568294 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f3cc005bf10 con 0x7f3cd815e080 2024-05-17T21:59:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:04.212 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:04.213 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 msgr2=0x7f3cb8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 0x7f3cb8071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0004430 tx=0x7f3cd0003a00 comp rx=0 tx=0).stop 2024-05-17T21:59:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 msgr2=0x7f3cd8145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000d7d0 tx=0x7f3cc000dc90 comp rx=0 tx=0).stop 2024-05-17T21:59:04.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 shutdown_connections 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3cd815e490 0x7f3cd8145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd815e080 0x7f3cd8145530 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3cb806f400 0x7f3cb8071910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 --2- 172.21.15.83:0/2024568294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cd80d1040 0x7f3cd8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 >> 172.21.15.83:0/2024568294 conn(0x7f3cd806cd90 msgr2=0x7f3cd8143170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 shutdown_connections 2024-05-17T21:59:04.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.221+0000 7f3cdd48b640 1 -- 172.21.15.83:0/2024568294 wait complete. 2024-05-17T21:59:04.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 Processor -- start 2024-05-17T21:59:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 -- start start 2024-05-17T21:59:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b9806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b980d4260 0x7f9b980d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9806e610 con 0x7f9b9815ffa0 2024-05-17T21:59:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9b980d34e0 con 0x7f9b980d4260 2024-05-17T21:59:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b9dfce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b980d3780 con 0x7f9b9806dcc0 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b980d4260 0x7f9b980d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b980d4260 0x7f9b980d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60406/0 (socket says 172.21.15.83:60406) 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b977fe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9b9815ffa0 0x7f9b980d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:60274/0 (socket says 172.21.15.83:60274) 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b97fff640 1 -- 172.21.15.83:0/390309664 learned_addr learned my addr 172.21.15.83:0/390309664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b96ffd640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b9806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.385+0000 7f9b97fff640 1 -- 172.21.15.83:0/390309664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 msgr2=0x7f9b9806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b97fff640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b9806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b97fff640 1 -- 172.21.15.83:0/390309664 >> v1:172.21.15.155:6789/0 conn(0x7f9b9815ffa0 legacy=0x7f9b980d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b97fff640 1 -- 172.21.15.83:0/390309664 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b980d3a60 con 0x7f9b980d4260 2024-05-17T21:59:04.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b97fff640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b980d4260 0x7f9b980d4670 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009e50 tx=0x7f9b9001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52edec9fb8e0b5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b967fc640 1 -- 172.21.15.83:0/390309664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a5780 con 0x7f9b980d4260 2024-05-17T21:59:04.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b967fc640 1 -- 172.21.15.83:0/390309664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b900a5920 con 0x7f9b980d4260 2024-05-17T21:59:04.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b967fc640 1 -- 172.21.15.83:0/390309664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a5cf0 con 0x7f9b980d4260 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 -- 172.21.15.83:0/390309664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b980d4260 msgr2=0x7f9b980d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b980d4260 0x7f9b980d4670 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009e50 tx=0x7f9b9001ca30 comp rx=0 tx=0).stop 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 -- 172.21.15.83:0/390309664 shutdown_connections 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b980d4260 0x7f9b980d4670 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/390309664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b9806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 -- 172.21.15.83:0/390309664 >> 172.21.15.83:0/390309664 conn(0x7f9b980d28f0 msgr2=0x7f9b980d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 -- 172.21.15.83:0/390309664 shutdown_connections 2024-05-17T21:59:04.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.389+0000 7f9b9dfce640 1 -- 172.21.15.83:0/390309664 wait complete. 2024-05-17T21:59:04.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 Processor -- start 2024-05-17T21:59:04.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 -- start start 2024-05-17T21:59:04.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b980b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b980d4260 0x7f9b981fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b980d17e0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b980d1660 con 0x7f9b980d4260 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b9dfce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9816cb80 con 0x7f9b9806dcc0 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b97fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60408/0 (socket says 172.21.15.83:60408) 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.393+0000 7f9b97fff640 1 -- 172.21.15.83:0/3579747527 learned_addr learned my addr 172.21.15.83:0/3579747527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b977fe640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b980b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b96ffd640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b980d4260 0x7f9b981fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 -- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 msgr2=0x7f9b980b97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b980b97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 -- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b980d4260 msgr2=0x7f9b981fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b980d4260 0x7f9b981fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 -- 172.21.15.83:0/3579747527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b981605b0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b97fff640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9b900af040 tx=0x7f9b900197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90009510 con 0x7f9b9815ffa0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b900be070 con 0x7f9b9815ffa0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b980b9d10 con 0x7f9b9815ffa0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900b88b0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b980bb130 con 0x7f9b9815ffa0 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9806e0d0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b900b8a50 con 0x7f9b9815ffa0 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 0x7f9b84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b977fe640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 0x7f9b84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f9b9012a2b0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.397+0000 7f9b977fe640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 0x7f9b84071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000f70 tx=0x7f9b88007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.401+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9b90016620 con 0x7f9b9815ffa0 2024-05-17T21:59:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.553+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b980b9f40 con 0x7f9b9815ffa0 2024-05-17T21:59:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.553+0000 7f9b7ffff640 1 -- 172.21.15.83:0/3579747527 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16589 (secure 0 0 0) 0x7f9b900f71a0 con 0x7f9b9815ffa0 2024-05-17T21:59:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:04.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":43,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:01.785846+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:57.908827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:59.153525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 msgr2=0x7f9b84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 0x7f9b84071940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000f70 tx=0x7f9b88007c90 comp rx=0 tx=0).stop 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 msgr2=0x7f9b981fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9b900af040 tx=0x7f9b900197d0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 shutdown_connections 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b9815ffa0 0x7f9b981fcde0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b8406f430 0x7f9b84071940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b980d4260 0x7f9b981fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 --2- 172.21.15.83:0/3579747527 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b9806dcc0 0x7f9b980b97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 >> 172.21.15.83:0/3579747527 conn(0x7f9b980d28f0 msgr2=0x7f9b9815dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 shutdown_connections 2024-05-17T21:59:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.557+0000 7f9b9dfce640 1 -- 172.21.15.83:0/3579747527 wait complete. 2024-05-17T21:59:04.568 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T21:59:04.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:59:04.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.717+0000 7fdab3d0d640 1 Processor -- start 2024-05-17T21:59:04.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.717+0000 7fdab3d0d640 1 -- start start 2024-05-17T21:59:04.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab3d0d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdaac073540 0x7fdaac168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab3d0d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaac15e790 con 0x7fdaac144080 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab3d0d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaac15ea40 con 0x7fdaac0d1040 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab3d0d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdaac15ed20 con 0x7fdaac073540 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdaac144080 0x7fdaac072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41662/0 (socket says 172.21.15.83:41662) 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 -- 172.21.15.83:0/960111025 learned_addr learned my addr 172.21.15.83:0/960111025 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab22aa640 1 --2- 172.21.15.83:0/960111025 >> v2:172.21.15.155:3300/0 conn(0x7fdaac073540 0x7fdaac168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239884052 0 0) 0x7fdaac15e790 con 0x7fdaac144080 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda98003530 con 0x7fdaac144080 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 780747334 0 0) 0x7fda98003530 con 0x7fdaac144080 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 >> v2:172.21.15.155:3300/0 conn(0x7fdaac073540 msgr2=0x7fdaac168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 --2- 172.21.15.83:0/960111025 >> v2:172.21.15.155:3300/0 conn(0x7fdaac073540 0x7fdaac168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 >> v1:172.21.15.155:6789/0 conn(0x7fdaac0d1040 legacy=0x7fdaac15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac15f000 con 0x7fdaac144080 2024-05-17T21:59:04.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fda9c001700 con 0x7fdaac144080 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 >> v1:172.21.15.83:6789/0 conn(0x7fdaac144080 legacy=0x7fdaac072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda98003a60 0x7fda98003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 0x7fda98004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98005260 0x7fda9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac15ed20 con 0x7fda98004810 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda9800c1f0 con 0x7fda98003a60 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda9800c3f0 con 0x7fda98005260 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 0x7fda98004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab1aa9640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda98003a60 0x7fda98003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab22aa640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98005260 0x7fda9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 -- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98005260 msgr2=0x7fda9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98005260 0x7fda9800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 -- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda98003a60 msgr2=0x7fda98003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda98003a60 0x7fda98003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab22aa640 1 -- 172.21.15.83:0/960111025 reap_dead start 2024-05-17T21:59:04.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.721+0000 7fdab12a8640 1 -- 172.21.15.83:0/960111025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98003530 con 0x7fda98004810 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab12a8640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 0x7fda98004c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdaac073ed0 tx=0x7fda9c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30da8863f324f3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c007d70 con 0x7fda98004810 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c0ac020 con 0x7fda98004810 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab0aa7640 1 -- 172.21.15.83:0/960111025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0058e0 con 0x7fda98004810 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 msgr2=0x7fda98004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 0x7fda98004c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdaac073ed0 tx=0x7fda9c01f210 comp rx=0 tx=0).stop 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- 172.21.15.83:0/960111025 shutdown_connections 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/960111025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda98004810 0x7fda98004c20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- 172.21.15.83:0/960111025 >> 172.21.15.83:0/960111025 conn(0x7fdaac06cd90 msgr2=0x7fdaac0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- 172.21.15.83:0/960111025 shutdown_connections 2024-05-17T21:59:04.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- 172.21.15.83:0/960111025 wait complete. 2024-05-17T21:59:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 Processor -- start 2024-05-17T21:59:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.725+0000 7fdab3d0d640 1 -- start start 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98004810 0x7fdaac13ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaac160b90 0x7fdaac15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac144490 con 0x7fdaac146090 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac06f210 con 0x7fdaac160b90 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdaac15e080 con 0x7fda98004810 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60432/0 (socket says 172.21.15.83:60432) 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 -- 172.21.15.83:0/3457667707 learned_addr learned my addr 172.21.15.83:0/3457667707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab12a8640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaac160b90 0x7fdaac15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab1aa9640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98004810 0x7fdaac13ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 -- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98004810 msgr2=0x7fdaac13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98004810 0x7fdaac13ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 -- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaac160b90 msgr2=0x7fdaac15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaac160b90 0x7fdaac15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 -- 172.21.15.83:0/3457667707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac15d4c0 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab22aa640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0cfe60 tx=0x7fdaa400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4016620 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaa401b070 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1469e0 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4016a20 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac15f6c0 con 0x7fdaac146090 2024-05-17T21:59:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac15e3b0 con 0x7fdaac146090 2024-05-17T21:59:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdaa4016bc0 con 0x7fdaac146090 2024-05-17T21:59:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.729+0000 7fdaa27fc640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 0x7fda90071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.733+0000 7fdab1aa9640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 0x7fda90071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.733+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fdaa4090fb0 con 0x7fdaac146090 2024-05-17T21:59:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.733+0000 7fdab1aa9640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 0x7fda90071940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdaac06b180 tx=0x7fdaa8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:04.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.733+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdaa4025020 con 0x7fdaac146090 2024-05-17T21:59:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.881+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdaac0b1160 con 0x7fda9006f430 2024-05-17T21:59:04.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.893+0000 7fdaa27fc640 1 -- 172.21.15.83:0/3457667707 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239845 (secure 0 0 0) 0x7fdaac0b1160 con 0x7fda9006f430 2024-05-17T21:59:04.887 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:04.889 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.893+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 msgr2=0x7fda90071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.893+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 0x7fda90071940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdaac06b180 tx=0x7fdaa8009040 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.893+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 msgr2=0x7fdaac1464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.893+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0cfe60 tx=0x7fdaa400f520 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 shutdown_connections 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaac146090 0x7fdaac1464a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda9006f430 0x7fda90071940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaac160b90 0x7fdaac15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 --2- 172.21.15.83:0/3457667707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda98004810 0x7fdaac13ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 >> 172.21.15.83:0/3457667707 conn(0x7fdaac06cd90 msgr2=0x7fdaac06fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 shutdown_connections 2024-05-17T21:59:04.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:04.897+0000 7fdab3d0d640 1 -- 172.21.15.83:0/3457667707 wait complete. 2024-05-17T21:59:04.907 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-05-17T21:59:02.995608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220936,"kb_used_data":4736,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753784,"statfs":{"total":773094113280,"available":772867874816,"internally_reserved":0,"allocated":4849664,"data_stored":2612162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.219791"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.778793+0000","last_change":"2024-05-17T21:58:58.778793+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.778793+0000","last_undegraded":"2024-05-17T21:58:58.778793+0000","last_fullsized":"2024-05-17T21:58:58.778793+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.787067+0000","last_change":"2024-05-17T21:58:58.787067+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.787067+0000","last_undegraded":"2024-05-17T21:58:58.787067+0000","last_fullsized":"2024-05-17T21:58:58.787067+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.785636+0000","last_change":"2024-05-17T21:58:58.785636+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.785636+0000","last_undegraded":"2024-05-17T21:58:58.785636+0000","last_fullsized":"2024-05-17T21:58:58.785636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.790606+0000","last_change":"2024-05-17T21:58:58.790606+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.790606+0000","last_undegraded":"2024-05-17T21:58:58.790606+0000","last_fullsized":"2024-05-17T21:58:58.790606+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.780754+0000","last_change":"2024-05-17T21:58:58.780754+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.780754+0000","last_undegraded":"2024-05-17T21:58:58.780754+0000","last_fullsized":"2024-05-17T21:58:58.780754+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.786931+0000","last_change":"2024-05-17T21:58:58.786931+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.786931+0000","last_undegraded":"2024-05-17T21:58:58.786931+0000","last_fullsized":"2024-05-17T21:58:58.786931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.780600+0000","last_change":"2024-05-17T21:58:58.780600+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.780600+0000","last_undegraded":"2024-05-17T21:58:58.780600+0000","last_fullsized":"2024-05-17T21:58:58.780600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.784987+0000","last_change":"2024-05-17T21:58:58.784987+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.784987+0000","last_undegraded":"2024-05-17T21:58:58.784987+0000","last_fullsized":"2024-05-17T21:58:58.784987+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.785115+0000","last_change":"2024-05-17T21:58:58.785115+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.785115+0000","last_undegraded":"2024-05-17T21:58:58.785115+0000","last_fullsized":"2024-05-17T21:58:58.785115+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.784154+0000","last_change":"2024-05-17T21:58:58.784154+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.784154+0000","last_undegraded":"2024-05-17T21:58:58.784154+0000","last_fullsized":"2024-05-17T21:58:58.784154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.789878+0000","last_change":"2024-05-17T21:58:58.789878+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.789878+0000","last_undegraded":"2024-05-17T21:58:58.789878+0000","last_fullsized":"2024-05-17T21:58:58.789878+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.791494+0000","last_change":"2024-05-17T21:58:58.791494+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.791494+0000","last_undegraded":"2024-05-17T21:58:58.791494+0000","last_fullsized":"2024-05-17T21:58:58.791494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.779869+0000","last_change":"2024-05-17T21:58:58.779869+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.779869+0000","last_undegraded":"2024-05-17T21:58:58.779869+0000","last_fullsized":"2024-05-17T21:58:58.779869+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.787637+0000","last_change":"2024-05-17T21:58:58.787637+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.787637+0000","last_undegraded":"2024-05-17T21:58:58.787637+0000","last_fullsized":"2024-05-17T21:58:58.787637+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.778702+0000","last_change":"2024-05-17T21:58:58.778702+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.778702+0000","last_undegraded":"2024-05-17T21:58:58.778702+0000","last_fullsized":"2024-05-17T21:58:58.778702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.784357+0000","last_change":"2024-05-17T21:58:58.784357+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.784357+0000","last_undegraded":"2024-05-17T21:58:58.784357+0000","last_fullsized":"2024-05-17T21:58:58.784357+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.791392+0000","last_change":"2024-05-17T21:58:58.791392+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.791392+0000","last_undegraded":"2024-05-17T21:58:58.791392+0000","last_fullsized":"2024-05-17T21:58:58.791392+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.785160+0000","last_change":"2024-05-17T21:58:58.785160+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.785160+0000","last_undegraded":"2024-05-17T21:58:58.785160+0000","last_fullsized":"2024-05-17T21:58:58.785160+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.780709+0000","last_change":"2024-05-17T21:58:58.780709+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.780709+0000","last_undegraded":"2024-05-17T21:58:58.780709+0000","last_fullsized":"2024-05-17T21:58:58.780709+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.778647+0000","last_change":"2024-05-17T21:58:58.778647+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.778647+0000","last_undegraded":"2024-05-17T21:58:58.778647+0000","last_fullsized":"2024-05-17T21:58:58.778647+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.778853+0000","last_change":"2024-05-17T21:58:58.778853+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.778853+0000","last_undegraded":"2024-05-17T21:58:58.778853+0000","last_fullsized":"2024-05-17T21:58:58.778853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.781173+0000","last_change":"2024-05-17T21:58:58.781173+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.781173+0000","last_undegraded":"2024-05-17T21:58:58.781173+0000","last_fullsized":"2024-05-17T21:58:58.781173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.783807+0000","last_change":"2024-05-17T21:58:58.783807+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.783807+0000","last_undegraded":"2024-05-17T21:58:58.783807+0000","last_fullsized":"2024-05-17T21:58:58.783807+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.787351+0000","last_change":"2024-05-17T21:58:58.787351+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.787351+0000","last_undegraded":"2024-05-17T21:58:58.787351+0000","last_fullsized":"2024-05-17T21:58:58.787351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.778867+0000","last_change":"2024-05-17T21:58:58.778867+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.778867+0000","last_undegraded":"2024-05-17T21:58:58.778867+0000","last_fullsized":"2024-05-17T21:58:58.778867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.788552+0000","last_change":"2024-05-17T21:58:58.788552+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.788552+0000","last_undegraded":"2024-05-17T21:58:58.788552+0000","last_fullsized":"2024-05-17T21:58:58.788552+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.783920+0000","last_change":"2024-05-17T21:58:58.783920+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.783920+0000","last_undegraded":"2024-05-17T21:58:58.783920+0000","last_fullsized":"2024-05-17T21:58:58.783920+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.789006+0000","last_change":"2024-05-17T21:58:58.789006+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.789006+0000","last_undegraded":"2024-05-17T21:58:58.789006+0000","last_fullsized":"2024-05-17T21:58:58.789006+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.785153+0000","last_change":"2024-05-17T21:58:58.785153+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.785153+0000","last_undegraded":"2024-05-17T21:58:58.785153+0000","last_fullsized":"2024-05-17T21:58:58.785153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.785452+0000","last_change":"2024-05-17T21:58:58.785452+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.785452+0000","last_undegraded":"2024-05-17T21:58:58.785452+0000","last_fullsized":"2024-05-17T21:58:58.785452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.783547+0000","last_change":"2024-05-17T21:58:58.783547+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.783547+0000","last_undegraded":"2024-05-17T21:58:58.783547+0000","last_fullsized":"2024-05-17T21:58:58.783547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-17T21:58:59.774319+0000","last_change":"2024-05-17T21:58:59.774319+0000","last_active":"2024-05-17T21:58:59.774319+0000","last_peered":"2024-05-17T21:58:59.774319+0000","last_clean":"2024-05-17T21:58:59.774319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:59.774319+0000","last_undegraded":"2024-05-17T21:58:59.774319+0000","last_fullsized":"2024-05-17T21:58:59.774319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-05-17T21:58:58.780709+0000","last_change":"2024-05-17T21:58:58.780709+0000","last_active":"2024-05-17T21:58:58.770182+0000","last_peered":"2024-05-17T21:58:58.770182+0000","last_clean":"2024-05-17T21:58:58.770182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-17T21:58:58.780709+0000","last_undegraded":"2024-05-17T21:58:58.780709+0000","last_fullsized":"2024-05-17T21:58:58.780709+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":145,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-17T21:58:58.791546+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:58:58.791546+0000","last_peered":"2024-05-17T21:58:58.791546+0000","last_clean":"2024-05-17T21:58:58.791546+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:58:58.791546+0000","last_undegraded":"2024-05-17T21:58:58.791546+0000","last_fullsized":"2024-05-17T21:58:58.791546+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1048576,"data_stored":767842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":177474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":177474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":177474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":177474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":177474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":773659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:59:04.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T21:59:05.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 Processor -- start 2024-05-17T21:59:05.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 -- start start 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee1c15d2f0 0x7fee1c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1c15c830 con 0x7fee1c15fe20 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee1c1608a0 con 0x7fee1c073540 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee22b7e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fee1c160b80 con 0x7fee1c15d2f0 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee1c15d2f0 0x7fee1c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee1c15d2f0 0x7fee1c159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60442/0 (socket says 172.21.15.83:60442) 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 -- 172.21.15.83:0/3885512895 learned_addr learned my addr 172.21.15.83:0/3885512895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 -- 172.21.15.83:0/3885512895 >> v1:172.21.15.155:6789/0 conn(0x7fee1c073540 legacy=0x7fee1c0b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 -- 172.21.15.83:0/3885512895 >> v1:172.21.15.83:6789/0 conn(0x7fee1c15fe20 legacy=0x7fee1c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.101+0000 7fee2091a640 1 -- 172.21.15.83:0/3885512895 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c15caf0 con 0x7fee1c15d2f0 2024-05-17T21:59:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee2091a640 1 --2- 172.21.15.83:0/3885512895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15d2f0 0x7fee1c159c20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00aa10 tx=0x7fee0c01d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08a0da76ce80f51 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee1b7fe640 1 -- 172.21.15.83:0/3885512895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0b3070 con 0x7fee1c15d2f0 2024-05-17T21:59:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee1b7fe640 1 -- 172.21.15.83:0/3885512895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0c0a6810 con 0x7fee1c15d2f0 2024-05-17T21:59:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee1b7fe640 1 -- 172.21.15.83:0/3885512895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c00b040 con 0x7fee1c15d2f0 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- 172.21.15.83:0/3885512895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15d2f0 msgr2=0x7fee1c159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 --2- 172.21.15.83:0/3885512895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15d2f0 0x7fee1c159c20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00aa10 tx=0x7fee0c01d930 comp rx=0 tx=0).stop 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- 172.21.15.83:0/3885512895 shutdown_connections 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 --2- 172.21.15.83:0/3885512895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15d2f0 0x7fee1c159c20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- 172.21.15.83:0/3885512895 >> 172.21.15.83:0/3885512895 conn(0x7fee1c06cd90 msgr2=0x7fee1c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- 172.21.15.83:0/3885512895 shutdown_connections 2024-05-17T21:59:05.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- 172.21.15.83:0/3885512895 wait complete. 2024-05-17T21:59:05.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 Processor -- start 2024-05-17T21:59:05.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.105+0000 7fee22b7e640 1 -- start start 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee1c15d2f0 0x7fee1c073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 0x7fee1c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0b1010 con 0x7fee1c15fe20 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee1c0b0e90 con 0x7fee1c158d30 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee1c0cf1d0 con 0x7fee1c15d2f0 2024-05-17T21:59:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60444/0 (socket says 172.21.15.83:60444) 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 -- 172.21.15.83:0/4073798001 learned_addr learned my addr 172.21.15.83:0/4073798001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee2111b640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 0x7fee1c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee2091a640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee1c15d2f0 0x7fee1c073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 -- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee1c15d2f0 msgr2=0x7fee1c073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee1c15d2f0 0x7fee1c073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 -- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 msgr2=0x7fee1c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 0x7fee1c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 -- 172.21.15.83:0/4073798001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1c157c90 con 0x7fee1c15fe20 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee2111b640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 0x7fee1c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee1bfff640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c3c0 tx=0x7fee1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10017070 con 0x7fee1c15fe20 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee100052d0 con 0x7fee1c15fe20 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1c157f70 con 0x7fee1c15fe20 2024-05-17T21:59:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10003820 con 0x7fee1c15fe20 2024-05-17T21:59:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1c158530 con 0x7fee1c15fe20 2024-05-17T21:59:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1c159c20 con 0x7fee1c15fe20 2024-05-17T21:59:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee1001a070 con 0x7fee1c15fe20 2024-05-17T21:59:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 0x7fee08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fee1008e6b0 con 0x7fee1c15fe20 2024-05-17T21:59:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.109+0000 7fee2091a640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 0x7fee08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.113+0000 7fee2091a640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 0x7fee08071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0b0000 tx=0x7fee0c0a8070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.113+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fee10021080 con 0x7fee1c15fe20 2024-05-17T21:59:05.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.269+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee1c073ca0 con 0x7fee0806f400 2024-05-17T21:59:05.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.277+0000 7fee197fa640 1 -- 172.21.15.83:0/4073798001 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fee0c0fa850 con 0x7fee0806f400 2024-05-17T21:59:05.272 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 msgr2=0x7fee08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 0x7fee08071910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0b0000 tx=0x7fee0c0a8070 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 msgr2=0x7fee1c14ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee1000c3c0 tx=0x7fee1000c880 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 shutdown_connections 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee0806f400 0x7fee08071910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee1c158d30 0x7fee1c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee1c15fe20 0x7fee1c14ef20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 --2- 172.21.15.83:0/4073798001 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee1c15d2f0 0x7fee1c073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 >> 172.21.15.83:0/4073798001 conn(0x7fee1c06cd90 msgr2=0x7fee1c0b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:05.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 shutdown_connections 2024-05-17T21:59:05.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.281+0000 7fee22b7e640 1 -- 172.21.15.83:0/4073798001 wait complete. 2024-05-17T21:59:05.293 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-05-17T21:59:04.995976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221320,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753400,"statfs":{"total":773094113280,"available":772867481600,"internally_reserved":0,"allocated":5283840,"data_stored":2806620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001017"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912774+0000","last_change":"2024-05-17T21:59:00.833398+0000","last_active":"2024-05-17T21:59:01.912774+0000","last_peered":"2024-05-17T21:59:01.912774+0000","last_clean":"2024-05-17T21:59:01.912774+0000","last_became_active":"2024-05-17T21:59:00.830746+0000","last_became_peered":"2024-05-17T21:59:00.830746+0000","last_unstale":"2024-05-17T21:59:01.912774+0000","last_undegraded":"2024-05-17T21:59:01.912774+0000","last_fullsized":"2024-05-17T21:59:01.912774+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:33:44.160199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.800451+0000","last_change":"2024-05-17T21:59:00.834460+0000","last_active":"2024-05-17T21:59:01.800451+0000","last_peered":"2024-05-17T21:59:01.800451+0000","last_clean":"2024-05-17T21:59:01.800451+0000","last_became_active":"2024-05-17T21:59:00.833611+0000","last_became_peered":"2024-05-17T21:59:00.833611+0000","last_unstale":"2024-05-17T21:59:01.800451+0000","last_undegraded":"2024-05-17T21:59:01.800451+0000","last_fullsized":"2024-05-17T21:59:01.800451+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:59.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796027+0000","last_change":"2024-05-17T21:59:00.800766+0000","last_active":"2024-05-17T21:59:01.796027+0000","last_peered":"2024-05-17T21:59:01.796027+0000","last_clean":"2024-05-17T21:59:01.796027+0000","last_became_active":"2024-05-17T21:59:00.799887+0000","last_became_peered":"2024-05-17T21:59:00.799887+0000","last_unstale":"2024-05-17T21:59:01.796027+0000","last_undegraded":"2024-05-17T21:59:01.796027+0000","last_fullsized":"2024-05-17T21:59:01.796027+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:33:40.664560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910866+0000","last_change":"2024-05-17T21:59:00.841979+0000","last_active":"2024-05-17T21:59:01.910866+0000","last_peered":"2024-05-17T21:59:01.910866+0000","last_clean":"2024-05-17T21:59:01.910866+0000","last_became_active":"2024-05-17T21:59:00.841683+0000","last_became_peered":"2024-05-17T21:59:00.841683+0000","last_unstale":"2024-05-17T21:59:01.910866+0000","last_undegraded":"2024-05-17T21:59:01.910866+0000","last_fullsized":"2024-05-17T21:59:01.910866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:42.071743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799700+0000","last_change":"2024-05-17T21:59:00.834150+0000","last_active":"2024-05-17T21:59:01.799700+0000","last_peered":"2024-05-17T21:59:01.799700+0000","last_clean":"2024-05-17T21:59:01.799700+0000","last_became_active":"2024-05-17T21:59:00.833278+0000","last_became_peered":"2024-05-17T21:59:00.833278+0000","last_unstale":"2024-05-17T21:59:01.799700+0000","last_undegraded":"2024-05-17T21:59:01.799700+0000","last_fullsized":"2024-05-17T21:59:01.799700+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:42.071743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803432+0000","last_change":"2024-05-17T21:59:00.838650+0000","last_active":"2024-05-17T21:59:01.803432+0000","last_peered":"2024-05-17T21:59:01.803432+0000","last_clean":"2024-05-17T21:59:01.803432+0000","last_became_active":"2024-05-17T21:59:00.838325+0000","last_became_peered":"2024-05-17T21:59:00.838325+0000","last_unstale":"2024-05-17T21:59:01.803432+0000","last_undegraded":"2024-05-17T21:59:01.803432+0000","last_fullsized":"2024-05-17T21:59:01.803432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:43:21.649022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912058+0000","last_change":"2024-05-17T21:59:00.830910+0000","last_active":"2024-05-17T21:59:01.912058+0000","last_peered":"2024-05-17T21:59:01.912058+0000","last_clean":"2024-05-17T21:59:01.912058+0000","last_became_active":"2024-05-17T21:59:00.830266+0000","last_became_peered":"2024-05-17T21:59:00.830266+0000","last_unstale":"2024-05-17T21:59:01.912058+0000","last_undegraded":"2024-05-17T21:59:01.912058+0000","last_fullsized":"2024-05-17T21:59:01.912058+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:06:10.425866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.909735+0000","last_change":"2024-05-17T21:59:00.841677+0000","last_active":"2024-05-17T21:59:01.909735+0000","last_peered":"2024-05-17T21:59:01.909735+0000","last_clean":"2024-05-17T21:59:01.909735+0000","last_became_active":"2024-05-17T21:59:00.841355+0000","last_became_peered":"2024-05-17T21:59:00.841355+0000","last_unstale":"2024-05-17T21:59:01.909735+0000","last_undegraded":"2024-05-17T21:59:01.909735+0000","last_fullsized":"2024-05-17T21:59:01.909735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:57.881187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911841+0000","last_change":"2024-05-17T21:59:00.828250+0000","last_active":"2024-05-17T21:59:01.911841+0000","last_peered":"2024-05-17T21:59:01.911841+0000","last_clean":"2024-05-17T21:59:01.911841+0000","last_became_active":"2024-05-17T21:59:00.824478+0000","last_became_peered":"2024-05-17T21:59:00.824478+0000","last_unstale":"2024-05-17T21:59:01.911841+0000","last_undegraded":"2024-05-17T21:59:01.911841+0000","last_fullsized":"2024-05-17T21:59:01.911841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:07:55.104355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910962+0000","last_change":"2024-05-17T21:59:00.841833+0000","last_active":"2024-05-17T21:59:01.910962+0000","last_peered":"2024-05-17T21:59:01.910962+0000","last_clean":"2024-05-17T21:59:01.910962+0000","last_became_active":"2024-05-17T21:59:00.841417+0000","last_became_peered":"2024-05-17T21:59:00.841417+0000","last_unstale":"2024-05-17T21:59:01.910962+0000","last_undegraded":"2024-05-17T21:59:01.910962+0000","last_fullsized":"2024-05-17T21:59:01.910962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:47.521901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911454+0000","last_change":"2024-05-17T21:59:00.820151+0000","last_active":"2024-05-17T21:59:01.911454+0000","last_peered":"2024-05-17T21:59:01.911454+0000","last_clean":"2024-05-17T21:59:01.911454+0000","last_became_active":"2024-05-17T21:59:00.819838+0000","last_became_peered":"2024-05-17T21:59:00.819838+0000","last_unstale":"2024-05-17T21:59:01.911454+0000","last_undegraded":"2024-05-17T21:59:01.911454+0000","last_fullsized":"2024-05-17T21:59:01.911454+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:59.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799938+0000","last_change":"2024-05-17T21:59:00.810359+0000","last_active":"2024-05-17T21:59:01.799938+0000","last_peered":"2024-05-17T21:59:01.799938+0000","last_clean":"2024-05-17T21:59:01.799938+0000","last_became_active":"2024-05-17T21:59:00.806995+0000","last_became_peered":"2024-05-17T21:59:00.806995+0000","last_unstale":"2024-05-17T21:59:01.799938+0000","last_undegraded":"2024-05-17T21:59:01.799938+0000","last_fullsized":"2024-05-17T21:59:01.799938+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:47.521901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796190+0000","last_change":"2024-05-17T21:59:00.822187+0000","last_active":"2024-05-17T21:59:01.796190+0000","last_peered":"2024-05-17T21:59:01.796190+0000","last_clean":"2024-05-17T21:59:01.796190+0000","last_became_active":"2024-05-17T21:59:00.821417+0000","last_became_peered":"2024-05-17T21:59:00.821417+0000","last_unstale":"2024-05-17T21:59:01.796190+0000","last_undegraded":"2024-05-17T21:59:01.796190+0000","last_fullsized":"2024-05-17T21:59:01.796190+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:46:38.930306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910034+0000","last_change":"2024-05-17T21:59:00.829602+0000","last_active":"2024-05-17T21:59:01.910034+0000","last_peered":"2024-05-17T21:59:01.910034+0000","last_clean":"2024-05-17T21:59:01.910034+0000","last_became_active":"2024-05-17T21:59:00.819910+0000","last_became_peered":"2024-05-17T21:59:00.819910+0000","last_unstale":"2024-05-17T21:59:01.910034+0000","last_undegraded":"2024-05-17T21:59:01.910034+0000","last_fullsized":"2024-05-17T21:59:01.910034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:33:40.664560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803288+0000","last_change":"2024-05-17T21:59:00.827773+0000","last_active":"2024-05-17T21:59:01.803288+0000","last_peered":"2024-05-17T21:59:01.803288+0000","last_clean":"2024-05-17T21:59:01.803288+0000","last_became_active":"2024-05-17T21:59:00.827522+0000","last_became_peered":"2024-05-17T21:59:00.827522+0000","last_unstale":"2024-05-17T21:59:01.803288+0000","last_undegraded":"2024-05-17T21:59:01.803288+0000","last_fullsized":"2024-05-17T21:59:01.803288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:48:28.814806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.914881+0000","last_change":"2024-05-17T21:59:00.827970+0000","last_active":"2024-05-17T21:59:01.914881+0000","last_peered":"2024-05-17T21:59:01.914881+0000","last_clean":"2024-05-17T21:59:01.914881+0000","last_became_active":"2024-05-17T21:59:00.827164+0000","last_became_peered":"2024-05-17T21:59:00.827164+0000","last_unstale":"2024-05-17T21:59:01.914881+0000","last_undegraded":"2024-05-17T21:59:01.914881+0000","last_fullsized":"2024-05-17T21:59:01.914881+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:41:04.708316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796401+0000","last_change":"2024-05-17T21:59:00.804129+0000","last_active":"2024-05-17T21:59:01.796401+0000","last_peered":"2024-05-17T21:59:01.796401+0000","last_clean":"2024-05-17T21:59:01.796401+0000","last_became_active":"2024-05-17T21:59:00.803276+0000","last_became_peered":"2024-05-17T21:59:00.803276+0000","last_unstale":"2024-05-17T21:59:01.796401+0000","last_undegraded":"2024-05-17T21:59:01.796401+0000","last_fullsized":"2024-05-17T21:59:01.796401+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:37.873929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796828+0000","last_change":"2024-05-17T21:59:00.823576+0000","last_active":"2024-05-17T21:59:01.796828+0000","last_peered":"2024-05-17T21:59:01.796828+0000","last_clean":"2024-05-17T21:59:01.796828+0000","last_became_active":"2024-05-17T21:59:00.821609+0000","last_became_peered":"2024-05-17T21:59:00.821609+0000","last_unstale":"2024-05-17T21:59:01.796828+0000","last_undegraded":"2024-05-17T21:59:01.796828+0000","last_fullsized":"2024-05-17T21:59:01.796828+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:19:29.182968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915652+0000","last_change":"2024-05-17T21:59:00.831272+0000","last_active":"2024-05-17T21:59:01.915652+0000","last_peered":"2024-05-17T21:59:01.915652+0000","last_clean":"2024-05-17T21:59:01.915652+0000","last_became_active":"2024-05-17T21:59:00.830432+0000","last_became_peered":"2024-05-17T21:59:00.830432+0000","last_unstale":"2024-05-17T21:59:01.915652+0000","last_undegraded":"2024-05-17T21:59:01.915652+0000","last_fullsized":"2024-05-17T21:59:01.915652+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:22.207669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796141+0000","last_change":"2024-05-17T21:59:00.816819+0000","last_active":"2024-05-17T21:59:01.796141+0000","last_peered":"2024-05-17T21:59:01.796141+0000","last_clean":"2024-05-17T21:59:01.796141+0000","last_became_active":"2024-05-17T21:59:00.816522+0000","last_became_peered":"2024-05-17T21:59:00.816522+0000","last_unstale":"2024-05-17T21:59:01.796141+0000","last_undegraded":"2024-05-17T21:59:01.796141+0000","last_fullsized":"2024-05-17T21:59:01.796141+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:26:03.972153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803770+0000","last_change":"2024-05-17T21:59:00.837232+0000","last_active":"2024-05-17T21:59:01.803770+0000","last_peered":"2024-05-17T21:59:01.803770+0000","last_clean":"2024-05-17T21:59:01.803770+0000","last_became_active":"2024-05-17T21:59:00.836933+0000","last_became_peered":"2024-05-17T21:59:00.836933+0000","last_unstale":"2024-05-17T21:59:01.803770+0000","last_undegraded":"2024-05-17T21:59:01.803770+0000","last_fullsized":"2024-05-17T21:59:01.803770+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:49:09.771286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910048+0000","last_change":"2024-05-17T21:59:00.842243+0000","last_active":"2024-05-17T21:59:01.910048+0000","last_peered":"2024-05-17T21:59:01.910048+0000","last_clean":"2024-05-17T21:59:01.910048+0000","last_became_active":"2024-05-17T21:59:00.841557+0000","last_became_peered":"2024-05-17T21:59:00.841557+0000","last_unstale":"2024-05-17T21:59:01.910048+0000","last_undegraded":"2024-05-17T21:59:01.910048+0000","last_fullsized":"2024-05-17T21:59:01.910048+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:37.873929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.914714+0000","last_change":"2024-05-17T21:59:00.846084+0000","last_active":"2024-05-17T21:59:01.914714+0000","last_peered":"2024-05-17T21:59:01.914714+0000","last_clean":"2024-05-17T21:59:01.914714+0000","last_became_active":"2024-05-17T21:59:00.844740+0000","last_became_peered":"2024-05-17T21:59:00.844740+0000","last_unstale":"2024-05-17T21:59:01.914714+0000","last_undegraded":"2024-05-17T21:59:01.914714+0000","last_fullsized":"2024-05-17T21:59:01.914714+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:22:03.342222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.914447+0000","last_change":"2024-05-17T21:59:00.845839+0000","last_active":"2024-05-17T21:59:01.914447+0000","last_peered":"2024-05-17T21:59:01.914447+0000","last_clean":"2024-05-17T21:59:01.914447+0000","last_became_active":"2024-05-17T21:59:00.844499+0000","last_became_peered":"2024-05-17T21:59:00.844499+0000","last_unstale":"2024-05-17T21:59:01.914447+0000","last_undegraded":"2024-05-17T21:59:01.914447+0000","last_fullsized":"2024-05-17T21:59:01.914447+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:51:54.664987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915500+0000","last_change":"2024-05-17T21:59:00.844239+0000","last_active":"2024-05-17T21:59:01.915500+0000","last_peered":"2024-05-17T21:59:01.915500+0000","last_clean":"2024-05-17T21:59:01.915500+0000","last_became_active":"2024-05-17T21:59:00.843253+0000","last_became_peered":"2024-05-17T21:59:00.843253+0000","last_unstale":"2024-05-17T21:59:01.915500+0000","last_undegraded":"2024-05-17T21:59:01.915500+0000","last_fullsized":"2024-05-17T21:59:01.915500+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:59:40.730799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.795834+0000","last_change":"2024-05-17T21:59:00.812497+0000","last_active":"2024-05-17T21:59:01.795834+0000","last_peered":"2024-05-17T21:59:01.795834+0000","last_clean":"2024-05-17T21:59:01.795834+0000","last_became_active":"2024-05-17T21:59:00.812145+0000","last_became_peered":"2024-05-17T21:59:00.812145+0000","last_unstale":"2024-05-17T21:59:01.795834+0000","last_undegraded":"2024-05-17T21:59:01.795834+0000","last_fullsized":"2024-05-17T21:59:01.795834+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:10:55.743871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796596+0000","last_change":"2024-05-17T21:59:00.806529+0000","last_active":"2024-05-17T21:59:01.796596+0000","last_peered":"2024-05-17T21:59:01.796596+0000","last_clean":"2024-05-17T21:59:01.796596+0000","last_became_active":"2024-05-17T21:59:00.805655+0000","last_became_peered":"2024-05-17T21:59:00.805655+0000","last_unstale":"2024-05-17T21:59:01.796596+0000","last_undegraded":"2024-05-17T21:59:01.796596+0000","last_fullsized":"2024-05-17T21:59:01.796596+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:12:08.365913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.800076+0000","last_change":"2024-05-17T21:59:00.809698+0000","last_active":"2024-05-17T21:59:01.800076+0000","last_peered":"2024-05-17T21:59:01.800076+0000","last_clean":"2024-05-17T21:59:01.800076+0000","last_became_active":"2024-05-17T21:59:00.805935+0000","last_became_peered":"2024-05-17T21:59:00.805935+0000","last_unstale":"2024-05-17T21:59:01.800076+0000","last_undegraded":"2024-05-17T21:59:01.800076+0000","last_fullsized":"2024-05-17T21:59:01.800076+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:06:10.425866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915150+0000","last_change":"2024-05-17T21:59:00.844719+0000","last_active":"2024-05-17T21:59:01.915150+0000","last_peered":"2024-05-17T21:59:01.915150+0000","last_clean":"2024-05-17T21:59:01.915150+0000","last_became_active":"2024-05-17T21:59:00.843321+0000","last_became_peered":"2024-05-17T21:59:00.843321+0000","last_unstale":"2024-05-17T21:59:01.915150+0000","last_undegraded":"2024-05-17T21:59:01.915150+0000","last_fullsized":"2024-05-17T21:59:01.915150+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:05:55.731318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.795899+0000","last_change":"2024-05-17T21:59:00.829730+0000","last_active":"2024-05-17T21:59:01.795899+0000","last_peered":"2024-05-17T21:59:01.795899+0000","last_clean":"2024-05-17T21:59:01.795899+0000","last_became_active":"2024-05-17T21:59:00.828904+0000","last_became_peered":"2024-05-17T21:59:00.828904+0000","last_unstale":"2024-05-17T21:59:01.795899+0000","last_undegraded":"2024-05-17T21:59:01.795899+0000","last_fullsized":"2024-05-17T21:59:01.795899+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:57.881187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803128+0000","last_change":"2024-05-17T21:59:00.843748+0000","last_active":"2024-05-17T21:59:01.803128+0000","last_peered":"2024-05-17T21:59:01.803128+0000","last_clean":"2024-05-17T21:59:01.803128+0000","last_became_active":"2024-05-17T21:59:00.843398+0000","last_became_peered":"2024-05-17T21:59:00.843398+0000","last_unstale":"2024-05-17T21:59:01.803128+0000","last_undegraded":"2024-05-17T21:59:01.803128+0000","last_fullsized":"2024-05-17T21:59:01.803128+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:01:14.733945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796442+0000","last_change":"2024-05-17T21:59:00.812824+0000","last_active":"2024-05-17T21:59:01.796442+0000","last_peered":"2024-05-17T21:59:01.796442+0000","last_clean":"2024-05-17T21:59:01.796442+0000","last_became_active":"2024-05-17T21:59:00.812153+0000","last_became_peered":"2024-05-17T21:59:00.812153+0000","last_unstale":"2024-05-17T21:59:01.796442+0000","last_undegraded":"2024-05-17T21:59:01.796442+0000","last_fullsized":"2024-05-17T21:59:01.796442+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:45:41.017216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912703+0000","last_change":"2024-05-17T21:59:00.822537+0000","last_active":"2024-05-17T21:59:01.912703+0000","last_peered":"2024-05-17T21:59:01.912703+0000","last_clean":"2024-05-17T21:59:01.912703+0000","last_became_active":"2024-05-17T21:59:00.821381+0000","last_became_peered":"2024-05-17T21:59:00.821381+0000","last_unstale":"2024-05-17T21:59:01.912703+0000","last_undegraded":"2024-05-17T21:59:01.912703+0000","last_fullsized":"2024-05-17T21:59:01.912703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:25:41.904376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.804499+0000","last_change":"2024-05-17T21:58:59.824071+0000","last_active":"2024-05-17T21:59:01.804499+0000","last_peered":"2024-05-17T21:59:01.804499+0000","last_clean":"2024-05-17T21:59:01.804499+0000","last_became_active":"2024-05-17T21:58:59.823829+0000","last_became_peered":"2024-05-17T21:58:59.823829+0000","last_unstale":"2024-05-17T21:59:01.804499+0000","last_undegraded":"2024-05-17T21:59:01.804499+0000","last_fullsized":"2024-05-17T21:59:01.804499+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:12:07.361776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915750+0000","last_change":"2024-05-17T21:59:00.848123+0000","last_active":"2024-05-17T21:59:01.915750+0000","last_peered":"2024-05-17T21:59:01.915750+0000","last_clean":"2024-05-17T21:59:01.915750+0000","last_became_active":"2024-05-17T21:59:00.847546+0000","last_became_peered":"2024-05-17T21:59:00.847546+0000","last_unstale":"2024-05-17T21:59:01.915750+0000","last_undegraded":"2024-05-17T21:59:01.915750+0000","last_fullsized":"2024-05-17T21:59:01.915750+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:27:35.670094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796982+0000","last_change":"2024-05-17T21:58:59.831703+0000","last_active":"2024-05-17T21:59:01.796982+0000","last_peered":"2024-05-17T21:59:01.796982+0000","last_clean":"2024-05-17T21:59:01.796982+0000","last_became_active":"2024-05-17T21:58:59.831011+0000","last_became_peered":"2024-05-17T21:58:59.831011+0000","last_unstale":"2024-05-17T21:59:01.796982+0000","last_undegraded":"2024-05-17T21:59:01.796982+0000","last_fullsized":"2024-05-17T21:59:01.796982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:05:54.727182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.800587+0000","last_change":"2024-05-17T21:59:00.831624+0000","last_active":"2024-05-17T21:59:01.800587+0000","last_peered":"2024-05-17T21:59:01.800587+0000","last_clean":"2024-05-17T21:59:01.800587+0000","last_became_active":"2024-05-17T21:59:00.813630+0000","last_became_peered":"2024-05-17T21:59:00.813630+0000","last_unstale":"2024-05-17T21:59:01.800587+0000","last_undegraded":"2024-05-17T21:59:01.800587+0000","last_fullsized":"2024-05-17T21:59:01.800587+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:07:55.104355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911938+0000","last_change":"2024-05-17T21:58:59.819808+0000","last_active":"2024-05-17T21:59:01.911938+0000","last_peered":"2024-05-17T21:59:01.911938+0000","last_clean":"2024-05-17T21:59:01.911938+0000","last_became_active":"2024-05-17T21:58:59.819344+0000","last_became_peered":"2024-05-17T21:58:59.819344+0000","last_unstale":"2024-05-17T21:59:01.911938+0000","last_undegraded":"2024-05-17T21:59:01.911938+0000","last_fullsized":"2024-05-17T21:59:01.911938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:55:42.601482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911852+0000","last_change":"2024-05-17T21:59:00.847857+0000","last_active":"2024-05-17T21:59:01.911852+0000","last_peered":"2024-05-17T21:59:01.911852+0000","last_clean":"2024-05-17T21:59:01.911852+0000","last_became_active":"2024-05-17T21:59:00.847508+0000","last_became_peered":"2024-05-17T21:59:00.847508+0000","last_unstale":"2024-05-17T21:59:01.911852+0000","last_undegraded":"2024-05-17T21:59:01.911852+0000","last_fullsized":"2024-05-17T21:59:01.911852+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:47.560158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911855+0000","last_change":"2024-05-17T21:58:59.826810+0000","last_active":"2024-05-17T21:59:01.911855+0000","last_peered":"2024-05-17T21:59:01.911855+0000","last_clean":"2024-05-17T21:59:01.911855+0000","last_became_active":"2024-05-17T21:58:59.826045+0000","last_became_peered":"2024-05-17T21:58:59.826045+0000","last_unstale":"2024-05-17T21:59:01.911855+0000","last_undegraded":"2024-05-17T21:59:01.911855+0000","last_fullsized":"2024-05-17T21:59:01.911855+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:16:11.270064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796195+0000","last_change":"2024-05-17T21:59:00.850435+0000","last_active":"2024-05-17T21:59:01.796195+0000","last_peered":"2024-05-17T21:59:01.796195+0000","last_clean":"2024-05-17T21:59:01.796195+0000","last_became_active":"2024-05-17T21:59:00.849613+0000","last_became_peered":"2024-05-17T21:59:00.849613+0000","last_unstale":"2024-05-17T21:59:01.796195+0000","last_undegraded":"2024-05-17T21:59:01.796195+0000","last_fullsized":"2024-05-17T21:59:01.796195+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:59.546842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803964+0000","last_change":"2024-05-17T21:58:59.828240+0000","last_active":"2024-05-17T21:59:01.803964+0000","last_peered":"2024-05-17T21:59:01.803964+0000","last_clean":"2024-05-17T21:59:01.803964+0000","last_became_active":"2024-05-17T21:58:59.827994+0000","last_became_peered":"2024-05-17T21:58:59.827994+0000","last_unstale":"2024-05-17T21:59:01.803964+0000","last_undegraded":"2024-05-17T21:59:01.803964+0000","last_fullsized":"2024-05-17T21:59:01.803964+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:58.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910289+0000","last_change":"2024-05-17T21:59:00.829127+0000","last_active":"2024-05-17T21:59:01.910289+0000","last_peered":"2024-05-17T21:59:01.910289+0000","last_clean":"2024-05-17T21:59:01.910289+0000","last_became_active":"2024-05-17T21:59:00.824171+0000","last_became_peered":"2024-05-17T21:59:00.824171+0000","last_unstale":"2024-05-17T21:59:01.910289+0000","last_undegraded":"2024-05-17T21:59:01.910289+0000","last_fullsized":"2024-05-17T21:59:01.910289+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:12:08.365913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796079+0000","last_change":"2024-05-17T21:58:59.814247+0000","last_active":"2024-05-17T21:59:01.796079+0000","last_peered":"2024-05-17T21:59:01.796079+0000","last_clean":"2024-05-17T21:59:01.796079+0000","last_became_active":"2024-05-17T21:58:59.813917+0000","last_became_peered":"2024-05-17T21:58:59.813917+0000","last_unstale":"2024-05-17T21:59:01.796079+0000","last_undegraded":"2024-05-17T21:59:01.796079+0000","last_fullsized":"2024-05-17T21:59:01.796079+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:22:02.338086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.909737+0000","last_change":"2024-05-17T21:59:00.841639+0000","last_active":"2024-05-17T21:59:01.909737+0000","last_peered":"2024-05-17T21:59:01.909737+0000","last_clean":"2024-05-17T21:59:01.909737+0000","last_became_active":"2024-05-17T21:59:00.841312+0000","last_became_peered":"2024-05-17T21:59:00.841312+0000","last_unstale":"2024-05-17T21:59:01.909737+0000","last_undegraded":"2024-05-17T21:59:01.909737+0000","last_fullsized":"2024-05-17T21:59:01.909737+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:25:41.904376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799326+0000","last_change":"2024-05-17T21:58:59.803127+0000","last_active":"2024-05-17T21:59:01.799326+0000","last_peered":"2024-05-17T21:59:01.799326+0000","last_clean":"2024-05-17T21:59:01.799326+0000","last_became_active":"2024-05-17T21:58:59.802546+0000","last_became_peered":"2024-05-17T21:58:59.802546+0000","last_unstale":"2024-05-17T21:59:01.799326+0000","last_undegraded":"2024-05-17T21:59:01.799326+0000","last_fullsized":"2024-05-17T21:59:01.799326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:21.203533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.914507+0000","last_change":"2024-05-17T21:59:00.846572+0000","last_active":"2024-05-17T21:59:01.914507+0000","last_peered":"2024-05-17T21:59:01.914507+0000","last_clean":"2024-05-17T21:59:01.914507+0000","last_became_active":"2024-05-17T21:59:00.845533+0000","last_became_peered":"2024-05-17T21:59:00.845533+0000","last_unstale":"2024-05-17T21:59:01.914507+0000","last_undegraded":"2024-05-17T21:59:01.914507+0000","last_fullsized":"2024-05-17T21:59:01.914507+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:56:44.696144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.909823+0000","last_change":"2024-05-17T21:58:59.802714+0000","last_active":"2024-05-17T21:59:01.909823+0000","last_peered":"2024-05-17T21:59:01.909823+0000","last_clean":"2024-05-17T21:59:01.909823+0000","last_became_active":"2024-05-17T21:58:59.802410+0000","last_became_peered":"2024-05-17T21:58:59.802410+0000","last_unstale":"2024-05-17T21:59:01.909823+0000","last_undegraded":"2024-05-17T21:59:01.909823+0000","last_fullsized":"2024-05-17T21:59:01.909823+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:24:47.826007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.797005+0000","last_change":"2024-05-17T21:59:00.824271+0000","last_active":"2024-05-17T21:59:01.797005+0000","last_peered":"2024-05-17T21:59:01.797005+0000","last_clean":"2024-05-17T21:59:01.797005+0000","last_became_active":"2024-05-17T21:59:00.822700+0000","last_became_peered":"2024-05-17T21:59:00.822700+0000","last_unstale":"2024-05-17T21:59:01.797005+0000","last_undegraded":"2024-05-17T21:59:01.797005+0000","last_fullsized":"2024-05-17T21:59:01.797005+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:42.071743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796642+0000","last_change":"2024-05-17T21:58:59.811536+0000","last_active":"2024-05-17T21:59:01.796642+0000","last_peered":"2024-05-17T21:59:01.796642+0000","last_clean":"2024-05-17T21:59:01.796642+0000","last_became_active":"2024-05-17T21:58:59.810502+0000","last_became_peered":"2024-05-17T21:58:59.810502+0000","last_unstale":"2024-05-17T21:59:01.796642+0000","last_undegraded":"2024-05-17T21:59:01.796642+0000","last_fullsized":"2024-05-17T21:59:01.796642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:56:43.692007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912512+0000","last_change":"2024-05-17T21:59:00.846749+0000","last_active":"2024-05-17T21:59:01.912512+0000","last_peered":"2024-05-17T21:59:01.912512+0000","last_clean":"2024-05-17T21:59:01.912512+0000","last_became_active":"2024-05-17T21:59:00.846114+0000","last_became_peered":"2024-05-17T21:59:00.846114+0000","last_unstale":"2024-05-17T21:59:01.912512+0000","last_undegraded":"2024-05-17T21:59:01.912512+0000","last_fullsized":"2024-05-17T21:59:01.912512+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:58.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911111+0000","last_change":"2024-05-17T21:58:59.825782+0000","last_active":"2024-05-17T21:59:01.911111+0000","last_peered":"2024-05-17T21:59:01.911111+0000","last_clean":"2024-05-17T21:59:01.911111+0000","last_became_active":"2024-05-17T21:58:59.825259+0000","last_became_peered":"2024-05-17T21:58:59.825259+0000","last_unstale":"2024-05-17T21:59:01.911111+0000","last_undegraded":"2024-05-17T21:59:01.911111+0000","last_fullsized":"2024-05-17T21:59:01.911111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:33:29.744952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803767+0000","last_change":"2024-05-17T21:59:00.818697+0000","last_active":"2024-05-17T21:59:01.803767+0000","last_peered":"2024-05-17T21:59:01.803767+0000","last_clean":"2024-05-17T21:59:01.803767+0000","last_became_active":"2024-05-17T21:59:00.818325+0000","last_became_peered":"2024-05-17T21:59:00.818325+0000","last_unstale":"2024-05-17T21:59:01.803767+0000","last_undegraded":"2024-05-17T21:59:01.803767+0000","last_fullsized":"2024-05-17T21:59:01.803767+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:54:44.082352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912447+0000","last_change":"2024-05-17T21:58:59.828131+0000","last_active":"2024-05-17T21:59:01.912447+0000","last_peered":"2024-05-17T21:59:01.912447+0000","last_clean":"2024-05-17T21:59:01.912447+0000","last_became_active":"2024-05-17T21:58:59.826790+0000","last_became_peered":"2024-05-17T21:58:59.826790+0000","last_unstale":"2024-05-17T21:59:01.912447+0000","last_undegraded":"2024-05-17T21:59:01.912447+0000","last_fullsized":"2024-05-17T21:59:01.912447+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:25:40.900239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912292+0000","last_change":"2024-05-17T21:59:00.835846+0000","last_active":"2024-05-17T21:59:01.912292+0000","last_peered":"2024-05-17T21:59:01.912292+0000","last_clean":"2024-05-17T21:59:01.912292+0000","last_became_active":"2024-05-17T21:59:00.834879+0000","last_became_peered":"2024-05-17T21:59:00.834879+0000","last_unstale":"2024-05-17T21:59:01.912292+0000","last_undegraded":"2024-05-17T21:59:01.912292+0000","last_fullsized":"2024-05-17T21:59:01.912292+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:07:55.104355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796294+0000","last_change":"2024-05-17T21:58:59.818417+0000","last_active":"2024-05-17T21:59:01.796294+0000","last_peered":"2024-05-17T21:59:01.796294+0000","last_clean":"2024-05-17T21:59:01.796294+0000","last_became_active":"2024-05-17T21:58:59.817929+0000","last_became_peered":"2024-05-17T21:58:59.817929+0000","last_unstale":"2024-05-17T21:59:01.796294+0000","last_undegraded":"2024-05-17T21:59:01.796294+0000","last_fullsized":"2024-05-17T21:59:01.796294+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:28:14.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.914803+0000","last_change":"2024-05-17T21:59:00.837193+0000","last_active":"2024-05-17T21:59:01.914803+0000","last_peered":"2024-05-17T21:59:01.914803+0000","last_clean":"2024-05-17T21:59:01.914803+0000","last_became_active":"2024-05-17T21:59:00.836190+0000","last_became_peered":"2024-05-17T21:59:00.836190+0000","last_unstale":"2024-05-17T21:59:01.914803+0000","last_undegraded":"2024-05-17T21:59:01.914803+0000","last_fullsized":"2024-05-17T21:59:01.914803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:35.457417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.804284+0000","last_change":"2024-05-17T21:58:59.807639+0000","last_active":"2024-05-17T21:59:01.804284+0000","last_peered":"2024-05-17T21:59:01.804284+0000","last_clean":"2024-05-17T21:59:01.804284+0000","last_became_active":"2024-05-17T21:58:59.807281+0000","last_became_peered":"2024-05-17T21:58:59.807281+0000","last_unstale":"2024-05-17T21:59:01.804284+0000","last_undegraded":"2024-05-17T21:59:01.804284+0000","last_fullsized":"2024-05-17T21:59:01.804284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:59:46.517764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911979+0000","last_change":"2024-05-17T21:59:00.828936+0000","last_active":"2024-05-17T21:59:01.911979+0000","last_peered":"2024-05-17T21:59:01.911979+0000","last_clean":"2024-05-17T21:59:01.911979+0000","last_became_active":"2024-05-17T21:59:00.828655+0000","last_became_peered":"2024-05-17T21:59:00.828655+0000","last_unstale":"2024-05-17T21:59:01.911979+0000","last_undegraded":"2024-05-17T21:59:01.911979+0000","last_fullsized":"2024-05-17T21:59:01.911979+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:58.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911882+0000","last_change":"2024-05-17T21:58:59.827782+0000","last_active":"2024-05-17T21:59:01.911882+0000","last_peered":"2024-05-17T21:59:01.911882+0000","last_clean":"2024-05-17T21:59:01.911882+0000","last_became_active":"2024-05-17T21:58:59.826049+0000","last_became_peered":"2024-05-17T21:58:59.826049+0000","last_unstale":"2024-05-17T21:59:01.911882+0000","last_undegraded":"2024-05-17T21:59:01.911882+0000","last_fullsized":"2024-05-17T21:59:01.911882+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:46.556021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803309+0000","last_change":"2024-05-17T21:59:00.819071+0000","last_active":"2024-05-17T21:59:01.803309+0000","last_peered":"2024-05-17T21:59:01.803309+0000","last_clean":"2024-05-17T21:59:01.803309+0000","last_became_active":"2024-05-17T21:59:00.818760+0000","last_became_peered":"2024-05-17T21:59:00.818760+0000","last_unstale":"2024-05-17T21:59:01.803309+0000","last_undegraded":"2024-05-17T21:59:01.803309+0000","last_fullsized":"2024-05-17T21:59:01.803309+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:56:33.553464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803195+0000","last_change":"2024-05-17T21:58:59.806525+0000","last_active":"2024-05-17T21:59:01.803195+0000","last_peered":"2024-05-17T21:59:01.803195+0000","last_clean":"2024-05-17T21:59:01.803195+0000","last_became_active":"2024-05-17T21:58:59.805682+0000","last_became_peered":"2024-05-17T21:58:59.805682+0000","last_unstale":"2024-05-17T21:59:01.803195+0000","last_undegraded":"2024-05-17T21:59:01.803195+0000","last_fullsized":"2024-05-17T21:59:01.803195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:41.067607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912156+0000","last_change":"2024-05-17T21:59:00.853693+0000","last_active":"2024-05-17T21:59:01.912156+0000","last_peered":"2024-05-17T21:59:01.912156+0000","last_clean":"2024-05-17T21:59:01.912156+0000","last_became_active":"2024-05-17T21:59:00.852893+0000","last_became_peered":"2024-05-17T21:59:00.852893+0000","last_unstale":"2024-05-17T21:59:01.912156+0000","last_undegraded":"2024-05-17T21:59:01.912156+0000","last_fullsized":"2024-05-17T21:59:01.912156+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:59.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.909906+0000","last_change":"2024-05-17T21:58:59.804625+0000","last_active":"2024-05-17T21:59:01.909906+0000","last_peered":"2024-05-17T21:59:01.909906+0000","last_clean":"2024-05-17T21:59:01.909906+0000","last_became_active":"2024-05-17T21:58:59.804249+0000","last_became_peered":"2024-05-17T21:58:59.804249+0000","last_unstale":"2024-05-17T21:59:01.909906+0000","last_undegraded":"2024-05-17T21:59:01.909906+0000","last_fullsized":"2024-05-17T21:59:01.909906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:54:57.869608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796049+0000","last_change":"2024-05-17T21:59:00.801323+0000","last_active":"2024-05-17T21:59:01.796049+0000","last_peered":"2024-05-17T21:59:01.796049+0000","last_clean":"2024-05-17T21:59:01.796049+0000","last_became_active":"2024-05-17T21:59:00.801011+0000","last_became_peered":"2024-05-17T21:59:00.801011+0000","last_unstale":"2024-05-17T21:59:01.796049+0000","last_undegraded":"2024-05-17T21:59:01.796049+0000","last_fullsized":"2024-05-17T21:59:01.796049+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:33:29.410465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796294+0000","last_change":"2024-05-17T21:58:59.808178+0000","last_active":"2024-05-17T21:59:01.796294+0000","last_peered":"2024-05-17T21:59:01.796294+0000","last_clean":"2024-05-17T21:59:01.796294+0000","last_became_active":"2024-05-17T21:58:59.807682+0000","last_became_peered":"2024-05-17T21:58:59.807682+0000","last_unstale":"2024-05-17T21:59:01.796294+0000","last_undegraded":"2024-05-17T21:59:01.796294+0000","last_fullsized":"2024-05-17T21:59:01.796294+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:30:50.834678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911860+0000","last_change":"2024-05-17T21:59:00.812121+0000","last_active":"2024-05-17T21:59:01.911860+0000","last_peered":"2024-05-17T21:59:01.911860+0000","last_clean":"2024-05-17T21:59:01.911860+0000","last_became_active":"2024-05-17T21:59:00.811813+0000","last_became_peered":"2024-05-17T21:59:00.811813+0000","last_unstale":"2024-05-17T21:59:01.911860+0000","last_undegraded":"2024-05-17T21:59:01.911860+0000","last_fullsized":"2024-05-17T21:59:01.911860+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:00:21.530177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.797160+0000","last_change":"2024-05-17T21:58:59.815998+0000","last_active":"2024-05-17T21:59:01.797160+0000","last_peered":"2024-05-17T21:59:01.797160+0000","last_clean":"2024-05-17T21:59:01.797160+0000","last_became_active":"2024-05-17T21:58:59.815676+0000","last_became_peered":"2024-05-17T21:58:59.815676+0000","last_unstale":"2024-05-17T21:59:01.797160+0000","last_undegraded":"2024-05-17T21:59:01.797160+0000","last_fullsized":"2024-05-17T21:59:01.797160+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:41:03.704180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915178+0000","last_change":"2024-05-17T21:59:00.829754+0000","last_active":"2024-05-17T21:59:01.915178+0000","last_peered":"2024-05-17T21:59:01.915178+0000","last_clean":"2024-05-17T21:59:01.915178+0000","last_became_active":"2024-05-17T21:59:00.828879+0000","last_became_peered":"2024-05-17T21:59:00.828879+0000","last_unstale":"2024-05-17T21:59:01.915178+0000","last_undegraded":"2024-05-17T21:59:01.915178+0000","last_fullsized":"2024-05-17T21:59:01.915178+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:41:06.689376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799707+0000","last_change":"2024-05-17T21:58:59.817482+0000","last_active":"2024-05-17T21:59:01.799707+0000","last_peered":"2024-05-17T21:59:01.799707+0000","last_clean":"2024-05-17T21:59:01.799707+0000","last_became_active":"2024-05-17T21:58:59.816663+0000","last_became_peered":"2024-05-17T21:58:59.816663+0000","last_unstale":"2024-05-17T21:59:01.799707+0000","last_undegraded":"2024-05-17T21:59:01.799707+0000","last_fullsized":"2024-05-17T21:59:01.799707+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:16:11.270064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915270+0000","last_change":"2024-05-17T21:59:00.833363+0000","last_active":"2024-05-17T21:59:01.915270+0000","last_peered":"2024-05-17T21:59:01.915270+0000","last_clean":"2024-05-17T21:59:01.915270+0000","last_became_active":"2024-05-17T21:59:00.832451+0000","last_became_peered":"2024-05-17T21:59:00.832451+0000","last_unstale":"2024-05-17T21:59:01.915270+0000","last_undegraded":"2024-05-17T21:59:01.915270+0000","last_fullsized":"2024-05-17T21:59:01.915270+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:55:43.605618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803772+0000","last_change":"2024-05-17T21:58:59.824492+0000","last_active":"2024-05-17T21:59:01.803772+0000","last_peered":"2024-05-17T21:59:01.803772+0000","last_clean":"2024-05-17T21:59:01.803772+0000","last_became_active":"2024-05-17T21:58:59.824137+0000","last_became_peered":"2024-05-17T21:58:59.824137+0000","last_unstale":"2024-05-17T21:59:01.803772+0000","last_undegraded":"2024-05-17T21:59:01.803772+0000","last_fullsized":"2024-05-17T21:59:01.803772+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:37:36.869792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803451+0000","last_change":"2024-05-17T21:59:00.817914+0000","last_active":"2024-05-17T21:59:01.803451+0000","last_peered":"2024-05-17T21:59:01.803451+0000","last_clean":"2024-05-17T21:59:01.803451+0000","last_became_active":"2024-05-17T21:59:00.817638+0000","last_became_peered":"2024-05-17T21:59:00.817638+0000","last_unstale":"2024-05-17T21:59:01.803451+0000","last_undegraded":"2024-05-17T21:59:01.803451+0000","last_fullsized":"2024-05-17T21:59:01.803451+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:53:17.384764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.804544+0000","last_change":"2024-05-17T21:58:59.828232+0000","last_active":"2024-05-17T21:59:01.804544+0000","last_peered":"2024-05-17T21:59:01.804544+0000","last_clean":"2024-05-17T21:59:01.804544+0000","last_became_active":"2024-05-17T21:58:59.827911+0000","last_became_peered":"2024-05-17T21:58:59.827911+0000","last_unstale":"2024-05-17T21:59:01.804544+0000","last_undegraded":"2024-05-17T21:59:01.804544+0000","last_fullsized":"2024-05-17T21:59:01.804544+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:06:09.421729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912251+0000","last_change":"2024-05-17T21:59:00.844398+0000","last_active":"2024-05-17T21:59:01.912251+0000","last_peered":"2024-05-17T21:59:01.912251+0000","last_clean":"2024-05-17T21:59:01.912251+0000","last_became_active":"2024-05-17T21:59:00.844007+0000","last_became_peered":"2024-05-17T21:59:00.844007+0000","last_unstale":"2024-05-17T21:59:01.912251+0000","last_undegraded":"2024-05-17T21:59:01.912251+0000","last_fullsized":"2024-05-17T21:59:01.912251+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:57.881187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799203+0000","last_change":"2024-05-17T21:58:59.825475+0000","last_active":"2024-05-17T21:59:01.799203+0000","last_peered":"2024-05-17T21:59:01.799203+0000","last_clean":"2024-05-17T21:59:01.799203+0000","last_became_active":"2024-05-17T21:58:59.824851+0000","last_became_peered":"2024-05-17T21:58:59.824851+0000","last_unstale":"2024-05-17T21:59:01.799203+0000","last_undegraded":"2024-05-17T21:59:01.799203+0000","last_fullsized":"2024-05-17T21:59:01.799203+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:18.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.795917+0000","last_change":"2024-05-17T21:59:00.826757+0000","last_active":"2024-05-17T21:59:01.795917+0000","last_peered":"2024-05-17T21:59:01.795917+0000","last_clean":"2024-05-17T21:59:01.795917+0000","last_became_active":"2024-05-17T21:59:00.826232+0000","last_became_peered":"2024-05-17T21:59:00.826232+0000","last_unstale":"2024-05-17T21:59:01.795917+0000","last_undegraded":"2024-05-17T21:59:01.795917+0000","last_fullsized":"2024-05-17T21:59:01.795917+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:27.475671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910017+0000","last_change":"2024-05-17T21:58:59.826092+0000","last_active":"2024-05-17T21:59:01.910017+0000","last_peered":"2024-05-17T21:59:01.910017+0000","last_clean":"2024-05-17T21:59:01.910017+0000","last_became_active":"2024-05-17T21:58:59.825560+0000","last_became_peered":"2024-05-17T21:58:59.825560+0000","last_unstale":"2024-05-17T21:59:01.910017+0000","last_undegraded":"2024-05-17T21:59:01.910017+0000","last_fullsized":"2024-05-17T21:59:01.910017+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:53:09.874473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910032+0000","last_change":"2024-05-17T21:59:00.825316+0000","last_active":"2024-05-17T21:59:01.910032+0000","last_peered":"2024-05-17T21:59:01.910032+0000","last_clean":"2024-05-17T21:59:01.910032+0000","last_became_active":"2024-05-17T21:59:00.824633+0000","last_became_peered":"2024-05-17T21:59:00.824633+0000","last_unstale":"2024-05-17T21:59:01.910032+0000","last_undegraded":"2024-05-17T21:59:01.910032+0000","last_fullsized":"2024-05-17T21:59:01.910032+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:59.546842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911478+0000","last_change":"2024-05-17T21:58:59.827621+0000","last_active":"2024-05-17T21:59:01.911478+0000","last_peered":"2024-05-17T21:59:01.911478+0000","last_clean":"2024-05-17T21:59:01.911478+0000","last_became_active":"2024-05-17T21:58:59.826221+0000","last_became_peered":"2024-05-17T21:58:59.826221+0000","last_unstale":"2024-05-17T21:59:01.911478+0000","last_undegraded":"2024-05-17T21:59:01.911478+0000","last_fullsized":"2024-05-17T21:59:01.911478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:18.630947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.915981+0000","last_change":"2024-05-17T21:59:00.832815+0000","last_active":"2024-05-17T21:59:01.915981+0000","last_peered":"2024-05-17T21:59:01.915981+0000","last_clean":"2024-05-17T21:59:01.915981+0000","last_became_active":"2024-05-17T21:59:00.832068+0000","last_became_peered":"2024-05-17T21:59:00.832068+0000","last_unstale":"2024-05-17T21:59:01.915981+0000","last_undegraded":"2024-05-17T21:59:01.915981+0000","last_fullsized":"2024-05-17T21:59:01.915981+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:26:19.635084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.804473+0000","last_change":"2024-05-17T21:58:59.816660+0000","last_active":"2024-05-17T21:59:01.804473+0000","last_peered":"2024-05-17T21:59:01.804473+0000","last_clean":"2024-05-17T21:59:01.804473+0000","last_became_active":"2024-05-17T21:58:59.816171+0000","last_became_peered":"2024-05-17T21:58:59.816171+0000","last_unstale":"2024-05-17T21:59:01.804473+0000","last_undegraded":"2024-05-17T21:59:01.804473+0000","last_fullsized":"2024-05-17T21:59:01.804473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:19:28.178832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.910713+0000","last_change":"2024-05-17T21:59:00.825421+0000","last_active":"2024-05-17T21:59:01.910713+0000","last_peered":"2024-05-17T21:59:01.910713+0000","last_clean":"2024-05-17T21:59:01.910713+0000","last_became_active":"2024-05-17T21:59:00.825060+0000","last_became_peered":"2024-05-17T21:59:00.825060+0000","last_unstale":"2024-05-17T21:59:01.910713+0000","last_undegraded":"2024-05-17T21:59:01.910713+0000","last_fullsized":"2024-05-17T21:59:01.910713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:19:29.182968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912603+0000","last_change":"2024-05-17T21:58:59.828235+0000","last_active":"2024-05-17T21:59:01.912603+0000","last_peered":"2024-05-17T21:59:01.912603+0000","last_clean":"2024-05-17T21:59:01.912603+0000","last_became_active":"2024-05-17T21:58:59.827551+0000","last_became_peered":"2024-05-17T21:58:59.827551+0000","last_unstale":"2024-05-17T21:59:01.912603+0000","last_undegraded":"2024-05-17T21:59:01.912603+0000","last_fullsized":"2024-05-17T21:59:01.912603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:33:39.660424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912466+0000","last_change":"2024-05-17T21:59:00.843721+0000","last_active":"2024-05-17T21:59:01.912466+0000","last_peered":"2024-05-17T21:59:01.912466+0000","last_clean":"2024-05-17T21:59:01.912466+0000","last_became_active":"2024-05-17T21:59:00.842133+0000","last_became_peered":"2024-05-17T21:59:00.842133+0000","last_unstale":"2024-05-17T21:59:01.912466+0000","last_undegraded":"2024-05-17T21:59:01.912466+0000","last_fullsized":"2024-05-17T21:59:01.912466+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:00:21.530177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912247+0000","last_change":"2024-05-17T21:58:59.819481+0000","last_active":"2024-05-17T21:59:01.912247+0000","last_peered":"2024-05-17T21:59:01.912247+0000","last_clean":"2024-05-17T21:59:01.912247+0000","last_became_active":"2024-05-17T21:58:59.819228+0000","last_became_peered":"2024-05-17T21:58:59.819228+0000","last_unstale":"2024-05-17T21:59:01.912247+0000","last_undegraded":"2024-05-17T21:59:01.912247+0000","last_fullsized":"2024-05-17T21:59:01.912247+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:59:39.726662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.803886+0000","last_change":"2024-05-17T21:59:00.816996+0000","last_active":"2024-05-17T21:59:01.803886+0000","last_peered":"2024-05-17T21:59:01.803886+0000","last_clean":"2024-05-17T21:59:01.803886+0000","last_became_active":"2024-05-17T21:59:00.816496+0000","last_became_peered":"2024-05-17T21:59:00.816496+0000","last_unstale":"2024-05-17T21:59:01.803886+0000","last_undegraded":"2024-05-17T21:59:01.803886+0000","last_fullsized":"2024-05-17T21:59:01.803886+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:21:53.280519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.912492+0000","last_change":"2024-05-17T21:58:59.827757+0000","last_active":"2024-05-17T21:59:01.912492+0000","last_peered":"2024-05-17T21:59:01.912492+0000","last_clean":"2024-05-17T21:59:01.912492+0000","last_became_active":"2024-05-17T21:58:59.826716+0000","last_became_peered":"2024-05-17T21:58:59.826716+0000","last_unstale":"2024-05-17T21:59:01.912492+0000","last_undegraded":"2024-05-17T21:59:01.912492+0000","last_fullsized":"2024-05-17T21:59:01.912492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:56.877051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.804201+0000","last_change":"2024-05-17T21:59:00.820582+0000","last_active":"2024-05-17T21:59:01.804201+0000","last_peered":"2024-05-17T21:59:01.804201+0000","last_clean":"2024-05-17T21:59:01.804201+0000","last_became_active":"2024-05-17T21:59:00.819505+0000","last_became_peered":"2024-05-17T21:59:00.819505+0000","last_unstale":"2024-05-17T21:59:01.804201+0000","last_undegraded":"2024-05-17T21:59:01.804201+0000","last_fullsized":"2024-05-17T21:59:01.804201+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:16:29.632031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.796413+0000","last_change":"2024-05-17T21:58:59.807625+0000","last_active":"2024-05-17T21:59:01.796413+0000","last_peered":"2024-05-17T21:59:01.796413+0000","last_clean":"2024-05-17T21:59:01.796413+0000","last_became_active":"2024-05-17T21:58:59.807136+0000","last_became_peered":"2024-05-17T21:58:59.807136+0000","last_unstale":"2024-05-17T21:59:01.796413+0000","last_undegraded":"2024-05-17T21:59:01.796413+0000","last_fullsized":"2024-05-17T21:59:01.796413+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:34.453280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.799347+0000","last_change":"2024-05-17T21:59:00.816292+0000","last_active":"2024-05-17T21:59:01.799347+0000","last_peered":"2024-05-17T21:59:01.799347+0000","last_clean":"2024-05-17T21:59:01.799347+0000","last_became_active":"2024-05-17T21:59:00.815506+0000","last_became_peered":"2024-05-17T21:59:00.815506+0000","last_unstale":"2024-05-17T21:59:01.799347+0000","last_undegraded":"2024-05-17T21:59:01.799347+0000","last_fullsized":"2024-05-17T21:59:01.799347+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:19:29.182968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911137+0000","last_change":"2024-05-17T21:58:59.822755+0000","last_active":"2024-05-17T21:59:01.911137+0000","last_peered":"2024-05-17T21:59:01.911137+0000","last_clean":"2024-05-17T21:59:01.911137+0000","last_became_active":"2024-05-17T21:58:59.822472+0000","last_became_peered":"2024-05-17T21:58:59.822472+0000","last_unstale":"2024-05-17T21:59:01.911137+0000","last_undegraded":"2024-05-17T21:59:01.911137+0000","last_fullsized":"2024-05-17T21:59:01.911137+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:41:05.685239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.795922+0000","last_change":"2024-05-17T21:59:00.822256+0000","last_active":"2024-05-17T21:59:01.795922+0000","last_peered":"2024-05-17T21:59:01.795922+0000","last_clean":"2024-05-17T21:59:01.795922+0000","last_became_active":"2024-05-17T21:59:00.821305+0000","last_became_peered":"2024-05-17T21:59:00.821305+0000","last_unstale":"2024-05-17T21:59:01.795922+0000","last_undegraded":"2024-05-17T21:59:01.795922+0000","last_fullsized":"2024-05-17T21:59:01.795922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:25:41.904376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.909989+0000","last_change":"2024-05-17T21:58:59.805448+0000","last_active":"2024-05-17T21:59:01.909989+0000","last_peered":"2024-05-17T21:59:01.909989+0000","last_clean":"2024-05-17T21:59:01.909989+0000","last_became_active":"2024-05-17T21:58:59.805171+0000","last_became_peered":"2024-05-17T21:58:59.805171+0000","last_unstale":"2024-05-17T21:59:01.909989+0000","last_undegraded":"2024-05-17T21:59:01.909989+0000","last_fullsized":"2024-05-17T21:59:01.909989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:15:20.937688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911073+0000","last_change":"2024-05-17T21:59:00.829254+0000","last_active":"2024-05-17T21:59:01.911073+0000","last_peered":"2024-05-17T21:59:01.911073+0000","last_clean":"2024-05-17T21:59:01.911073+0000","last_became_active":"2024-05-17T21:59:00.829031+0000","last_became_peered":"2024-05-17T21:59:00.829031+0000","last_unstale":"2024-05-17T21:59:01.911073+0000","last_undegraded":"2024-05-17T21:59:01.911073+0000","last_fullsized":"2024-05-17T21:59:01.911073+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:59.774319+0000","last_clean_scrub_stamp":"2024-05-17T21:58:59.774319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:06:10.425866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.800448+0000","last_change":"2024-05-17T21:58:59.815582+0000","last_active":"2024-05-17T21:59:01.800448+0000","last_peered":"2024-05-17T21:59:01.800448+0000","last_clean":"2024-05-17T21:59:01.800448+0000","last_became_active":"2024-05-17T21:58:59.814392+0000","last_became_peered":"2024-05-17T21:58:59.814392+0000","last_unstale":"2024-05-17T21:59:01.800448+0000","last_undegraded":"2024-05-17T21:59:01.800448+0000","last_fullsized":"2024-05-17T21:59:01.800448+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:58:58.770182+0000","last_clean_scrub_stamp":"2024-05-17T21:58:58.770182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:27:34.665957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":151,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-05-17T21:59:01.911576+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:59:01.911576+0000","last_peered":"2024-05-17T21:59:01.911576+0000","last_clean":"2024-05-17T21:59:01.911576+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:59:01.911576+0000","last_undegraded":"2024-05-17T21:59:01.911576+0000","last_fullsized":"2024-05-17T21:59:01.911576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":793091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":203305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":202703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":204245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":203271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":204267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":202703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":793035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T21:59:05.298 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T21:59:05.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T21:59:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.497+0000 7f224dc71640 1 Processor -- start 2024-05-17T21:59:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.497+0000 7f224dc71640 1 -- start start 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f224dc71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f22480d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f224dc71640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22480d10d0 con 0x7f2248143f20 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f224dc71640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2248070120 con 0x7f22480b3d00 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f224dc71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22480b4110 con 0x7f2248073590 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2247fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2248143f20 0x7f22481447f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41670/0 (socket says 172.21.15.83:41670) 2024-05-17T21:59:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2247fff640 1 -- 172.21.15.83:0/1523293262 learned_addr learned my addr 172.21.15.83:0/1523293262 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2246ffd640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f22480d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788116790 0 0) 0x7f22480d10d0 con 0x7f2248143f20 2024-05-17T21:59:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2230003530 con 0x7f2248143f20 2024-05-17T21:59:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3211796701 0 0) 0x7f2230003530 con 0x7f2248143f20 2024-05-17T21:59:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 msgr2=0x7f22480d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f22480d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 >> v1:172.21.15.155:6789/0 conn(0x7f22480b3d00 legacy=0x7f2248072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248070400 con 0x7f2248143f20 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2240001a80 con 0x7f2248143f20 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 >> v1:172.21.15.83:6789/0 conn(0x7f2248143f20 legacy=0x7f22481447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f2230003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f2230004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230004d50 0x7f2230005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22480b4110 con 0x7f2230003a40 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f223000bbc0 con 0x7f2230004d50 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f223000bdc0 con 0x7f2248073590 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f2230003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2246ffd640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f2230004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2247fff640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230004d50 0x7f2230005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 -- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 msgr2=0x7f2230004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248073590 0x7f2230004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 -- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230004d50 msgr2=0x7f2230005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230004d50 0x7f2230005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f22477fe640 1 -- 172.21.15.83:0/1523293262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2230003530 con 0x7f2230003a40 2024-05-17T21:59:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.501+0000 7f2247fff640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2230004d50 0x7f2230005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f22477fe640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f2230003eb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2238001790 tx=0x7f223801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ed48bfd6d1a1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380013e0 con 0x7f2230003a40 2024-05-17T21:59:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22380050d0 con 0x7f2230003a40 2024-05-17T21:59:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f22467fc640 1 -- 172.21.15.83:0/1523293262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380054c0 con 0x7f2230003a40 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- 172.21.15.83:0/1523293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 msgr2=0x7f2230003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 --2- 172.21.15.83:0/1523293262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f2230003eb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2238001790 tx=0x7f223801c3f0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f2247fff640 1 -- 172.21.15.83:0/1523293262 reap_dead start 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- 172.21.15.83:0/1523293262 shutdown_connections 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- 172.21.15.83:0/1523293262 >> 172.21.15.83:0/1523293262 conn(0x7f224806cd90 msgr2=0x7f22480b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- 172.21.15.83:0/1523293262 shutdown_connections 2024-05-17T21:59:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- 172.21.15.83:0/1523293262 wait complete. 2024-05-17T21:59:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 Processor -- start 2024-05-17T21:59:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.505+0000 7f224dc71640 1 -- start start 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 0x7f22481600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 0x7f2248162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2248070c00 con 0x7f2230003a40 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f22480b45c0 con 0x7f2248163b80 2024-05-17T21:59:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2248163340 con 0x7f2248160610 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60468/0 (socket says 172.21.15.83:60468) 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 -- 172.21.15.83:0/2738960445 learned_addr learned my addr 172.21.15.83:0/2738960445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f2247fff640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 0x7f2248162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f2246ffd640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 0x7f22481600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 -- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 msgr2=0x7f2248162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 0x7f2248162620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 -- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 msgr2=0x7f22481600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 0x7f22481600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 -- 172.21.15.83:0/2738960445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224815e420 con 0x7f2230003a40 2024-05-17T21:59:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f2247fff640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 0x7f2248162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f2246ffd640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 0x7f22481600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f22477fe640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2238001790 tx=0x7f2230005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380054a0 con 0x7f2230003a40 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22380be070 con 0x7f2230003a40 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224815e700 con 0x7f2230003a40 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380013c0 con 0x7f2230003a40 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224815ecc0 con 0x7f2230003a40 2024-05-17T21:59:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2248072e80 con 0x7f2230003a40 2024-05-17T21:59:05.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.509+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22380c3460 con 0x7f2230003a40 2024-05-17T21:59:05.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.513+0000 7f222ffff640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 0x7f2234071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.513+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2238129db0 con 0x7f2230003a40 2024-05-17T21:59:05.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.513+0000 7f2246ffd640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 0x7f2234071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.513+0000 7f2246ffd640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 0x7f2234071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2248146430 tx=0x7f223c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.517+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2238016150 con 0x7f2230003a40 2024-05-17T21:59:05.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.737+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2248146a20 con 0x7f2230003a40 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f222ffff640 1 -- 172.21.15.83:0/2738960445 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f22380f6ea0 con 0x7f2230003a40 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T21:59:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T21:59:05.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 msgr2=0x7f2234071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 0x7f2234071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2248146430 tx=0x7f223c008040 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 msgr2=0x7f224815f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2238001790 tx=0x7f2230005840 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 shutdown_connections 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2248160610 0x7f2248162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f223406f400 0x7f2234071910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2248163b80 0x7f22481600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 --2- 172.21.15.83:0/2738960445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2230003a40 0x7f224815f4c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 >> 172.21.15.83:0/2738960445 conn(0x7f224806cd90 msgr2=0x7f22481459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 shutdown_connections 2024-05-17T21:59:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.745+0000 7f224dc71640 1 -- 172.21.15.83:0/2738960445 wait complete. 2024-05-17T21:59:05.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 Processor -- start 2024-05-17T21:59:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 -- start start 2024-05-17T21:59:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f07740d1040 0x7f0774144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0774073540 0x7f077406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077415ea40 con 0x7f077415e080 2024-05-17T21:59:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f077415ed20 con 0x7f0774073540 2024-05-17T21:59:05.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0779d42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f077415f000 con 0x7f07740d1040 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0774073540 0x7f077406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f076bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f077415e080 0x7f0774072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41680/0 (socket says 172.21.15.83:41680) 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0774073540 0x7f077406f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60470/0 (socket says 172.21.15.83:60470) 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f076bfff640 1 -- 172.21.15.83:0/1455699246 learned_addr learned my addr 172.21.15.83:0/1455699246 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f07737fe640 1 --2- 172.21.15.83:0/1455699246 >> v2:172.21.15.155:3300/0 conn(0x7f07740d1040 0x7f0774144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 -- 172.21.15.83:0/1455699246 >> v2:172.21.15.155:3300/0 conn(0x7f07740d1040 msgr2=0x7f0774144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 --2- 172.21.15.83:0/1455699246 >> v2:172.21.15.155:3300/0 conn(0x7f07740d1040 0x7f0774144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 -- 172.21.15.83:0/1455699246 >> v1:172.21.15.83:6789/0 conn(0x7f077415e080 legacy=0x7f0774072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0773fff640 1 -- 172.21.15.83:0/1455699246 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077406ef30 con 0x7f0774073540 2024-05-17T21:59:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.921+0000 7f0772ffd640 1 -- 172.21.15.83:0/1455699246 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999006383 0 0) 0x7f077415ea40 con 0x7f077415e080 2024-05-17T21:59:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0773fff640 1 --2- 172.21.15.83:0/1455699246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774073540 0x7f077406f730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0774071d60 tx=0x7f076c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329aab58c664c9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0772ffd640 1 -- 172.21.15.83:0/1455699246 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0b2070 con 0x7f0774073540 2024-05-17T21:59:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0772ffd640 1 -- 172.21.15.83:0/1455699246 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c0a5900 con 0x7f0774073540 2024-05-17T21:59:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0772ffd640 1 -- 172.21.15.83:0/1455699246 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0a5cd0 con 0x7f0774073540 2024-05-17T21:59:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- 172.21.15.83:0/1455699246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774073540 msgr2=0x7f077406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 --2- 172.21.15.83:0/1455699246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774073540 0x7f077406f730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0774071d60 tx=0x7f076c01c930 comp rx=0 tx=0).stop 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- 172.21.15.83:0/1455699246 shutdown_connections 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 --2- 172.21.15.83:0/1455699246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774073540 0x7f077406f730 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 --2- 172.21.15.83:0/1455699246 >> v2:172.21.15.155:3300/0 conn(0x7f07740d1040 0x7f0774144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- 172.21.15.83:0/1455699246 >> 172.21.15.83:0/1455699246 conn(0x7f077406cd90 msgr2=0x7f07740b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- 172.21.15.83:0/1455699246 shutdown_connections 2024-05-17T21:59:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- 172.21.15.83:0/1455699246 wait complete. 2024-05-17T21:59:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 Processor -- start 2024-05-17T21:59:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.925+0000 7f0779d42640 1 -- start start 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07740d1040 0x7f077415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077415cdf0 0x7f077415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0774070810 con 0x7f077415e080 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0774070690 con 0x7f077415cdf0 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0774070990 con 0x7f07740d1040 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60486/0 (socket says 172.21.15.83:60486) 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 -- 172.21.15.83:0/2131268082 learned_addr learned my addr 172.21.15.83:0/2131268082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f07737fe640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07740d1040 0x7f077415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 -- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07740d1040 msgr2=0x7f077415c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07740d1040 0x7f077415c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 -- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077415cdf0 msgr2=0x7f077415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0773fff640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077415cdf0 0x7f077415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077415cdf0 0x7f077415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 -- 172.21.15.83:0/2131268082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077415d740 con 0x7f077415e080 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f076bfff640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c3c0 tx=0x7f075c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0074e0 con 0x7f077415e080 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c0047f0 con 0x7f077415e080 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077415de10 con 0x7f077415e080 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c004be0 con 0x7f077415e080 2024-05-17T21:59:05.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07740ad900 con 0x7f077415e080 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077406f730 con 0x7f077415e080 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.929+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f075c011070 con 0x7f077415e080 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.933+0000 7f0770ff9640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 0x7f0754071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.933+0000 7f07737fe640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 0x7f0754071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.933+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f075c08ee00 con 0x7f077415e080 2024-05-17T21:59:05.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.933+0000 7f07737fe640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 0x7f0754071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07640096f0 tx=0x7f0764008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:05.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:05.937+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f075c05bcf0 con 0x7f077415e080 2024-05-17T21:59:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.133+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07740b20a0 con 0x7f077415e080 2024-05-17T21:59:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.133+0000 7f0770ff9640 1 -- 172.21.15.83:0/2131268082 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f075c05b5f0 con 0x7f077415e080 2024-05-17T21:59:06.128 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:06.128 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:06.128 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 msgr2=0x7f0754071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 0x7f0754071940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07640096f0 tx=0x7f0764008040 comp rx=0 tx=0).stop 2024-05-17T21:59:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 msgr2=0x7f077415c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c3c0 tx=0x7f075c00c880 comp rx=0 tx=0).stop 2024-05-17T21:59:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 shutdown_connections 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f075406f430 0x7f0754071940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f077415cdf0 0x7f077415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077415e080 0x7f077415c8b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 --2- 172.21.15.83:0/2131268082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07740d1040 0x7f077415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 >> 172.21.15.83:0/2131268082 conn(0x7f077406cd90 msgr2=0x7f07740b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 shutdown_connections 2024-05-17T21:59:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.137+0000 7f0779d42640 1 -- 172.21.15.83:0/2131268082 wait complete. 2024-05-17T21:59:06.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T21:59:00.239967+0000', 'modified': '2024-05-17T21:59:03.215524+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5157}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5157': {'gid': 5157, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2531920228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 2531920228}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2531920228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5157, 'qdb_cluster': [5157]} 2024-05-17T21:59:06.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T21:59:06.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T21:59:06.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 Processor -- start 2024-05-17T21:59:06.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 -- start start 2024-05-17T21:59:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bfc0cdf40 0x7f8bfc15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bfc0b5640 con 0x7f8bfc0cf830 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bfc0b58e0 con 0x7f8bfc15d530 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8c02411640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8bfc070870 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8bfc0cf830 0x7f8bfc0b3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41686/0 (socket says 172.21.15.83:41686) 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfbfff640 1 -- 172.21.15.83:0/1586756561 learned_addr learned my addr 172.21.15.83:0/1586756561 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/1586756561 >> v2:172.21.15.83:3300/0 conn(0x7f8bfc0cdf40 0x7f8bfc15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfaffd640 1 -- 172.21.15.83:0/1586756561 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862305210 0 0) 0x7f8bfc0b5640 con 0x7f8bfc0cf830 2024-05-17T21:59:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfaffd640 1 -- 172.21.15.83:0/1586756561 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be8003530 con 0x7f8bfc0cf830 2024-05-17T21:59:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1586756561 >> v1:172.21.15.155:6789/0 conn(0x7f8bfc15d530 legacy=0x7f8bfc1663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1586756561 >> v1:172.21.15.83:6789/0 conn(0x7f8bfc0cf830 legacy=0x7f8bfc0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.717+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1586756561 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc06f060 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/1586756561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc15cff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aaad1121e812ab server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8bfaffd640 1 -- 172.21.15.83:0/1586756561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40b1070 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8bfaffd640 1 -- 172.21.15.83:0/1586756561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be40ac410 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8bfaffd640 1 -- 172.21.15.83:0/1586756561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40ac7e0 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 -- 172.21.15.83:0/1586756561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 msgr2=0x7f8bfc15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 --2- 172.21.15.83:0/1586756561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc15cff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401c4a0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 -- 172.21.15.83:0/1586756561 shutdown_connections 2024-05-17T21:59:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 --2- 172.21.15.83:0/1586756561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc15cff0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 -- 172.21.15.83:0/1586756561 >> 172.21.15.83:0/1586756561 conn(0x7f8bfc06cd90 msgr2=0x7f8bfc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 -- 172.21.15.83:0/1586756561 shutdown_connections 2024-05-17T21:59:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.721+0000 7f8c02411640 1 -- 172.21.15.83:0/1586756561 wait complete. 2024-05-17T21:59:06.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 Processor -- start 2024-05-17T21:59:06.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- start start 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0cf830 0x7f8bfc152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc15d530 0x7f8bfc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc16aab0 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc16a930 con 0x7f8bfc15d530 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc16ac30 con 0x7f8bfc0cf830 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60512/0 (socket says 172.21.15.83:60512) 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 -- 172.21.15.83:0/2482051184 learned_addr learned my addr 172.21.15.83:0/2482051184 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 -- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0cf830 msgr2=0x7f8bfc152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0cf830 0x7f8bfc152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c009ae640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc15d530 0x7f8bfc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 -- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc15d530 msgr2=0x7f8bfc15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc15d530 0x7f8bfc15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 -- 172.21.15.83:0/2482051184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc151a00 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c420 tx=0x7f8bf400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/2482051184 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017070 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/2482051184 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf4005cf0 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc151ca0 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/2482051184 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40072d0 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.725+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc152260 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/2482051184 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bf401a070 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bf8ff9640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 0x7f8bdc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bf8ff9640 1 -- 172.21.15.83:0/2482051184 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8bf408e620 con 0x7f8bfc0cdf40 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 0x7f8bdc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 0x7f8bdc071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be4004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 Processor -- start 2024-05-17T21:59:06.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 -- start start 2024-05-17T21:59:06.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 0x56082233e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x56082233f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608221dcca0 con 0x5608221ee600 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56082214f110 con 0x5608221f8fa0 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56082213ecb0 con 0x5608221d7d30 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60526/0 (socket says 172.21.15.83:60526) 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3912409077 learned_addr learned my addr 172.21.15.83:0/3912409077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 0x56082233e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8c009ae640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x56082233f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 msgr2=0x56082233f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x56082233f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 msgr2=0x56082233e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 0x56082233e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3912409077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560822103370 con 0x5608221ee600 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 0x56082233e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a622e305e3cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.729+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/3912409077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40c7070 con 0x5608221ee600 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/3912409077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be40b7450 con 0x5608221ee600 2024-05-17T21:59:06.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bf1ffb640 1 -- 172.21.15.83:0/3912409077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40b1040 con 0x5608221ee600 2024-05-17T21:59:06.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- 172.21.15.83:0/3912409077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 msgr2=0x56082233edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be40a7000 comp rx=0 tx=0).stop 2024-05-17T21:59:06.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- 172.21.15.83:0/3912409077 shutdown_connections 2024-05-17T21:59:06.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x56082233f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x56082233edf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- 172.21.15.83:0/3912409077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221f8fa0 0x56082233e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- 172.21.15.83:0/3912409077 >> 172.21.15.83:0/3912409077 conn(0x560821e00460 msgr2=0x5608222f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- 172.21.15.83:0/3912409077 shutdown_connections 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- 172.21.15.83:0/3912409077 wait complete. 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- start start 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 Processor -- start 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x5608222fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221f8fa0 0x560822300460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608220fad40 con 0x5608221ee600 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560822142cf0 con 0x5608221d7d30 2024-05-17T21:59:06.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5608220e7e90 con 0x5608221f8fa0 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c0968b000 10 client.0 _ll_register_callbacks cb 0x56082211cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60540/0 (socket says 172.21.15.83:60540) 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1972553058 learned_addr learned my addr 172.21.15.83:0/1972553058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x5608222fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8c009ae640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221f8fa0 0x560822300460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221f8fa0 msgr2=0x560822300460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221f8fa0 0x560822300460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 msgr2=0x5608222fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x5608222fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/1972553058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082210d6e0 con 0x5608221ee600 2024-05-17T21:59:06.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x5608222fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cabe545c0047de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.733+0000 7f8bdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40c7040 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8be40a9040 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be40b7450 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8be40d3050 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 10 client.5304 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5608220fd690 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be40b7450 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8be40d6020 con 0x5608221ee600 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 10 client.5304 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:06.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 10 client.5304 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 20 client.5304 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821de57d0 0x560822345920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 4 client.5304 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608220daa00 con 0x560821de57d0 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821de57d0 0x560822345920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821de57d0 0x560822345920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d55d25385703870 server_cookie=b2ef87424b76e575 in_seq=0 out_seq=0 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 10 client.5304 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 1 -- 172.21.15.83:0/1972553058 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5608220daa00 con 0x560821de57d0 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 10 client.5304 handle_command_reply: tid=0 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 client.5304 shutdown 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 msgr2=0x5608222fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8be40097c0 tx=0x7f8be401ace0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 shutdown_connections 2024-05-17T21:59:06.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821de57d0 0x560822345920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221f8fa0 0x560822300460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221ee600 0x5608222fb7c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 --2- 172.21.15.83:0/1972553058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x5608222fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8bdaffd640 0 client.5304 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 >> 172.21.15.83:0/1972553058 conn(0x560821e00460 msgr2=0x5608222f3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 shutdown_connections 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 1 -- 172.21.15.83:0/1972553058 wait complete. 2024-05-17T21:59:06.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.737+0000 7f8c0968b000 20 client.5304 trim_cache size 0 max 16384 2024-05-17T21:59:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 Processor -- start 2024-05-17T21:59:06.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- start start 2024-05-17T21:59:06.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x560822355eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221ee600 0x5608223563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc0af560 con 0x5608221f8fa0 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc0ae8c0 con 0x5608221d7d30 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc0aef10 con 0x5608221ee600 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60546/0 (socket says 172.21.15.83:60546) 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 -- 172.21.15.83:0/914135305 learned_addr learned my addr 172.21.15.83:0/914135305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 -- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221ee600 msgr2=0x5608223563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221ee600 0x5608223563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 -- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 msgr2=0x560822355eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x560822355eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 -- 172.21.15.83:0/914135305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608222f7e90 con 0x5608221f8fa0 2024-05-17T21:59:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c009ae640 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009d10 tx=0x7f8bec00e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601102827934ea5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8bf17fa640 1 -- 172.21.15.83:0/914135305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec016070 con 0x5608221f8fa0 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8bf17fa640 1 -- 172.21.15.83:0/914135305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x5608221f8fa0 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8bf17fa640 1 -- 172.21.15.83:0/914135305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec012880 con 0x5608221f8fa0 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- 172.21.15.83:0/914135305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 msgr2=0x5608222fb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009d10 tx=0x7f8bec00e620 comp rx=0 tx=0).stop 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- 172.21.15.83:0/914135305 shutdown_connections 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x5608222fb400 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221ee600 0x5608223563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 --2- 172.21.15.83:0/914135305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221d7d30 0x560822355eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- 172.21.15.83:0/914135305 >> 172.21.15.83:0/914135305 conn(0x560821e00460 msgr2=0x5608222faae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- 172.21.15.83:0/914135305 shutdown_connections 2024-05-17T21:59:06.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.757+0000 7f8c0968b000 1 -- 172.21.15.83:0/914135305 wait complete. 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- start start 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 Processor -- start 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x5608222f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 0x5608222f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560822142000 con 0x5608221f8fa0 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5608220e9900 con 0x5608221ee600 2024-05-17T21:59:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56082214dee0 con 0x5608221d7d30 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 10 client.0 _ll_register_callbacks cb 0x7f8be40c5070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60558/0 (socket says 172.21.15.83:60558) 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 -- 172.21.15.83:0/2428262923 learned_addr learned my addr 172.21.15.83:0/2428262923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x5608222f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 0x5608222f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 -- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 msgr2=0x5608222f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x5608222f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 -- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 msgr2=0x5608222f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 0x5608222f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 -- 172.21.15.83:0/2428262923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608221c5cb0 con 0x5608221f8fa0 2024-05-17T21:59:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x5608222f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 0x5608222f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c009ae640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5608221418b0 tx=0x7f8bec004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4fb8f9d0eecf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec016040 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8bec01b450 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bec012450 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8bec02d050 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 10 client.5310 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5608220fe4f0 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec012450 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8bec030020 con 0x5608221f8fa0 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 10 client.5310 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 10 client.5310 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 20 client.5310 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821dd3830 0x5608223b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 4 client.5310 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "status"}) -- 0x560822151060 con 0x560821dd3830 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821dd3830 0x5608223b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.761+0000 7f8bdaffd640 10 client.5310 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821dd3830 0x5608223b0050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8fb4abc100881 server_cookie=686f24c70ff77f6 in_seq=0 out_seq=0 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8bdaffd640 1 -- 172.21.15.83:0/2428262923 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560822151060 con 0x560821dd3830 2024-05-17T21:59:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8bdaffd640 10 client.5310 handle_command_reply: tid=0 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 client.5310 shutdown 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 msgr2=0x56082236c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5608221418b0 tx=0x7f8bec004a10 comp rx=0 tx=0).stop 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 shutdown_connections 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x560821dd3830 0x5608223b0050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608221f8fa0 0x56082236c800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8bdaffd640 0 client.5310 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608221ee600 0x5608222f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 --2- 172.21.15.83:0/2428262923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5608221d7d30 0x5608222f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 >> 172.21.15.83:0/2428262923 conn(0x560821e00460 msgr2=0x5608222fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 shutdown_connections 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 1 -- 172.21.15.83:0/2428262923 wait complete. 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c0968b000 20 client.5310 trim_cache size 0 max 16384 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5157, 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T21:59:06.764 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.9794019919999997, 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 43, 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 43, 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.300465126000001 2024-05-17T21:59:06.765 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 msgr2=0x7f8bdc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 0x7f8bdc071910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be4004740 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 msgr2=0x7f8bfc1508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c420 tx=0x7f8bf400c8e0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 shutdown_connections 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bdc06f400 0x7f8bdc071910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc15d530 0x7f8bfc15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0cf830 0x7f8bfc152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 --2- 172.21.15.83:0/2482051184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc0cdf40 0x7f8bfc1508c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 >> 172.21.15.83:0/2482051184 conn(0x7f8bfc06cd90 msgr2=0x7f8bfc070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 shutdown_connections 2024-05-17T21:59:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.765+0000 7f8c02411640 1 -- 172.21.15.83:0/2482051184 wait complete. 2024-05-17T21:59:06.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T21:59:06.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 Processor -- start 2024-05-17T21:59:06.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 -- start start 2024-05-17T21:59:06.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b64073540 0x7f7b64168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6415e790 con 0x7f7b640d1040 2024-05-17T21:59:06.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6415ea40 con 0x7f7b64144080 2024-05-17T21:59:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b693d0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7b6415ed20 con 0x7f7b64073540 2024-05-17T21:59:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b637fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b64073540 0x7f7b64168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b62ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7b640d1040 0x7f7b6415e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41698/0 (socket says 172.21.15.83:41698) 2024-05-17T21:59:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b62ffd640 1 -- 172.21.15.83:0/2446951666 learned_addr learned my addr 172.21.15.83:0/2446951666 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:06.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b637fe640 1 -- 172.21.15.83:0/2446951666 >> v1:172.21.15.155:6789/0 conn(0x7f7b64144080 legacy=0x7f7b64072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:06.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b637fe640 1 -- 172.21.15.83:0/2446951666 >> v1:172.21.15.83:6789/0 conn(0x7f7b640d1040 legacy=0x7f7b6415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b61ffb640 1 -- 172.21.15.83:0/2446951666 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861555058 0 0) 0x7f7b6415e790 con 0x7f7b640d1040 2024-05-17T21:59:06.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.925+0000 7f7b637fe640 1 -- 172.21.15.83:0/2446951666 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6415f000 con 0x7f7b64073540 2024-05-17T21:59:06.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b637fe640 1 --2- 172.21.15.83:0/2446951666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b64073540 0x7f7b64168840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b580097c0 tx=0x7f7b5801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe5f0cc62b3b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b61ffb640 1 -- 172.21.15.83:0/2446951666 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b580a7070 con 0x7f7b64073540 2024-05-17T21:59:06.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b61ffb640 1 -- 172.21.15.83:0/2446951666 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b580b0070 con 0x7f7b64073540 2024-05-17T21:59:06.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b61ffb640 1 -- 172.21.15.83:0/2446951666 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b580ab890 con 0x7f7b64073540 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 -- 172.21.15.83:0/2446951666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b64073540 msgr2=0x7f7b64168840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 --2- 172.21.15.83:0/2446951666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b64073540 0x7f7b64168840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b580097c0 tx=0x7f7b5801c470 comp rx=0 tx=0).stop 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 -- 172.21.15.83:0/2446951666 shutdown_connections 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 --2- 172.21.15.83:0/2446951666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b64073540 0x7f7b64168840 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 -- 172.21.15.83:0/2446951666 >> 172.21.15.83:0/2446951666 conn(0x7f7b6406cd90 msgr2=0x7f7b640b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 -- 172.21.15.83:0/2446951666 shutdown_connections 2024-05-17T21:59:06.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.929+0000 7f7b693d0640 1 -- 172.21.15.83:0/2446951666 wait complete. 2024-05-17T21:59:06.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 Processor -- start 2024-05-17T21:59:06.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 -- start start 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b64144080 0x7f7b6413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 0x7f7b6413e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6416b720 con 0x7f7b640d1040 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6416b5a0 con 0x7f7b64146bf0 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6416b8a0 con 0x7f7b64144080 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60570/0 (socket says 172.21.15.83:60570) 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 -- 172.21.15.83:0/966306228 learned_addr learned my addr 172.21.15.83:0/966306228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b637fe640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 0x7f7b6413e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b627fc640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b64144080 0x7f7b6413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 -- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b64144080 msgr2=0x7f7b6413dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b64144080 0x7f7b6413dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 -- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 msgr2=0x7f7b6413e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 0x7f7b6413e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 -- 172.21.15.83:0/966306228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b64136450 con 0x7f7b640d1040 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b637fe640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 0x7f7b6413e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b62ffd640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7a0 tx=0x7f7b5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c018070 con 0x7f7b640d1040 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c00a640 con 0x7f7b640d1040 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b641f9ec0 con 0x7f7b640d1040 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.933+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00aa30 con 0x7f7b640d1040 2024-05-17T21:59:06.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b641f8080 con 0x7f7b640d1040 2024-05-17T21:59:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6415e080 con 0x7f7b640d1040 2024-05-17T21:59:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7b5c01b070 con 0x7f7b640d1040 2024-05-17T21:59:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b4b7fe640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 0x7f7b50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7b5c08e850 con 0x7f7b640d1040 2024-05-17T21:59:06.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b627fc640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 0x7f7b50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:06.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.937+0000 7f7b627fc640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 0x7f7b50071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b640b32b0 tx=0x7f7b54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:06.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:06.941+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7b5c05b740 con 0x7f7b640d1040 2024-05-17T21:59:07.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7b640b2490 con 0x7f7b640d1040 2024-05-17T21:59:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b4b7fe640 1 -- 172.21.15.83:0/966306228 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f7b5c05b040 con 0x7f7b640d1040 2024-05-17T21:59:07.118 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T21:59:07.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 msgr2=0x7f7b50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 0x7f7b50071940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b640b32b0 tx=0x7f7b54008040 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 msgr2=0x7f7b6415d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7a0 tx=0x7f7b5c00cc60 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 shutdown_connections 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7b5006f430 0x7f7b50071940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b64146bf0 0x7f7b6413e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b64144080 0x7f7b6413dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 --2- 172.21.15.83:0/966306228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b640d1040 0x7f7b6415d4e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.125+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 >> 172.21.15.83:0/966306228 conn(0x7f7b6406cd90 msgr2=0x7f7b6413b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.129+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 shutdown_connections 2024-05-17T21:59:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07.129+0000 7f7b693d0640 1 -- 172.21.15.83:0/966306228 wait complete. 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:07.138 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T21:59:07.139 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T21:59:07.139 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T21:59:07.139 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:59:07.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.139 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T21:59:07.145 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T21:59:07.146 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T21:59:07.147 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T21:59:07.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.148 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T21:59:07.197 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T21:59:07.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.198 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:59:07.249 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:59:07.250 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T21:59:07.250 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T21:59:07.250 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T21:59:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T21:59:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:59:07.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.372 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T21:59:07.378 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T21:59:07.378 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T21:59:07.378 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:59:07.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T21:59:07.436 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T21:59:07.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:59:07.437 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:07.490 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T21:59:07.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T21:59:07.491 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:07.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:59:07.599 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:59:07.646 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:59:07.647 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:07.648 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:07.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.649 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:59:07.697 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T21:59:07.697 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T21:59:07.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:59:07.756 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:59:07.804 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:59:07.805 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:59:07.806 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:59:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:59:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:07.807 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:07.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:07.808 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:59:07.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T21:59:07.857+0000 7f9f5e8a4580 -1 init, newargv = 0x7f9f4c005830 newargc=15 2024-05-17T21:59:07.852 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[38007]: starting ceph client 2024-05-17T21:59:07.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[38007]: starting fuse 2024-05-17T21:59:08.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T21:59:08.975 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T21:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T21:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T21:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T21:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T21:59:09.022 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T21:59:09.023 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T21:59:09.024 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:09.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.025 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T21:59:09.073 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T21:59:09.074 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T21:59:09.074 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T21:59:09.075 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T21:59:09.076 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T21:59:09.076 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:59:09.076 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T21:59:09.076 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:59:09.198 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.38007.asok 2024-05-17T21:59:09.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:59:09.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38007.asok 2024-05-17T21:59:09.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38007.asok status 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "38007", 2024-05-17T21:59:09.392 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5328, 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 5328 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3431547517 2024-05-17T21:59:09.393 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3431547517 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.5328 192.168.0.1:0/3431547517", 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/3431547517", 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 23, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 43, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T21:59:09.394 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:09.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.409 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T21:59:09.409 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:09.415 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T21:59:09.415 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:09.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:09.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.469 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-17T21:59:09.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:09.529 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-17T21:59:09.583 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T21:59:09.583 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T21:59:09.583 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T21:59:09.583 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T21:59:09.584 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-17T21:59:09.584 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T21:59:09.695 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T21:59:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T21:59:09.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 Processor -- start 2024-05-17T21:59:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 -- start start 2024-05-17T21:59:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f376c0b5500 0x7f376c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f376c15fbe0 0x7f376c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376c0afa50 con 0x7f376c0b1030 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f376c0afd30 con 0x7f376c15fbe0 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3772ec8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f376c0b0010 con 0x7f376c0b5500 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f376c15fbe0 0x7f376c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3771465640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f376c0b1030 0x7f376c0b1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47252/0 (socket says 172.21.15.83:47252) 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3771465640 1 -- 172.21.15.83:0/301269626 learned_addr learned my addr 172.21.15.83:0/301269626 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f3770c64640 1 --2- 172.21.15.83:0/301269626 >> v2:172.21.15.155:3300/0 conn(0x7f376c0b5500 0x7f376c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376bfff640 1 -- 172.21.15.83:0/301269626 >> v2:172.21.15.155:3300/0 conn(0x7f376c0b5500 msgr2=0x7f376c0af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376bfff640 1 --2- 172.21.15.83:0/301269626 >> v2:172.21.15.155:3300/0 conn(0x7f376c0b5500 0x7f376c0af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376bfff640 1 -- 172.21.15.83:0/301269626 >> v1:172.21.15.83:6789/0 conn(0x7f376c0b1030 legacy=0x7f376c0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376bfff640 1 -- 172.21.15.83:0/301269626 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0b02f0 con 0x7f376c15fbe0 2024-05-17T21:59:09.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.917+0000 7f376b7fe640 1 -- 172.21.15.83:0/301269626 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539960616 0 0) 0x7f376c0afa50 con 0x7f376c0b1030 2024-05-17T21:59:09.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f376bfff640 1 --2- 172.21.15.83:0/301269626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c15fbe0 0x7f376c0b0af0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f376c073ca0 tx=0x7f375c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e853dd66160db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:09.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f376b7fe640 1 -- 172.21.15.83:0/301269626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0b0070 con 0x7f376c15fbe0 2024-05-17T21:59:09.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f376b7fe640 1 -- 172.21.15.83:0/301269626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c0ab600 con 0x7f376c15fbe0 2024-05-17T21:59:09.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f376b7fe640 1 -- 172.21.15.83:0/301269626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0ab9d0 con 0x7f376c15fbe0 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 -- 172.21.15.83:0/301269626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c15fbe0 msgr2=0x7f376c0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 --2- 172.21.15.83:0/301269626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c15fbe0 0x7f376c0b0af0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f376c073ca0 tx=0x7f375c01c3f0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 -- 172.21.15.83:0/301269626 shutdown_connections 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 --2- 172.21.15.83:0/301269626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c15fbe0 0x7f376c0b0af0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 --2- 172.21.15.83:0/301269626 >> v2:172.21.15.155:3300/0 conn(0x7f376c0b5500 0x7f376c0af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 -- 172.21.15.83:0/301269626 >> 172.21.15.83:0/301269626 conn(0x7f376c09b8b0 msgr2=0x7f376c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 -- 172.21.15.83:0/301269626 shutdown_connections 2024-05-17T21:59:09.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.921+0000 7f3772ec8640 1 -- 172.21.15.83:0/301269626 wait complete. 2024-05-17T21:59:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 Processor -- start 2024-05-17T21:59:09.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- start start 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f376c0b1030 0x7f376c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 0x7f376c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f376c160170 con 0x7f376c0b5500 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f376c15fff0 con 0x7f376c15fbe0 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f376c16cac0 con 0x7f376c0b1030 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53060/0 (socket says 172.21.15.83:53060) 2024-05-17T21:59:09.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 -- 172.21.15.83:0/569628494 learned_addr learned my addr 172.21.15.83:0/569628494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3771465640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 0x7f376c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 -- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f376c0b1030 msgr2=0x7f376c1573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3770c64640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f376c0b1030 0x7f376c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f376c0b1030 0x7f376c1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 -- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 msgr2=0x7f376c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 0x7f376c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 -- 172.21.15.83:0/569628494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c0a00c0 con 0x7f376c0b5500 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3771465640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 0x7f376c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f376bfff640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f375c0ae040 tx=0x7f375c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0bc070 con 0x7f376c0b5500 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c0b1040 con 0x7f376c0b5500 2024-05-17T21:59:09.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c0a05d0 con 0x7f376c0b5500 2024-05-17T21:59:09.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0b0050 con 0x7f376c0b5500 2024-05-17T21:59:09.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c09c970 con 0x7f376c0b5500 2024-05-17T21:59:09.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.925+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c0af470 con 0x7f376c0b5500 2024-05-17T21:59:09.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.929+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f375c0bf070 con 0x7f376c0b5500 2024-05-17T21:59:09.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.929+0000 7f37697fa640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:09.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.929+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f375c129390 con 0x7f376c0b5500 2024-05-17T21:59:09.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.929+0000 7f3770c64640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:09.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.929+0000 7f3770c64640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 0x7f3758071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37600096f0 tx=0x7f3760008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:09.933+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f375c016070 con 0x7f376c0b5500 2024-05-17T21:59:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.133+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376c0a0360 con 0x7f376c0b5500 2024-05-17T21:59:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.133+0000 7f37697fa640 1 -- 172.21.15.83:0/569628494 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f375c0f62f0 con 0x7f376c0b5500 2024-05-17T21:59:10.128 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:10.128 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 msgr2=0x7f3758071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 0x7f3758071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f37600096f0 tx=0x7f3760008040 comp rx=0 tx=0).stop 2024-05-17T21:59:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 msgr2=0x7f376c09fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f375c0ae040 tx=0x7f375c01ac30 comp rx=0 tx=0).stop 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 shutdown_connections 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f376c15fbe0 0x7f376c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c0b5500 0x7f376c09fb80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 --2- 172.21.15.83:0/569628494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f376c0b1030 0x7f376c1573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 >> 172.21.15.83:0/569628494 conn(0x7f376c09b8b0 msgr2=0x7f376c159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 shutdown_connections 2024-05-17T21:59:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.137+0000 7f3772ec8640 1 -- 172.21.15.83:0/569628494 wait complete. 2024-05-17T21:59:10.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T21:59:10.341 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T21:59:10.341 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T21:59:10.342 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-17T21:59:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 Processor -- start 2024-05-17T21:59:10.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 -- start start 2024-05-17T21:59:10.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 0x55a5c0bd1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0d2cfb0 0x55a5c0c15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0b1d1b0 con 0x55a5c0bd2390 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0b78ef0 con 0x55a5c0d2cfb0 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee45a6c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a5c0c15030 con 0x55a5c0c17ed0 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53090/0 (socket says 172.21.15.83:53090) 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 -- 172.21.15.83:0/1248832612 learned_addr learned my addr 172.21.15.83:0/1248832612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3f54e640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 0x55a5c0bd1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 -- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 msgr2=0x55a5c0bd1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 0x55a5c0bd1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 -- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0d2cfb0 msgr2=0x55a5c0c15e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3fd4f640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0d2cfb0 0x55a5c0c15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0d2cfb0 0x55a5c0c15e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 -- 172.21.15.83:0/1248832612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c0af7c10 con 0x55a5c0bd2390 2024-05-17T21:59:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3f54e640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 0x55a5c0bd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.417+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee28006f70 tx=0x7fee2800e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4bda7aa8bc3363 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3dd4b640 1 -- 172.21.15.83:0/1248832612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800e2e0 con 0x55a5c0bd2390 2024-05-17T21:59:10.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3dd4b640 1 -- 172.21.15.83:0/1248832612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2800c500 con 0x55a5c0bd2390 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3dd4b640 1 -- 172.21.15.83:0/1248832612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800c8b0 con 0x55a5c0bd2390 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- 172.21.15.83:0/1248832612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 msgr2=0x55a5c0d2ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee28006f70 tx=0x7fee2800e9b0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- 172.21.15.83:0/1248832612 shutdown_connections 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0d2cfb0 0x55a5c0c15e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0d2ca70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1248832612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0c17ed0 0x55a5c0bd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- 172.21.15.83:0/1248832612 >> 172.21.15.83:0/1248832612 conn(0x55a5c0bb9e40 msgr2=0x55a5c0c153f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- 172.21.15.83:0/1248832612 shutdown_connections 2024-05-17T21:59:10.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- 172.21.15.83:0/1248832612 wait complete. 2024-05-17T21:59:10.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 Processor -- start 2024-05-17T21:59:10.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- start start 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0bfb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0c17ed0 0x55a5c0bfbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0d2b630 0x55a5c0bfc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0ca2f40 con 0x55a5c0bd2390 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5c0d2b4b0 con 0x55a5c0c17ed0 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee45a6c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a5c0ca31e0 con 0x55a5c0d2b630 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0bfb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0bfb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53104/0 (socket says 172.21.15.83:53104) 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 -- 172.21.15.83:0/1292509628 learned_addr learned my addr 172.21.15.83:0/1292509628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 -- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0d2b630 msgr2=0x55a5c0bfc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3fd4f640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0d2b630 0x55a5c0bfc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0c17ed0 0x55a5c0bfbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5c0d2b630 0x55a5c0bfc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 -- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0c17ed0 msgr2=0x55a5c0bfbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5c0c17ed0 0x55a5c0bfbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 -- 172.21.15.83:0/1292509628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c0d3ae10 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3f54e640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0bfb6d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fee3800f590 tx=0x7fee3800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3cd49640 1 -- 172.21.15.83:0/1292509628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38017070 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.421+0000 7fee3cd49640 1 -- 172.21.15.83:0/1292509628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee3800a830 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5c0d3c230 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3cd49640 1 -- 172.21.15.83:0/1292509628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800e040 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c0d3c770 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3cd49640 1 -- 172.21.15.83:0/1292509628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee3801a070 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3cd49640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee2006dde0 0x7fee200702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3cd49640 1 -- 172.21.15.83:0/1292509628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fee3808df90 con 0x55a5c0bd2390 2024-05-17T21:59:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee2006dde0 0x7fee200702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55a5c0d3bb70 0x55a5c0d46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x55a5c0d3dc20 con 0x55a5c0d3bb70 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3ed4d640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee2006dde0 0x7fee200702f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee28006e60 tx=0x7fee28008000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3fd4f640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55a5c0d3bb70 0x55a5c0d46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.425+0000 7fee3fd4f640 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55a5c0d3bb70 0x55a5c0d46bb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee3fd4f640 1 -- 172.21.15.83:0/1292509628 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fee300ac070 con 0x55a5c0d3bb70 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55a5c0d3bb70 msgr2=0x55a5c0d46bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55a5c0d3bb70 0x55a5c0d46bb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee2006dde0 msgr2=0x7fee200702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee2006dde0 0x7fee200702f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee28006e60 tx=0x7fee28008000 comp rx=0 tx=0).stop 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 msgr2=0x55a5c0bfb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 --2- 172.21.15.83:0/1292509628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5c0bd2390 0x55a5c0bfb6d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fee3800f590 tx=0x7fee3800fa50 comp rx=0 tx=0).stop 2024-05-17T21:59:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee3fd4f640 1 -- 172.21.15.83:0/1292509628 reap_dead start 2024-05-17T21:59:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 shutdown_connections 2024-05-17T21:59:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 >> 172.21.15.83:0/1292509628 conn(0x55a5c0bb9e40 msgr2=0x55a5c0c153f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 shutdown_connections 2024-05-17T21:59:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.429+0000 7fee45a6c080 1 -- 172.21.15.83:0/1292509628 wait complete. 2024-05-17T21:59:10.427 INFO:teuthology.orchestra.run.smithi083.stdout:Yalphaparent_a 2024-05-17T21:59:10.427 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-17T21:59:10.493 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:10.493 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "alpha", 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 2 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "parent_a", 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 7, 2024-05-17T21:59:10.494 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2024-05-17T21:59:10.495 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:10.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:10.498 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-17T21:59:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.613+0000 7fc1c04b6080 1 Processor -- start 2024-05-17T21:59:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.613+0000 7fc1c04b6080 1 -- start start 2024-05-17T21:59:10.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e859d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e85a4b290 0x564e859911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564e85a62d80 con 0x564e859d7100 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e858dc1b0 con 0x564e85a4b290 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x564e85a62f00 con 0x564e859d6ac0 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53126/0 (socket says 172.21.15.83:53126) 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 -- 172.21.15.83:0/2876399155 learned_addr learned my addr 172.21.15.83:0/2876399155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1ba799640 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e85a4b290 0x564e859911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 -- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 msgr2=0x564e859d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e859d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 -- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e85a4b290 msgr2=0x564e859911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e85a4b290 0x564e859911b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 -- 172.21.15.83:0/2876399155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e858da760 con 0x564e859d7100 2024-05-17T21:59:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009640 tx=0x7fc1a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0f7d5987a2cfb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b3fff640 1 -- 172.21.15.83:0/2876399155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4017070 con 0x564e859d7100 2024-05-17T21:59:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b3fff640 1 -- 172.21.15.83:0/2876399155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a4006690 con 0x564e859d7100 2024-05-17T21:59:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1b3fff640 1 -- 172.21.15.83:0/2876399155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4006a60 con 0x564e859d7100 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2876399155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 msgr2=0x564e85990c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009640 tx=0x7fc1a4005f90 comp rx=0 tx=0).stop 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2876399155 shutdown_connections 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e85a4b290 0x564e859911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e859d7100 0x564e85990c70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2876399155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e859d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2876399155 >> 172.21.15.83:0/2876399155 conn(0x564e85978e40 msgr2=0x564e859bcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2876399155 shutdown_connections 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2876399155 wait complete. 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 Processor -- start 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.617+0000 7fc1c04b6080 1 -- start start 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e85a5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e859d7100 0x564e85a5d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 0x564e859bd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564e85a472b0 con 0x564e85a4b290 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564e85a47550 con 0x564e859d7100 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x564e85a5c520 con 0x564e859d6ac0 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 0x564e859bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 0x564e859bd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53130/0 (socket says 172.21.15.83:53130) 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 -- 172.21.15.83:0/2021247025 learned_addr learned my addr 172.21.15.83:0/2021247025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b9f98640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e85a5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e859d7100 0x564e85a5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 -- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 msgr2=0x564e85a5ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e85a5ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 -- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e859d7100 msgr2=0x564e85a5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e859d7100 0x564e85a5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 -- 172.21.15.83:0/2021247025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e858b6c10 con 0x564e85a4b290 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b9f98640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564e859d6ac0 0x564e85a5ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e859d7100 0x564e85a5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1ba799640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 0x564e859bd170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4008200 tx=0x7fc1b40086c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b2ffd640 1 -- 172.21.15.83:0/2021247025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4016020 con 0x564e85a4b290 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b2ffd640 1 -- 172.21.15.83:0/2021247025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b4012630 con 0x564e85a4b290 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e85a49800 con 0x564e85a4b290 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b2ffd640 1 -- 172.21.15.83:0/2021247025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e040 con 0x564e85a4b290 2024-05-17T21:59:10.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e85a49d40 con 0x564e85a4b290 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b2ffd640 1 -- 172.21.15.83:0/2021247025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1b4019020 con 0x564e85a4b290 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.621+0000 7fc1b2ffd640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19c06de10 0x7fc19c070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b9f98640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19c06de10 0x7fc19c070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b2ffd640 1 -- 172.21.15.83:0/2021247025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b408d930 con 0x564e85a4b290 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x564e859bd6b0 0x564e85a5c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x564e85a5dcd0 con 0x564e859bd6b0 2024-05-17T21:59:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b9f98640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19c06de10 0x7fc19c070320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0ac000 tx=0x7fc1ac003030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x564e859bd6b0 0x564e85a5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b9797640 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x564e859bd6b0 0x564e85a5c950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1b9797640 1 -- 172.21.15.83:0/2021247025 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v43'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc1a401a040 con 0x564e859bd6b0 2024-05-17T21:59:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x564e859bd6b0 msgr2=0x564e85a5c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x564e859bd6b0 0x564e85a5c950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19c06de10 msgr2=0x7fc19c070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19c06de10 0x7fc19c070320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0ac000 tx=0x7fc1ac003030 comp rx=0 tx=0).stop 2024-05-17T21:59:10.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 msgr2=0x564e859bd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 --2- 172.21.15.83:0/2021247025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e85a4b290 0x564e859bd170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4008200 tx=0x7fc1b40086c0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1ba799640 1 -- 172.21.15.83:0/2021247025 reap_dead start 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 shutdown_connections 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 >> 172.21.15.83:0/2021247025 conn(0x564e85978e40 msgr2=0x564e859d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.625+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 shutdown_connections 2024-05-17T21:59:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.629+0000 7fc1c04b6080 1 -- 172.21.15.83:0/2021247025 wait complete. 2024-05-17T21:59:10.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-17T21:59:10.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:10.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-17T21:59:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.785+0000 7f65cbfe8640 1 Processor -- start 2024-05-17T21:59:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 -- start start 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65c40b4e90 0x7f65c40b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b57b0 0x7f65c40b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c415e350 con 0x7f65c415f110 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f65c415ec80 con 0x7f65c40b4e90 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65cbfe8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65c415e630 con 0x7f65c40b57b0 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9583640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b57b0 0x7f65c40b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65c40b4e90 0x7f65c40b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65c40b4e90 0x7f65c40b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:44752/0 (socket says 172.21.15.83:44752) 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 -- 172.21.15.83:0/2793924570 learned_addr learned my addr 172.21.15.83:0/2793924570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65ca585640 1 --1- 172.21.15.83:0/2793924570 >> v1:172.21.15.155:6789/0 conn(0x7f65c415f110 0x7f65c415f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:34510/0 (socket says 172.21.15.83:34510) 2024-05-17T21:59:10.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 -- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b57b0 msgr2=0x7f65c40b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 --2- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b57b0 0x7f65c40b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 -- 172.21.15.83:0/2793924570 >> v1:172.21.15.155:6789/0 conn(0x7f65c415f110 legacy=0x7f65c415f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.789+0000 7f65c9d84640 1 -- 172.21.15.83:0/2793924570 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c415e910 con 0x7f65c40b4e90 2024-05-17T21:59:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65c9d84640 1 --2- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c40b4e90 0x7f65c40b6b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009870 tx=0x7f65c001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ae263d0febfd3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65c8d82640 1 -- 172.21.15.83:0/2793924570 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00b1070 con 0x7f65c40b4e90 2024-05-17T21:59:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65c8d82640 1 -- 172.21.15.83:0/2793924570 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00a8050 con 0x7f65c40b4e90 2024-05-17T21:59:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65c8d82640 1 -- 172.21.15.83:0/2793924570 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00ac860 con 0x7f65c40b4e90 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 -- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c40b4e90 msgr2=0x7f65c40b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c40b4e90 0x7f65c40b6b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009870 tx=0x7f65c001c4d0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 -- 172.21.15.83:0/2793924570 shutdown_connections 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b57b0 0x7f65c40b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/2793924570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c40b4e90 0x7f65c40b6b20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 -- 172.21.15.83:0/2793924570 >> 172.21.15.83:0/2793924570 conn(0x7f65c409b8b0 msgr2=0x7f65c40b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 -- 172.21.15.83:0/2793924570 shutdown_connections 2024-05-17T21:59:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.793+0000 7f65cbfe8640 1 -- 172.21.15.83:0/2793924570 wait complete. 2024-05-17T21:59:10.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 Processor -- start 2024-05-17T21:59:10.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- start start 2024-05-17T21:59:10.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 Processor -- start 2024-05-17T21:59:10.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 -- start start 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b4e90 0x7f65c414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c415f110 0x7f65c40cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41697c0 con 0x7f65c40b57b0 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4169640 con 0x7f65c415f110 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65c4169940 con 0x7f65c40b4e90 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53138/0 (socket says 172.21.15.83:53138) 2024-05-17T21:59:10.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 -- 172.21.15.83:0/1367203421 learned_addr learned my addr 172.21.15.83:0/1367203421 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65ca585640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c415f110 0x7f65c40cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b4e90 0x7f65c414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 -- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b4e90 msgr2=0x7f65c414fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b4e90 0x7f65c414fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 -- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c415f110 msgr2=0x7f65c40cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c415f110 0x7f65c40cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 -- 172.21.15.83:0/1367203421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4140830 con 0x7f65c40b57b0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65c9583640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e870 tx=0x7f65b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65ba7fc640 1 -- 172.21.15.83:0/1367203421 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4017070 con 0x7f65c40b57b0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65ba7fc640 1 -- 172.21.15.83:0/1367203421 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b400c940 con 0x7f65c40b57b0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c4152100 con 0x7f65c40b57b0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65ba7fc640 1 -- 172.21.15.83:0/1367203421 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b400ccf0 con 0x7f65c40b57b0 2024-05-17T21:59:10.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c4152850 con 0x7f65c40b57b0 2024-05-17T21:59:10.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fa815fc90 0x7f0fa81641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa81646e0 con 0x7f0fa80b5f30 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa80b6aa0 con 0x7f0fa80d2e00 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0facfb1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa80b7060 con 0x7f0fa815fc90 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0fa659c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fa815fc90 0x7f0fa81641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0fa659c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0fa815fc90 0x7f0fa81641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:44776/0 (socket says 172.21.15.83:44776) 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.797+0000 7f0fa659c640 1 -- 172.21.15.83:0/1863259952 learned_addr learned my addr 172.21.15.83:0/1863259952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa659c640 1 -- 172.21.15.83:0/1863259952 >> v1:172.21.15.155:6789/0 conn(0x7f0fa80d2e00 legacy=0x7f0fa80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa659c640 1 -- 172.21.15.83:0/1863259952 >> v1:172.21.15.83:6789/0 conn(0x7f0fa80b5f30 legacy=0x7f0fa80d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa659c640 1 -- 172.21.15.83:0/1863259952 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa80b6d80 con 0x7f0fa815fc90 2024-05-17T21:59:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65ba7fc640 1 -- 172.21.15.83:0/1367203421 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f65b401a070 con 0x7f65c40b57b0 2024-05-17T21:59:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65ba7fc640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 0x7f65a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 0x7f65a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65ba7fc640 1 -- 172.21.15.83:0/1367203421 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f65b408f1f0 con 0x7f65c40b57b0 2024-05-17T21:59:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 0x7f65a8071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f65c00ae000 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65d3262000 1 Processor -- start 2024-05-17T21:59:10.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f65d3262000 1 -- start start 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa659c640 1 --2- 172.21.15.83:0/1863259952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa815fc90 0x7f0fa81641a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009870 tx=0x7f0f9801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b781695047cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa559a640 1 -- 172.21.15.83:0/1863259952 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980b0070 con 0x7f0fa815fc90 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa559a640 1 -- 172.21.15.83:0/1863259952 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f980ab600 con 0x7f0fa815fc90 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.801+0000 7f0fa559a640 1 -- 172.21.15.83:0/1863259952 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980ab9d0 con 0x7f0fa815fc90 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- 172.21.15.83:0/1863259952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa815fc90 msgr2=0x7f0fa81641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 --2- 172.21.15.83:0/1863259952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa815fc90 0x7f0fa81641a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009870 tx=0x7f0f9801c4d0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- 172.21.15.83:0/1863259952 shutdown_connections 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 --2- 172.21.15.83:0/1863259952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa815fc90 0x7f0fa81641a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- 172.21.15.83:0/1863259952 >> 172.21.15.83:0/1863259952 conn(0x7f0fa80b45e0 msgr2=0x7f0fa80b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c1fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56ab7df0 0x557a56c203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a56a31220 con 0x557a56adc240 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a56a20dc0 con 0x557a56acdf60 2024-05-17T21:59:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557a56a24e00 con 0x557a56ab7df0 2024-05-17T21:59:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- 172.21.15.83:0/1863259952 shutdown_connections 2024-05-17T21:59:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- 172.21.15.83:0/1863259952 wait complete. 2024-05-17T21:59:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53152/0 (socket says 172.21.15.83:53152) 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 -- 172.21.15.83:0/3226571280 learned_addr learned my addr 172.21.15.83:0/3226571280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9583640 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c1fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65ca585640 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56ab7df0 0x557a56c203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 -- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56ab7df0 msgr2=0x557a56c203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56ab7df0 0x557a56c203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 -- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 msgr2=0x557a56c1fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c1fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 -- 172.21.15.83:0/3226571280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a569fcbf0 con 0x557a56adc240 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65c9d84640 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f65c00af040 tx=0x7f65c00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3525d749ba1bd335 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65a77fe640 1 -- 172.21.15.83:0/3226571280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00c7070 con 0x557a56adc240 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65a77fe640 1 -- 172.21.15.83:0/3226571280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00a8040 con 0x557a56adc240 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65a77fe640 1 -- 172.21.15.83:0/3226571280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00b1040 con 0x557a56adc240 2024-05-17T21:59:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- 172.21.15.83:0/3226571280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 msgr2=0x557a56c1f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f65c00af040 tx=0x7f65c00a7000 comp rx=0 tx=0).stop 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- 172.21.15.83:0/3226571280 shutdown_connections 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56ab7df0 0x557a56c203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c1fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 --2- 172.21.15.83:0/3226571280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c1f950 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- 172.21.15.83:0/3226571280 >> 172.21.15.83:0/3226571280 conn(0x557a566e2810 msgr2=0x557a56bd40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- 172.21.15.83:0/3226571280 shutdown_connections 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f65d3262000 1 -- 172.21.15.83:0/3226571280 wait complete. 2024-05-17T21:59:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 Processor -- start 2024-05-17T21:59:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.805+0000 7f0facfb1640 1 -- start start 2024-05-17T21:59:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- start start 2024-05-17T21:59:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 Processor -- start 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa80b5f30 0x7f0fa8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fa815fc90 0x7f0fa816a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa81648c0 0x7f0fa806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa80b6700 con 0x7f0fa80b5f30 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa80b5aa0 con 0x7f0fa81648c0 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa80b6880 con 0x7f0fa815fc90 2024-05-17T21:59:10.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa80b5f30 0x7f0fa8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa80b5f30 0x7f0fa8157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53164/0 (socket says 172.21.15.83:53164) 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 -- 172.21.15.83:0/1831484666 learned_addr learned my addr 172.21.15.83:0/1831484666 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa6d9d640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa81648c0 0x7f0fa806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa5d9b640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fa815fc90 0x7f0fa816a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 -- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fa815fc90 msgr2=0x7f0fa816a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fa815fc90 0x7f0fa816a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 -- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa81648c0 msgr2=0x7f0fa806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fa81648c0 0x7f0fa806feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 -- 172.21.15.83:0/1831484666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa81f90d0 con 0x7f0fa80b5f30 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0fa659c640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa80b5f30 0x7f0fa8157f60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0f980ae040 tx=0x7f0f9801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980b0040 con 0x7f0fa80b5f30 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f980bc070 con 0x7f0fa80b5f30 2024-05-17T21:59:10.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 -- 172.21.15.83:0/1831484666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa81f93b0 con 0x7f0fa80b5f30 2024-05-17T21:59:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980b6880 con 0x7f0fa80b5f30 2024-05-17T21:59:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0facfb1640 1 -- 172.21.15.83:0/1831484666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81f9970 con 0x7f0fa80b5f30 2024-05-17T21:59:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56ab7df0 0x557a56c25b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a569a9f30 con 0x557a56adc240 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a56a1d740 con 0x557a56ab7df0 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557a569a7cc0 con 0x557a56acdf60 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 10 client.0 _ll_register_callbacks cb 0x557a56bc16a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53176/0 (socket says 172.21.15.83:53176) 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 -- 172.21.15.83:0/2835544960 learned_addr learned my addr 172.21.15.83:0/2835544960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65c9583640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65c9d84640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56ab7df0 0x557a56c25b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 -- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 msgr2=0x557a56c26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 -- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56ab7df0 msgr2=0x557a56c25b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56ab7df0 0x557a56c25b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 -- 172.21.15.83:0/2835544960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a569da460 con 0x557a56adc240 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65ca585640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65bc007a70 tx=0x7f65bc0018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf19e700feb51a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc01b060 con 0x557a56adc240 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f65bc016b10 con 0x557a56adc240 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00a8040 con 0x557a56adc240 2024-05-17T21:59:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f0f94ff9640 1 -- 172.21.15.83:0/1831484666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0f80005280 con 0x7f0fa80b5f30 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f65bc016d10 con 0x557a56adc240 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 10 client.5370 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x557a569df9e0 con 0x557a56adc240 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00a8040 con 0x557a56adc240 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f65bc0124d0 con 0x557a56adc240 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 10 client.5370 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 10 client.5370 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 20 client.5370 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.809+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566c7b80 0x557a56c2fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 4 client.5370 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a569bcd40 con 0x557a566c7b80 2024-05-17T21:59:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65c9d84640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566c7b80 0x557a56c2fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f980ab480 con 0x7f0fa80b5f30 2024-05-17T21:59:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65a4ff9640 10 client.5370 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0f96ffd640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0f8806f400 0x7f0f88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65c9d84640 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566c7b80 0x557a56c2fee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287f87e8f76a3131 server_cookie=d4626422bdb24576 in_seq=0 out_seq=0 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0fa5d9b640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0f8806f400 0x7f0f88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f0f98129650 con 0x7f0fa80b5f30 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+505 (secure 0 0 0) 0x7f0f98016150 con 0x7f0fa80b5f30 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0f94ff9640 1 -- 172.21.15.83:0/1831484666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f0f80005560 con 0x7f0fa80b5f30 2024-05-17T21:59:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65a4ff9640 1 -- 172.21.15.83:0/2835544960 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a569bcd40 con 0x557a566c7b80 2024-05-17T21:59:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65a4ff9640 10 client.5370 handle_command_reply: tid=0 2024-05-17T21:59:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f0fa5d9b640 1 --2- 172.21.15.83:0/1831484666 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0f8806f400 0x7f0f88071910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c007fd0 tx=0x7f0f9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 client.5370 shutdown 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 msgr2=0x557a56c26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65bc007a70 tx=0x7f65bc0018b0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 shutdown_connections 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566c7b80 0x557a56c2fee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56adc240 0x557a56c26580 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65a4ff9640 0 client.5370 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:10.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 --2- 172.21.15.83:0/2835544960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56ab7df0 0x557a56c25b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 >> 172.21.15.83:0/2835544960 conn(0x557a566e2810 msgr2=0x557a56bd40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 shutdown_connections 2024-05-17T21:59:10.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 1 -- 172.21.15.83:0/2835544960 wait complete. 2024-05-17T21:59:10.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.813+0000 7f65d3262000 20 client.5370 trim_cache size 0 max 16384 2024-05-17T21:59:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 Processor -- start 2024-05-17T21:59:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- start start 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56adc240 0x557a56c2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4202c10 con 0x557a56ab7df0 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4201fa0 con 0x557a56acdf60 2024-05-17T21:59:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f65c42025c0 con 0x557a56adc240 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53178/0 (socket says 172.21.15.83:53178) 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 -- 172.21.15.83:0/1214768526 learned_addr learned my addr 172.21.15.83:0/1214768526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65ca585640 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56adc240 0x557a56c2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 -- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56adc240 msgr2=0x557a56c2a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56adc240 0x557a56c2a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 -- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 msgr2=0x557a56c29b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9583640 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c29b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 -- 172.21.15.83:0/1214768526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a56bdad50 con 0x557a56ab7df0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65c9d84640 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b9010 tx=0x7f65c0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76203acca3696606 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65a6ffd640 1 -- 172.21.15.83:0/1214768526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00c7040 con 0x557a56ab7df0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65a6ffd640 1 -- 172.21.15.83:0/1214768526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00a8040 con 0x557a56ab7df0 2024-05-17T21:59:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65a6ffd640 1 -- 172.21.15.83:0/1214768526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00c2880 con 0x557a56ab7df0 2024-05-17T21:59:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- 172.21.15.83:0/1214768526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 msgr2=0x557a56c8abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b9010 tx=0x7f65c0004740 comp rx=0 tx=0).stop 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- 172.21.15.83:0/1214768526 shutdown_connections 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56adc240 0x557a56c2a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56acdf60 0x557a56c29b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 --2- 172.21.15.83:0/1214768526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c8abd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- 172.21.15.83:0/1214768526 >> 172.21.15.83:0/1214768526 conn(0x557a566e2810 msgr2=0x557a56c25f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- 172.21.15.83:0/1214768526 shutdown_connections 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.833+0000 7f65d3262000 1 -- 172.21.15.83:0/1214768526 wait complete. 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- start start 2024-05-17T21:59:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 Processor -- start 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56adc240 0x557a56c92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a569efdd0 con 0x557a56ab7df0 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557a56a2fff0 con 0x557a56adc240 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557a56a24110 con 0x557a56acdf60 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 10 client.0 _ll_register_callbacks cb 0x7f65bc005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:10.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53182/0 (socket says 172.21.15.83:53182) 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 -- 172.21.15.83:0/782973587 learned_addr learned my addr 172.21.15.83:0/782973587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 -- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 msgr2=0x557a56c927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 -- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56adc240 msgr2=0x557a56c92d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56adc240 0x557a56c92d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 -- 172.21.15.83:0/782973587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a56aa5a10 con 0x557a56ab7df0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9d84640 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f65c00190e0 tx=0x7f65c001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f997febaedc902 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00c7040 con 0x557a56ab7df0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f65c00c2680 con 0x557a56ab7df0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00a8040 con 0x557a56ab7df0 2024-05-17T21:59:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f65c00c2880 con 0x557a56ab7df0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 10 client.5376 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x557a569dddc0 con 0x557a56ab7df0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00a8040 con 0x557a56ab7df0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f65c00c1070 con 0x557a56ab7df0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 10 client.5376 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 10 client.5376 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 20 client.5376 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566b5be0 0x557a56c4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 4 client.5376 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x557a56a33170 con 0x557a566b5be0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9583640 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566b5be0 0x557a56c4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65a4ff9640 10 client.5376 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:10.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.837+0000 7f65c9583640 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566b5be0 0x557a56c4d730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29728abb9852e39 server_cookie=5bc4547bfdfd4492 in_seq=0 out_seq=0 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65a4ff9640 1 -- 172.21.15.83:0/782973587 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a56a33170 con 0x557a566b5be0 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65a4ff9640 10 client.5376 handle_command_reply: tid=0 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 client.5376 shutdown 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 msgr2=0x557a56c92290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f65c00190e0 tx=0x7f65c001ad10 comp rx=0 tx=0).stop 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 shutdown_connections 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x557a566b5be0 0x557a56c4d730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a56adc240 0x557a56c92d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557a56acdf60 0x557a56c927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65a4ff9640 0 client.5376 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 --2- 172.21.15.83:0/782973587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a56ab7df0 0x557a56c92290 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 >> 172.21.15.83:0/782973587 conn(0x557a566e2810 msgr2=0x557a56c28b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 shutdown_connections 2024-05-17T21:59:10.841 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T21:59:10.841+0000 7ff4794a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5328=0-4194304@1} caps={5328=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5328 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x563b1b6bcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T21:59:10.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 1 -- 172.21.15.83:0/782973587 wait complete. 2024-05-17T21:59:10.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65d3262000 20 client.5376 trim_cache size 0 max 16384 2024-05-17T21:59:10.842 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:10.842 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T21:59:10.842 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0a58743c-2923-4837-b8c6-5e8a355bc6af", 2024-05-17T21:59:10.842 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T21:59:10.842 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 msgr2=0x7f65a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 0x7f65a8071910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f65c00ae000 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 msgr2=0x7f65c414eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f65b400e870 tx=0x7f65b400ec70 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 shutdown_connections 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f65a806f400 0x7f65a8071910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65c415f110 0x7f65c40cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65c40b57b0 0x7f65c414eb80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 --2- 172.21.15.83:0/1367203421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65c40b4e90 0x7f65c414fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 >> 172.21.15.83:0/1367203421 conn(0x7f65c409b8b0 msgr2=0x7f65c406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:10.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 shutdown_connections 2024-05-17T21:59:10.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:10.841+0000 7f65cbfe8640 1 -- 172.21.15.83:0/1367203421 wait complete. 2024-05-17T21:59:10.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:10.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-17T21:59:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.005+0000 7f311da36640 1 Processor -- start 2024-05-17T21:59:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.005+0000 7f311da36640 1 -- start start 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.005+0000 7f311da36640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f311806c790 0x7f311806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.005+0000 7f311da36640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 0x7f311815f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f311806d0e0 con 0x7f311815ea50 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f311815bae0 con 0x7f311806c790 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f31180af470 con 0x7f311815f240 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f3116ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f311815ea50 0x7f31180b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:47264/0 (socket says 172.21.15.83:47264) 2024-05-17T21:59:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f311806c790 0x7f311806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f3116ffd640 1 -- 172.21.15.83:0/2591316324 learned_addr learned my addr 172.21.15.83:0/2591316324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31177fe640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 0x7f311815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 -- 172.21.15.83:0/2591316324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 msgr2=0x7f311815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 0x7f311815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 -- 172.21.15.83:0/2591316324 >> v1:172.21.15.83:6789/0 conn(0x7f311815ea50 legacy=0x7f31180b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 -- 172.21.15.83:0/2591316324 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311815e290 con 0x7f311806c790 2024-05-17T21:59:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31177fe640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 0x7f311815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:11.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f31167fc640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311806c790 0x7f311806cba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f31100097c0 tx=0x7f311001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16c2cf6e44acbb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f3115ffb640 1 -- 172.21.15.83:0/2591316324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100b0070 con 0x7f311806c790 2024-05-17T21:59:11.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f3115ffb640 1 -- 172.21.15.83:0/2591316324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31100ab600 con 0x7f311806c790 2024-05-17T21:59:11.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f3115ffb640 1 -- 172.21.15.83:0/2591316324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100ab9d0 con 0x7f311806c790 2024-05-17T21:59:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- 172.21.15.83:0/2591316324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311806c790 msgr2=0x7f311806cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311806c790 0x7f311806cba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f31100097c0 tx=0x7f311001c4a0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- 172.21.15.83:0/2591316324 shutdown_connections 2024-05-17T21:59:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311815f240 0x7f311815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 --2- 172.21.15.83:0/2591316324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311806c790 0x7f311806cba0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- 172.21.15.83:0/2591316324 >> 172.21.15.83:0/2591316324 conn(0x7f311809b8b0 msgr2=0x7f31180b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- 172.21.15.83:0/2591316324 shutdown_connections 2024-05-17T21:59:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.009+0000 7f311da36640 1 -- 172.21.15.83:0/2591316324 wait complete. 2024-05-17T21:59:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 Processor -- start 2024-05-17T21:59:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 -- start start 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311806c790 0x7f3118140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f311815ea50 0x7f31181411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31180b3f10 con 0x7f311815f240 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31180b5020 con 0x7f311815ea50 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f311da36640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f311815ee60 con 0x7f311806c790 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53206/0 (socket says 172.21.15.83:53206) 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 -- 172.21.15.83:0/1371067701 learned_addr learned my addr 172.21.15.83:0/1371067701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:11.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31167fc640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f311815ea50 0x7f31181411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311806c790 0x7f3118140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 -- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311806c790 msgr2=0x7f3118140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311806c790 0x7f3118140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 -- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f311815ea50 msgr2=0x7f31181411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f311815ea50 0x7f31181411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 -- 172.21.15.83:0/1371067701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311814ef50 con 0x7f311815f240 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.013+0000 7f31177fe640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f310800c820 tx=0x7f310800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 -- 172.21.15.83:0/1371067701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108015020 con 0x7f311815f240 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 -- 172.21.15.83:0/1371067701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3108004240 con 0x7f311815f240 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311814f230 con 0x7f311815f240 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 -- 172.21.15.83:0/1371067701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108003620 con 0x7f311815f240 2024-05-17T21:59:11.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311815fa90 con 0x7f311815f240 2024-05-17T21:59:11.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 -- 172.21.15.83:0/1371067701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3108018020 con 0x7f311815f240 2024-05-17T21:59:11.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 0x7f30f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 0x7f30f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f310ffff640 1 -- 172.21.15.83:0/1371067701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f310808f3c0 con 0x7f311815f240 2024-05-17T21:59:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 0x7f30f8071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f31000012f0 tx=0x7f3100009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f3124cb0000 1 Processor -- start 2024-05-17T21:59:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.017+0000 7f3124cb0000 1 -- start start 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x562374704e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x562374705360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623745a3a90 con 0x56237458c090 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562374517120 con 0x56237459ef80 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562374506cc0 con 0x56237458b880 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53220/0 (socket says 172.21.15.83:53220) 2024-05-17T21:59:11.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 -- 172.21.15.83:0/1184006456 learned_addr learned my addr 172.21.15.83:0/1184006456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31167fc640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x562374705360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x562374704e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 -- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 msgr2=0x562374705360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x562374705360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 -- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 msgr2=0x562374704e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x562374704e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 -- 172.21.15.83:0/1184006456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623744c3110 con 0x56237458c090 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31167fc640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x562374705360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x562374704e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f31177fe640 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3108057bb0 tx=0x7f31080642d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88953196d382b0a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f310d7fa640 1 -- 172.21.15.83:0/1184006456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310805ce60 con 0x56237458c090 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f310d7fa640 1 -- 172.21.15.83:0/1184006456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310805bb30 con 0x56237458c090 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f310d7fa640 1 -- 172.21.15.83:0/1184006456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310805fee0 con 0x56237458c090 2024-05-17T21:59:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- 172.21.15.83:0/1184006456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 msgr2=0x5623747058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3108057bb0 tx=0x7f31080642d0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- 172.21.15.83:0/1184006456 shutdown_connections 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623747058a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x562374705360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1184006456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x562374704e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- 172.21.15.83:0/1184006456 >> 172.21.15.83:0/1184006456 conn(0x5623745da010 msgr2=0x5623746ba5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- 172.21.15.83:0/1184006456 shutdown_connections 2024-05-17T21:59:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- 172.21.15.83:0/1184006456 wait complete. 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 -- start start 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.021+0000 7f3124cb0000 1 Processor -- start 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623746c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x5623746c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623744c2ec0 con 0x56237458c090 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56237450ad00 con 0x56237459ef80 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5623741d0680 con 0x56237458b880 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 10 client.0 _ll_register_callbacks cb 0x5623745be1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53224/0 (socket says 172.21.15.83:53224) 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 -- 172.21.15.83:0/2820979017 learned_addr learned my addr 172.21.15.83:0/2820979017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31177fe640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x5623746c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3116ffd640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623746c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 -- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 msgr2=0x5623746c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623746c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 -- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 msgr2=0x5623746c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x5623746c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 -- 172.21.15.83:0/2820979017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623744c5890 con 0x56237458c090 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f31167fc640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31100ae040 tx=0x7f311001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6949db673ad5670 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100b0070 con 0x56237458c090 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f31100b5440 con 0x56237458c090 2024-05-17T21:59:11.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310805bb30 con 0x56237458c090 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f31100c2050 con 0x56237458c090 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 10 client.5391 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5623744c7650 con 0x56237458c090 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310805bb30 con 0x56237458c090 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f31100c5020 con 0x56237458c090 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 10 client.5391 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 10 client.5391 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 20 client.5391 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741ad930 0x56237470be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 4 client.5391 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623744a2c40 con 0x5623741ad930 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3116ffd640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741ad930 0x56237470be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 10 client.5391 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:11.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3116ffd640 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741ad930 0x56237470be90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91f53d1caf61fe8 server_cookie=dd9934c2744c3b84 in_seq=0 out_seq=0 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 1 -- 172.21.15.83:0/2820979017 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623744a2c40 con 0x5623741ad930 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f30f67fc640 10 client.5391 handle_command_reply: tid=0 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 client.5391 shutdown 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 msgr2=0x5623746c1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31100ae040 tx=0x7f311001ace0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 shutdown_connections 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.025+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741ad930 0x56237470be90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237459ef80 0x5623746c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56237458c090 0x5623746c1f70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f30f67fc640 0 client.5391 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 --2- 172.21.15.83:0/2820979017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623746c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 >> 172.21.15.83:0/2820979017 conn(0x5623745da010 msgr2=0x5623746ba5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 shutdown_connections 2024-05-17T21:59:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 1 -- 172.21.15.83:0/2820979017 wait complete. 2024-05-17T21:59:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.029+0000 7f3124cb0000 20 client.5391 trim_cache size 0 max 16384 2024-05-17T21:59:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 Processor -- start 2024-05-17T21:59:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 -- start start 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623747740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458c090 0x562374774630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31182055c0 con 0x5623745da010 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f311815dc20 con 0x56237458c090 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f311815cf80 con 0x56237458b880 2024-05-17T21:59:11.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53230/0 (socket says 172.21.15.83:53230) 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 -- 172.21.15.83:0/1576370202 learned_addr learned my addr 172.21.15.83:0/1576370202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 -- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 msgr2=0x5623747740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623747740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 -- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458c090 msgr2=0x562374774630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458c090 0x562374774630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.045+0000 7f3116ffd640 1 -- 172.21.15.83:0/1576370202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623744a68c0 con 0x5623745da010 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3116ffd640 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f310000ecb0 tx=0x7f3100012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387310e232e7862d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f310cff9640 1 -- 172.21.15.83:0/1576370202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100007a20 con 0x5623745da010 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f310cff9640 1 -- 172.21.15.83:0/1576370202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3100029020 con 0x5623745da010 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f310cff9640 1 -- 172.21.15.83:0/1576370202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100005c60 con 0x5623745da010 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- 172.21.15.83:0/1576370202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 msgr2=0x562374773bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f310000ecb0 tx=0x7f3100012000 comp rx=0 tx=0).stop 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- 172.21.15.83:0/1576370202 shutdown_connections 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458c090 0x562374774630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458b880 0x5623747740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- 172.21.15.83:0/1576370202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x562374773bb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- 172.21.15.83:0/1576370202 >> 172.21.15.83:0/1576370202 conn(0x5623746be400 msgr2=0x5623746c09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- 172.21.15.83:0/1576370202 shutdown_connections 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- 172.21.15.83:0/1576370202 wait complete. 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- start start 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 Processor -- start 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458b880 0x5623746bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458c090 0x562374734d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623744ae0f0 con 0x5623745da010 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f311815d5d0 con 0x56237458b880 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56237450f330 con 0x56237458c090 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3124cb0000 10 client.0 _ll_register_callbacks cb 0x7f31100ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f31177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53246/0 (socket says 172.21.15.83:53246) 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f31177fe640 1 -- 172.21.15.83:0/3870592545 learned_addr learned my addr 172.21.15.83:0/3870592545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f3116ffd640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458b880 0x5623746bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.049+0000 7f31167fc640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458c090 0x562374734d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 -- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458c090 msgr2=0x562374734d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458c090 0x562374734d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 -- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458b880 msgr2=0x5623746bc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458b880 0x5623746bc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 -- 172.21.15.83:0/3870592545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623744c1fa0 con 0x5623745da010 2024-05-17T21:59:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f31177fe640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f310808d940 tx=0x7f310808d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04fd794b26906a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310805ce60 con 0x5623745da010 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3108003db0 con 0x5623745da010 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3100029020 con 0x5623745da010 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f310805bb30 con 0x5623745da010 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 10 client.5397 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5623741c1220 con 0x5623745da010 2024-05-17T21:59:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3100029020 con 0x5623745da010 2024-05-17T21:59:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f310805ad90 con 0x5623745da010 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 10 client.5397 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 10 client.5397 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 20 client.5397 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741c6fe0 0x562374735800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 4 client.5397 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562374519070 con 0x5623741c6fe0 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f3116ffd640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741c6fe0 0x562374735800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.053+0000 7f30f67fc640 10 client.5397 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3116ffd640 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741c6fe0 0x562374735800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7629f83ba229f526 server_cookie=38612af8051410d1 in_seq=0 out_seq=0 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f30f67fc640 1 -- 172.21.15.83:0/3870592545 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x562374519070 con 0x5623741c6fe0 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f30f67fc640 10 client.5397 handle_command_reply: tid=0 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 client.5397 shutdown 2024-05-17T21:59:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 msgr2=0x5623747352c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f310808d940 tx=0x7f310808d7c0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 shutdown_connections 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x5623741c6fe0 0x562374735800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623745da010 0x5623747352c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56237458c090 0x562374734d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 --2- 172.21.15.83:0/3870592545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56237458b880 0x5623746bc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f30f67fc640 0 client.5397 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 >> 172.21.15.83:0/3870592545 conn(0x5623746be400 msgr2=0x5623746c13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 shutdown_connections 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 1 -- 172.21.15.83:0/3870592545 wait complete. 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f3124cb0000 20 client.5397 trim_cache size 0 max 16384 2024-05-17T21:59:11.053 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "0a58743c-2923-4837-b8c6-5e8a355bc6af": { 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0a58743c-2923-4837-b8c6-5e8a355bc6af", 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:11.054 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 msgr2=0x7f30f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 0x7f30f8071940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f31000012f0 tx=0x7f3100009040 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 msgr2=0x7f311814fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f310800c820 tx=0x7f310800cce0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 shutdown_connections 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311815f240 0x7f311814fd10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f30f806f430 0x7f30f8071940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f311815ea50 0x7f31181411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 --2- 172.21.15.83:0/1371067701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f311806c790 0x7f3118140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 >> 172.21.15.83:0/1371067701 conn(0x7f311809b8b0 msgr2=0x7f31180d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 shutdown_connections 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.057+0000 7f311da36640 1 -- 172.21.15.83:0/1371067701 wait complete. 2024-05-17T21:59:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:11.061+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 7 ==== log(4 entries from seq 373 at 2024-05-17T21:59:11.002047+0000) ==== 1118+0+0 (secure 0 0 0) 0x7f0f98127c90 con 0x7f0fa80b5f30 2024-05-17T21:59:11.070 INFO:tasks.cephfs.filesystem:scrub status for tag:0a58743c-2923-4837-b8c6-5e8a355bc6af - {'path': '/', 'tag': '0a58743c-2923-4837-b8c6-5e8a355bc6af', 'options': 'recursive,repair'} 2024-05-17T21:59:12.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:12.073+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-17T21:59:10.842481+0000) ==== 1597+0+0 (secure 0 0 0) 0x7f0f980f4940 con 0x7f0fa80b5f30 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: cluster: 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: id: 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: health: HEALTH_OK 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: services: 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-17T21:59:12.069 INFO:teuthology.orchestra.run.smithi083.stdout: mgr: y(active, since 119s), standbys: x 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: mds: 1/1 daemons up, 3 standby 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: osd: 8 osds: 8 up (since 112s), 8 in (since 2m) 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: data: 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: volumes: 1/1 healthy 2024-05-17T21:59:12.070 INFO:teuthology.orchestra.run.smithi083.stdout: pools: 3 pools, 97 pgs 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: objects: 24 objects, 579 KiB 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: usage: 216 MiB used, 720 GiB / 720 GiB avail 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: pgs: 97 active+clean 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: io: 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:12.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.842481+0000 mds.a [INF] scrub queued for path: / 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.842484+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.842489+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.843627+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.843637+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-17T21:59:12.072 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:10.843908+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T21:59:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:16.093+0000 7f0f96ffd640 1 -- 172.21.15.83:0/1831484666 <== mon.0 v2:172.21.15.83:3300/0 9 ==== log(1 entries from seq 7 at 2024-05-17T21:59:15.468823+0000) ==== 189+0+0 (secure 0 0 0) 0x7f0f980f2160 con 0x7f0fa80b5f30 2024-05-17T21:59:16.087 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T21:59:15.468823+0000 mds.a [INF] scrub summary: idle 2024-05-17T21:59:29.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:59:29.850 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T21:59:41.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:41.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-17T21:59:41.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 Processor -- start 2024-05-17T21:59:41.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 -- start start 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46940b3d00 0x7f46940d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 0x7f4694072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46940737d0 con 0x7f46940733c0 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f46940b4110 con 0x7f46940b3d00 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469c3cd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4694070770 con 0x7f4694143f20 2024-05-17T21:59:41.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46940b3d00 0x7f46940d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46940b3d00 0x7f46940d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39010/0 (socket says 172.21.15.83:39010) 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 -- 172.21.15.83:0/1856361147 learned_addr learned my addr 172.21.15.83:0/1856361147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 -- 172.21.15.83:0/1856361147 >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 msgr2=0x7f4694072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f4699968640 1 --2- 172.21.15.83:0/1856361147 >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 0x7f4694072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 --2- 172.21.15.83:0/1856361147 >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 0x7f4694072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 -- 172.21.15.83:0/1856361147 >> v1:172.21.15.155:6789/0 conn(0x7f46940733c0 legacy=0x7f469406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f469a169640 1 -- 172.21.15.83:0/1856361147 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46940d10d0 con 0x7f46940b3d00 2024-05-17T21:59:41.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.244+0000 7f4699968640 1 --2- 172.21.15.83:0/1856361147 >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 0x7f4694072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469a169640 1 --2- 172.21.15.83:0/1856361147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f46940d0a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46941615a0 tx=0x7f469001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5b799775ae58c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f4699167640 1 -- 172.21.15.83:0/1856361147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900b2070 con 0x7f46940b3d00 2024-05-17T21:59:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f4699167640 1 -- 172.21.15.83:0/1856361147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900ad680 con 0x7f46940b3d00 2024-05-17T21:59:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f4699167640 1 -- 172.21.15.83:0/1856361147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900ada50 con 0x7f46940b3d00 2024-05-17T21:59:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 -- 172.21.15.83:0/1856361147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 msgr2=0x7f46940d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 --2- 172.21.15.83:0/1856361147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f46940d0a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46941615a0 tx=0x7f469001c490 comp rx=0 tx=0).stop 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 -- 172.21.15.83:0/1856361147 shutdown_connections 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 --2- 172.21.15.83:0/1856361147 >> v2:172.21.15.155:3300/0 conn(0x7f4694143f20 0x7f4694072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 --2- 172.21.15.83:0/1856361147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f46940d0a10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 -- 172.21.15.83:0/1856361147 >> 172.21.15.83:0/1856361147 conn(0x7f469406cd90 msgr2=0x7f46940b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 -- 172.21.15.83:0/1856361147 shutdown_connections 2024-05-17T21:59:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.248+0000 7f469c3cd640 1 -- 172.21.15.83:0/1856361147 wait complete. 2024-05-17T21:59:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 Processor -- start 2024-05-17T21:59:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- start start 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 0x7f469415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 0x7f46941380d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4694070c00 con 0x7f46940b3d00 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4694144330 con 0x7f4694143f20 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4694138ca0 con 0x7f46940733c0 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39018/0 (socket says 172.21.15.83:39018) 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 -- 172.21.15.83:0/3151935724 learned_addr learned my addr 172.21.15.83:0/3151935724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469a96a640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 0x7f46941380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469a169640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 0x7f469415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 -- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 msgr2=0x7f469415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 0x7f469415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 -- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 msgr2=0x7f46941380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 0x7f46941380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 -- 172.21.15.83:0/3151935724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4694140700 con 0x7f46940b3d00 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469a96a640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 0x7f46941380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469a169640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 0x7f469415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f4699968640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f468400c7f0 tx=0x7f468400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f468affd640 1 -- 172.21.15.83:0/3151935724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684015070 con 0x7f46940b3d00 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f468affd640 1 -- 172.21.15.83:0/3151935724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4684007820 con 0x7f46940b3d00 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f468affd640 1 -- 172.21.15.83:0/3151935724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684007c10 con 0x7f46940b3d00 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46941409e0 con 0x7f46940b3d00 2024-05-17T21:59:41.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.252+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941414c0 con 0x7f46940b3d00 2024-05-17T21:59:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f468affd640 1 -- 172.21.15.83:0/3151935724 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4684018070 con 0x7f46940b3d00 2024-05-17T21:59:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f468affd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 0x7f4678071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f468affd640 1 -- 172.21.15.83:0/3151935724 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f468408f490 con 0x7f46940b3d00 2024-05-17T21:59:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f469a169640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 0x7f4678071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f469a169640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 0x7f4678071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f4690008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f46a3647000 1 Processor -- start 2024-05-17T21:59:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.256+0000 7f46a3647000 1 -- start start 2024-05-17T21:59:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd23b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2238090 0x561fd23b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fd224fa90 con 0x561fd224af80 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561fd21c3120 con 0x561fd2237880 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561fd21b2cc0 con 0x561fd2238090 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39024/0 (socket says 172.21.15.83:39024) 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 -- 172.21.15.83:0/3409006955 learned_addr learned my addr 172.21.15.83:0/3409006955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f4699968640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd23b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a96a640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2238090 0x561fd23b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 -- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2238090 msgr2=0x561fd23b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2238090 0x561fd23b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 -- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 msgr2=0x561fd23b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd23b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 -- 172.21.15.83:0/3409006955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd216f110 con 0x561fd224af80 2024-05-17T21:59:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f4699968640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd23b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f469a169640 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4690009160 tx=0x7f4690004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187f179c9dd24384 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f4677fff640 1 -- 172.21.15.83:0/3409006955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900b2040 con 0x561fd224af80 2024-05-17T21:59:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f4677fff640 1 -- 172.21.15.83:0/3409006955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900c7070 con 0x561fd224af80 2024-05-17T21:59:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f4677fff640 1 -- 172.21.15.83:0/3409006955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900b8cf0 con 0x561fd224af80 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- 172.21.15.83:0/3409006955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 msgr2=0x561fd23b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4690009160 tx=0x7f4690004740 comp rx=0 tx=0).stop 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- 172.21.15.83:0/3409006955 shutdown_connections 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2238090 0x561fd23b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd23b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 --2- 172.21.15.83:0/3409006955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd224af80 0x561fd23b0e20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- 172.21.15.83:0/3409006955 >> 172.21.15.83:0/3409006955 conn(0x561fd2286010 msgr2=0x561fd23665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- 172.21.15.83:0/3409006955 shutdown_connections 2024-05-17T21:59:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- 172.21.15.83:0/3409006955 wait complete. 2024-05-17T21:59:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 -- start start 2024-05-17T21:59:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.260+0000 7f46a3647000 1 Processor -- start 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2237880 0x561fd236da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 0x561fd2372be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fd216eec0 con 0x561fd2238090 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561fd21b6d00 con 0x561fd224af80 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561fd1e7c680 con 0x561fd2237880 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 10 client.0 _ll_register_callbacks cb 0x561fd226a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39032/0 (socket says 172.21.15.83:39032) 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 -- 172.21.15.83:0/662814143 learned_addr learned my addr 172.21.15.83:0/662814143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f469a169640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2237880 0x561fd236da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f469a96a640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 0x561fd2372be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 -- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2237880 msgr2=0x561fd236da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2237880 0x561fd236da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 -- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 msgr2=0x561fd2372be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 0x561fd2372be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 -- 172.21.15.83:0/662814143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd2171890 con 0x561fd2238090 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f469a96a640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 0x561fd2372be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f4699968640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f468408d980 tx=0x7f468405c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28e4208c8ec23e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:41.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468405c340 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46900c7070 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f46840575b0 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f468405bc40 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 10 client.5409 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x561fd2173650 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900c7070 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f46840577b0 con 0x561fd2238090 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 10 client.5409 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 10 client.5409 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 20 client.5409 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e59930 0x561fd23b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 4 client.5409 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd214ec40 con 0x561fd1e59930 2024-05-17T21:59:41.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f469a169640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e59930 0x561fd23b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f469a169640 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e59930 0x561fd23b7e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb55f88b250d0ec5 server_cookie=95d580551064fcce in_seq=0 out_seq=0 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 10 client.5409 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 1 -- 172.21.15.83:0/662814143 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fd214ec40 con 0x561fd1e59930 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 10 client.5409 handle_command_reply: tid=0 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 client.5409 shutdown 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 msgr2=0x561fd236dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f468408d980 tx=0x7f468405c260 comp rx=0 tx=0).stop 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 shutdown_connections 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e59930 0x561fd23b7e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd224af80 0x561fd2372be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd236dfb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46757fa640 0 client.5409 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 --2- 172.21.15.83:0/662814143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd2237880 0x561fd236da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 >> 172.21.15.83:0/662814143 conn(0x561fd2286010 msgr2=0x561fd23665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 shutdown_connections 2024-05-17T21:59:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 1 -- 172.21.15.83:0/662814143 wait complete. 2024-05-17T21:59:41.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.264+0000 7f46a3647000 20 client.5409 trim_cache size 0 max 16384 2024-05-17T21:59:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 Processor -- start 2024-05-17T21:59:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 -- start start 2024-05-17T21:59:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2238090 0x561fd23c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd23c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4694203510 con 0x561fd2237880 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46942025b0 con 0x561fd2238090 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46940b1b40 con 0x561fd224af80 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39042/0 (socket says 172.21.15.83:39042) 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 -- 172.21.15.83:0/3266079116 learned_addr learned my addr 172.21.15.83:0/3266079116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f4699968640 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2238090 0x561fd23c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 -- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 msgr2=0x561fd23c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd23c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 -- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2238090 msgr2=0x561fd23c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2238090 0x561fd23c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 -- 172.21.15.83:0/3266079116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd21528c0 con 0x561fd2237880 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f469a169640 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4690004780 tx=0x7f46900a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e374b478906bf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46777fe640 1 -- 172.21.15.83:0/3266079116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900b2040 con 0x561fd2237880 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46777fe640 1 -- 172.21.15.83:0/3266079116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900c7040 con 0x561fd2237880 2024-05-17T21:59:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.284+0000 7f46777fe640 1 -- 172.21.15.83:0/3266079116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900d52e0 con 0x561fd2237880 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- 172.21.15.83:0/3266079116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 msgr2=0x561fd2369260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4690004780 tx=0x7f46900a9000 comp rx=0 tx=0).stop 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- 172.21.15.83:0/3266079116 shutdown_connections 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd23c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2238090 0x561fd23c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- 172.21.15.83:0/3266079116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2237880 0x561fd2369260 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- 172.21.15.83:0/3266079116 >> 172.21.15.83:0/3266079116 conn(0x561fd2286010 msgr2=0x561fd23b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- 172.21.15.83:0/3266079116 shutdown_connections 2024-05-17T21:59:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- 172.21.15.83:0/3266079116 wait complete. 2024-05-17T21:59:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- start start 2024-05-17T21:59:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 Processor -- start 2024-05-17T21:59:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd2423dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd2367bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fd215a0f0 con 0x561fd2238090 2024-05-17T21:59:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4694201f90 con 0x561fd2237880 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561fd21bb330 con 0x561fd224af80 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46a3647000 10 client.0 _ll_register_callbacks cb 0x7f4684058310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39050/0 (socket says 172.21.15.83:39050) 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 -- 172.21.15.83:0/2681001613 learned_addr learned my addr 172.21.15.83:0/2681001613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f469a96a640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd2367bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f469a169640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd2423dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 -- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 msgr2=0x561fd2367bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd2367bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 -- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 msgr2=0x561fd2423dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd2423dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 -- 172.21.15.83:0/2681001613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd216dfa0 con 0x561fd2238090 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f469a169640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd2423dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f4699968640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4684058230 tx=0x7f4684013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314511b916098050 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.288+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840577b0 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f468405a910 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46900c7040 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4684082050 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 10 client.5415 fetch_fsmap learned FSMap version 23 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x561fd1e6d220 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900c7040 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f4684085020 con 0x561fd2238090 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 10 client.5415 fetch_fsmap finished waiting for FSMap version 23 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 10 client.5415 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 20 client.5415 populate_metadata read hostname 'smithi083' 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e47990 0x561fd2368100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 4 client.5415 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 --> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fd21c5070 con 0x561fd1e47990 2024-05-17T21:59:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469a169640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e47990 0x561fd2368100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 10 client.5415 ms_handle_connect on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469a169640 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e47990 0x561fd2368100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a605140216b78610 server_cookie=54afe2d77bed3642 in_seq=0 out_seq=0 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 1 -- 172.21.15.83:0/2681001613 <== mds.0 v2:172.21.15.83:6835/2531920228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fd21c5070 con 0x561fd1e47990 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 10 client.5415 handle_command_reply: tid=0 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 client.5415 shutdown 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 msgr2=0x561fd2424310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4684058230 tx=0x7f4684013040 comp rx=0 tx=0).stop 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 shutdown_connections 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:6835/2531920228,v1:172.21.15.83:6837/2531920228] conn(0x561fd1e47990 0x561fd2368100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561fd224af80 0x561fd2367bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fd2238090 0x561fd2424310 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46757fa640 0 client.5415 ms_handle_reset on v2:172.21.15.83:6835/2531920228 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 --2- 172.21.15.83:0/2681001613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fd2237880 0x561fd2423dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 >> 172.21.15.83:0/2681001613 conn(0x561fd2286010 msgr2=0x561fd236ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 shutdown_connections 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 1 -- 172.21.15.83:0/2681001613 wait complete. 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f46a3647000 20 client.5415 trim_cache size 0 max 16384 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T21:59:41.290 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 msgr2=0x7f4678071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 0x7f4678071910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f4690008040 comp rx=0 tx=0).stop 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 msgr2=0x7f4694137b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f468400c7f0 tx=0x7f468400ccb0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 shutdown_connections 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f467806f400 0x7f4678071910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4694143f20 0x7f46941380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46940b3d00 0x7f4694137b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 --2- 172.21.15.83:0/3151935724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46940733c0 0x7f469415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 >> 172.21.15.83:0/3151935724 conn(0x7f469406cd90 msgr2=0x7f46940d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 shutdown_connections 2024-05-17T21:59:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.292+0000 7f469c3cd640 1 -- 172.21.15.83:0/3151935724 wait complete. 2024-05-17T21:59:41.304 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T21:59:41.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:41.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.564+0000 7f13cac87640 1 Processor -- start 2024-05-17T21:59:41.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.564+0000 7f13cac87640 1 -- start start 2024-05-17T21:59:41.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13cac87640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13c41461c0 0x7f13c415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13cac87640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c41465d0 con 0x7f13c415e6b0 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13cac87640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c406ef30 con 0x7f13c40706e0 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13cac87640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f13c406f210 con 0x7f13c41461c0 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c8a23640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13c41461c0 0x7f13c415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13c415e6b0 0x7f13c406ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34072/0 (socket says 172.21.15.83:34072) 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c3fff640 1 -- 172.21.15.83:0/1680194069 learned_addr learned my addr 172.21.15.83:0/1680194069 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c8a23640 1 -- 172.21.15.83:0/1680194069 >> v1:172.21.15.155:6789/0 conn(0x7f13c40706e0 legacy=0x7f13c41605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c8a23640 1 -- 172.21.15.83:0/1680194069 >> v1:172.21.15.83:6789/0 conn(0x7f13c415e6b0 legacy=0x7f13c406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c8a23640 1 -- 172.21.15.83:0/1680194069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c406f4f0 con 0x7f13c41461c0 2024-05-17T21:59:41.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.568+0000 7f13c37fe640 1 -- 172.21.15.83:0/1680194069 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1979851555 0 0) 0x7f13c41465d0 con 0x7f13c415e6b0 2024-05-17T21:59:41.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13c8a23640 1 --2- 172.21.15.83:0/1680194069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c41461c0 0x7f13c415e170 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f13b40097c0 tx=0x7f13b401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236077b69725a5d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13c37fe640 1 -- 172.21.15.83:0/1680194069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b40a7070 con 0x7f13c41461c0 2024-05-17T21:59:41.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13c37fe640 1 -- 172.21.15.83:0/1680194069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b40b0070 con 0x7f13c41461c0 2024-05-17T21:59:41.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13c37fe640 1 -- 172.21.15.83:0/1680194069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b40ab890 con 0x7f13c41461c0 2024-05-17T21:59:41.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- 172.21.15.83:0/1680194069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c41461c0 msgr2=0x7f13c415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 --2- 172.21.15.83:0/1680194069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c41461c0 0x7f13c415e170 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f13b40097c0 tx=0x7f13b401c470 comp rx=0 tx=0).stop 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- 172.21.15.83:0/1680194069 shutdown_connections 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 --2- 172.21.15.83:0/1680194069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c41461c0 0x7f13c415e170 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- 172.21.15.83:0/1680194069 >> 172.21.15.83:0/1680194069 conn(0x7f13c406cd90 msgr2=0x7f13c40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- 172.21.15.83:0/1680194069 shutdown_connections 2024-05-17T21:59:41.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- 172.21.15.83:0/1680194069 wait complete. 2024-05-17T21:59:41.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 Processor -- start 2024-05-17T21:59:41.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.572+0000 7f13cac87640 1 -- start start 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13c40706e0 0x7f13c415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 0x7f13c413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40d10a0 con 0x7f13c415e6b0 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4160ec0 con 0x7f13c40706e0 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13c40d1220 con 0x7f13c41461c0 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39074/0 (socket says 172.21.15.83:39074) 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 -- 172.21.15.83:0/1846176935 learned_addr learned my addr 172.21.15.83:0/1846176935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c3fff640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 0x7f13c413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c8a23640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13c40706e0 0x7f13c415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 -- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 msgr2=0x7f13c413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 0x7f13c413c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 -- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13c40706e0 msgr2=0x7f13c415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13c40706e0 0x7f13c415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 -- 172.21.15.83:0/1846176935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c415b830 con 0x7f13c415e6b0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c3fff640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 0x7f13c413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c9224640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b970 tx=0x7f13bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc019070 con 0x7f13c415e6b0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc00cb40 con 0x7f13c415e6b0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c413e120 con 0x7f13c415e6b0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc014440 con 0x7f13c415e6b0 2024-05-17T21:59:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c413e6e0 con 0x7f13c415e6b0 2024-05-17T21:59:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c415e170 con 0x7f13c415e6b0 2024-05-17T21:59:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.576+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13bc01c070 con 0x7f13c415e6b0 2024-05-17T21:59:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.580+0000 7f13c17fa640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 0x7f13b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.580+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f13bc08ef80 con 0x7f13c415e6b0 2024-05-17T21:59:41.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.580+0000 7f13c8a23640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 0x7f13b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:41.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.580+0000 7f13c8a23640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 0x7f13b0071940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13b40af000 tx=0x7f13b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.584+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f13bc05bec0 con 0x7f13c415e6b0 2024-05-17T21:59:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13c40b41a0 con 0x7f13c415e6b0 2024-05-17T21:59:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13c17fa640 1 -- 172.21.15.83:0/1846176935 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f13bc05b7c0 con 0x7f13c415e6b0 2024-05-17T21:59:41.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:41.770 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 msgr2=0x7f13b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 0x7f13b0071940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13b40af000 tx=0x7f13b4008040 comp rx=0 tx=0).stop 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 msgr2=0x7f13c413c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b970 tx=0x7f13bc00be30 comp rx=0 tx=0).stop 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 shutdown_connections 2024-05-17T21:59:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c415e6b0 0x7f13c413c7b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13c41461c0 0x7f13c413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13b006f430 0x7f13b0071940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.776+0000 7f13cac87640 1 --2- 172.21.15.83:0/1846176935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13c40706e0 0x7f13c415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.780+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 >> 172.21.15.83:0/1846176935 conn(0x7f13c406cd90 msgr2=0x7f13c40d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.780+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 shutdown_connections 2024-05-17T21:59:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:41.780+0000 7f13cac87640 1 -- 172.21.15.83:0/1846176935 wait complete. 2024-05-17T21:59:41.790 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T21:59:41.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T21:59:41.968 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T21:59:41.969 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-17T21:59:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.044+0000 7f563078d080 1 Processor -- start 2024-05-17T21:59:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.044+0000 7f563078d080 1 -- start start 2024-05-17T21:59:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1418c8680 0x55b14183dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b14176d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1416b81b0 con 0x55b141822840 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b141713ef0 con 0x55b1418c8680 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b1417b0030 con 0x55b14176ce50 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39082/0 (socket says 172.21.15.83:39082) 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 -- 172.21.15.83:0/3682922333 learned_addr learned my addr 172.21.15.83:0/3682922333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 -- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 msgr2=0x55b14176d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b14176d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 -- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1418c8680 msgr2=0x55b14183dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1418c8680 0x55b14183dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 -- 172.21.15.83:0/3682922333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b141692c10 con 0x55b141822840 2024-05-17T21:59:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f562aa70640 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55b141795550 tx=0x7f5624005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127ba28d9bebba03 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f5628a6c640 1 -- 172.21.15.83:0/3682922333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624008b70 con 0x55b141822840 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f5628a6c640 1 -- 172.21.15.83:0/3682922333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562401a070 con 0x55b141822840 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f5628a6c640 1 -- 172.21.15.83:0/3682922333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624009e50 con 0x55b141822840 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- 172.21.15.83:0/3682922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 msgr2=0x55b141822c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55b141795550 tx=0x7f5624005a10 comp rx=0 tx=0).stop 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- 172.21.15.83:0/3682922333 shutdown_connections 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b141822840 0x55b141822c50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b14176d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 --2- 172.21.15.83:0/3682922333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1418c8680 0x55b14183dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- 172.21.15.83:0/3682922333 >> 172.21.15.83:0/3682922333 conn(0x55b141754e40 msgr2=0x55b1417968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- 172.21.15.83:0/3682922333 shutdown_connections 2024-05-17T21:59:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- 172.21.15.83:0/3682922333 wait complete. 2024-05-17T21:59:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 Processor -- start 2024-05-17T21:59:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.048+0000 7f563078d080 1 -- start start 2024-05-17T21:59:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b141820a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b141822840 0x55b1417b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 0x55b1417b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1418c95b0 con 0x55b1418c8680 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1417b1e00 con 0x55b141822840 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b1418214d0 con 0x55b14176ce50 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 0x55b1417b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 0x55b1417b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39094/0 (socket says 172.21.15.83:39094) 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 -- 172.21.15.83:0/1672744789 learned_addr learned my addr 172.21.15.83:0/1672744789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 -- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 msgr2=0x55b141820a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562a26f640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b141820a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f5629a6e640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b141822840 0x55b1417b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b14176ce50 0x55b141820a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 -- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b141822840 msgr2=0x55b1417b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b141822840 0x55b1417b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 -- 172.21.15.83:0/1672744789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b141821770 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562aa70640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 0x55b1417b0e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5624015990 tx=0x7f5624002fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 -- 172.21.15.83:0/1672744789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624024030 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 -- 172.21.15.83:0/1672744789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5624015d80 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b141821a10 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 -- 172.21.15.83:0/1672744789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400ae60 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b141821f50 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 -- 172.21.15.83:0/1672744789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5624027030 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f560c06dde0 0x7f560c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f561b7fe640 1 -- 172.21.15.83:0/1672744789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f56240900d0 con 0x55b1418c8680 2024-05-17T21:59:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f562a26f640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f560c06dde0 0x7f560c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55b1418223f0 0x55b14182cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.052+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) -- 0x55b1417af4d0 con 0x55b1418223f0 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f562a26f640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f560c06dde0 0x7f560c0702f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f561c0028f0 tx=0x7f561c0033b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f5629a6e640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55b1418223f0 0x55b14182cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f5629a6e640 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55b1418223f0 0x55b14182cdf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f5629a6e640 1 -- 172.21.15.83:0/1672744789 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f562000e450 con 0x55b1418223f0 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55b1418223f0 msgr2=0x55b14182cdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55b1418223f0 0x55b14182cdf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f560c06dde0 msgr2=0x7f560c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f560c06dde0 0x7f560c0702f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f561c0028f0 tx=0x7f561c0033b0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 msgr2=0x55b1417b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 --2- 172.21.15.83:0/1672744789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1418c8680 0x55b1417b0e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5624015990 tx=0x7f5624002fc0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f562aa70640 1 -- 172.21.15.83:0/1672744789 reap_dead start 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 shutdown_connections 2024-05-17T21:59:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 >> 172.21.15.83:0/1672744789 conn(0x55b141754e40 msgr2=0x55b141797210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:42.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 shutdown_connections 2024-05-17T21:59:42.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.056+0000 7f563078d080 1 -- 172.21.15.83:0/1672744789 wait complete. 2024-05-17T21:59:42.055 INFO:teuthology.orchestra.run.smithi083.stdout:Yalphaparent_a 2024-05-17T21:59:42.055 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "alpha", 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2024-05-17T21:59:42.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "parent_a", 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 7 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 7, 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:42.115 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T21:59:42.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:59:42.214 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:59:42.259 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T21:59:42.259 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:59:42.259 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:59:42.259 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:42.265 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:59:42.265 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:42.312 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T21:59:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.516+0000 7fba4189e640 1 Processor -- start 2024-05-17T21:59:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.516+0000 7fba4189e640 1 -- start start 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba4189e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba3c0cdf40 0x7fba3c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba4189e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c15d530 0x7fba3c1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba4189e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3c0b5640 con 0x7fba3c0cf830 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba4189e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c070870 con 0x7fba3c0cdf40 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba4189e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0b58e0 con 0x7fba3c15d530 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba3c0cdf40 0x7fba3c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba3c0cdf40 0x7fba3c15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39106/0 (socket says 172.21.15.83:39106) 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 -- 172.21.15.83:0/433289214 learned_addr learned my addr 172.21.15.83:0/433289214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 -- 172.21.15.83:0/433289214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c15d530 msgr2=0x7fba3c1663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba3b7fe640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c15d530 0x7fba3c1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c15d530 0x7fba3c1663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 -- 172.21.15.83:0/433289214 >> v1:172.21.15.155:6789/0 conn(0x7fba3c0cf830 legacy=0x7fba3c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.520+0000 7fba327fc640 1 -- 172.21.15.83:0/433289214 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c06f060 con 0x7fba3c0cdf40 2024-05-17T21:59:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba327fc640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cdf40 0x7fba3c15cff0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0cefe0 tx=0x7fba2401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b4481f95085e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba3a7fc640 1 -- 172.21.15.83:0/433289214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240b4070 con 0x7fba3c0cdf40 2024-05-17T21:59:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba3a7fc640 1 -- 172.21.15.83:0/433289214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240a7a50 con 0x7fba3c0cdf40 2024-05-17T21:59:42.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba3a7fc640 1 -- 172.21.15.83:0/433289214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240a7e20 con 0x7fba3c0cdf40 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- 172.21.15.83:0/433289214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cdf40 msgr2=0x7fba3c15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cdf40 0x7fba3c15cff0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0cefe0 tx=0x7fba2401ebf0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- 172.21.15.83:0/433289214 shutdown_connections 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c15d530 0x7fba3c1663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 --2- 172.21.15.83:0/433289214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cdf40 0x7fba3c15cff0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- 172.21.15.83:0/433289214 >> 172.21.15.83:0/433289214 conn(0x7fba3c06cd90 msgr2=0x7fba3c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- 172.21.15.83:0/433289214 shutdown_connections 2024-05-17T21:59:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- 172.21.15.83:0/433289214 wait complete. 2024-05-17T21:59:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 Processor -- start 2024-05-17T21:59:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.524+0000 7fba4189e640 1 -- start start 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 0x7fba3c140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 0x7fba3c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c073a20 con 0x7fba3c0cf830 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c0738a0 con 0x7fba3c15d530 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c073ba0 con 0x7fba3c0cdf40 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39110/0 (socket says 172.21.15.83:39110) 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 -- 172.21.15.83:0/1271416586 learned_addr learned my addr 172.21.15.83:0/1271416586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba3b7fe640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 0x7fba3c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba3affd640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 0x7fba3c140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 -- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 msgr2=0x7fba3c140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 0x7fba3c140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 -- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 msgr2=0x7fba3c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 0x7fba3c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 -- 172.21.15.83:0/1271416586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c14e1e0 con 0x7fba3c0cf830 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba3b7fe640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 0x7fba3c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba3affd640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 0x7fba3c140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba327fc640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b2c0 tx=0x7fba2401b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240b4040 con 0x7fba3c0cf830 2024-05-17T21:59:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240c1070 con 0x7fba3c0cf830 2024-05-17T21:59:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c14e4c0 con 0x7fba3c0cf830 2024-05-17T21:59:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240b96a0 con 0x7fba3c0cf830 2024-05-17T21:59:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c14ea80 con 0x7fba3c0cf830 2024-05-17T21:59:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c1531a0 con 0x7fba3c0cf830 2024-05-17T21:59:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.528+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fba240b7070 con 0x7fba3c0cf830 2024-05-17T21:59:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.532+0000 7fba38ff9640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 0x7fba1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.532+0000 7fba3affd640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 0x7fba1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.532+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fba2412d030 con 0x7fba3c0cf830 2024-05-17T21:59:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.532+0000 7fba3affd640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 0x7fba1c071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00a850 tx=0x7fba2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:42.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.532+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fba240187e0 con 0x7fba3c0cf830 2024-05-17T21:59:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.708+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba3c15cff0 con 0x7fba3c0cf830 2024-05-17T21:59:42.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.708+0000 7fba38ff9640 1 -- 172.21.15.83:0/1271416586 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fba240fa050 con 0x7fba3c0cf830 2024-05-17T21:59:42.703 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T21:59:42.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 msgr2=0x7fba1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 0x7fba1c071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00a850 tx=0x7fba2c008040 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 msgr2=0x7fba3c152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b2c0 tx=0x7fba2401b990 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 shutdown_connections 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba1c06f400 0x7fba1c071910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba3c15d530 0x7fba3c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba3c0cf830 0x7fba3c152770 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 --2- 172.21.15.83:0/1271416586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba3c0cdf40 0x7fba3c140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 >> 172.21.15.83:0/1271416586 conn(0x7fba3c06cd90 msgr2=0x7fba3c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 shutdown_connections 2024-05-17T21:59:42.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:42.712+0000 7fba4189e640 1 -- 172.21.15.83:0/1271416586 wait complete. 2024-05-17T21:59:42.725 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T21:59:42.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:42.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T21:59:42.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[38007]: fuse finished with error 0 and tester_r 0 2024-05-17T21:59:42.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T21:59:42.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T21:59:42.738 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T21:59:42.859 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:59:42.859 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:59:42.859 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:42.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:59:42.867 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:59:42.867 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:59:42.867 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:42.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:59:42.872 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T21:59:42.873 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:59:42.873 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:42.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:59:42.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 Processor -- start 2024-05-17T21:59:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 -- start start 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71c0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71c015fe20 0x7f71c0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c015c830 con 0x7f71c0072e30 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f71c01608a0 con 0x7f71c015fe20 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c7810640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71c0160b80 con 0x7f71c015d2f0 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c5dad640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f71c0072e30 0x7f71c0073240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34090/0 (socket says 172.21.15.83:34090) 2024-05-17T21:59:43.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c5dad640 1 -- 172.21.15.83:0/3717183594 learned_addr learned my addr 172.21.15.83:0/3717183594 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 --2- 172.21.15.83:0/3717183594 >> v2:172.21.15.155:3300/0 conn(0x7f71c015fe20 0x7f71c0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c55ac640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71c0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283214913 0 0) 0x7f71c015c830 con 0x7f71c0072e30 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71ac003530 con 0x7f71c0072e30 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3900978013 0 0) 0x7f71ac003530 con 0x7f71c0072e30 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 msgr2=0x7f71c0159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71c0159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 >> v2:172.21.15.155:3300/0 conn(0x7f71c015fe20 msgr2=0x7f71c0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 --2- 172.21.15.83:0/3717183594 >> v2:172.21.15.155:3300/0 conn(0x7f71c015fe20 0x7f71c0160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c015caf0 con 0x7f71c0072e30 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f71b8001a80 con 0x7f71c0072e30 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 >> v1:172.21.15.83:6789/0 conn(0x7f71c0072e30 legacy=0x7f71c0073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71ac003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac003fe0 0x7f71ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac004d50 0x7f71ac005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01608a0 con 0x7f71ac003fe0 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0160b80 con 0x7f71ac004d50 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac00bb60 con 0x7f71c015d2f0 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac003fe0 0x7f71ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c55ac640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71ac003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c5dad640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac004d50 0x7f71ac005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 -- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 msgr2=0x7f71ac003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c015d2f0 0x7f71ac003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 -- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac004d50 msgr2=0x7f71ac005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac004d50 0x7f71ac005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c4dab640 1 -- 172.21.15.83:0/3717183594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac003530 con 0x7f71ac003fe0 2024-05-17T21:59:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.076+0000 7f71c5dad640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac004d50 0x7f71ac005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:43.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c4dab640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac003fe0 0x7f71ac004810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f71c0072ca0 tx=0x7f71b001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d2a9b719c174da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00096b0 con 0x7f71ac003fe0 2024-05-17T21:59:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b00a6500 con 0x7f71ac003fe0 2024-05-17T21:59:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71b7fff640 1 -- 172.21.15.83:0/3717183594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00a68f0 con 0x7f71ac003fe0 2024-05-17T21:59:43.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 -- 172.21.15.83:0/3717183594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac003fe0 msgr2=0x7f71ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 --2- 172.21.15.83:0/3717183594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71ac003fe0 0x7f71ac004810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f71c0072ca0 tx=0x7f71b001ca10 comp rx=0 tx=0).stop 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c5dad640 1 -- 172.21.15.83:0/3717183594 reap_dead start 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 -- 172.21.15.83:0/3717183594 shutdown_connections 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 -- 172.21.15.83:0/3717183594 >> 172.21.15.83:0/3717183594 conn(0x7f71c006cd90 msgr2=0x7f71c00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 -- 172.21.15.83:0/3717183594 shutdown_connections 2024-05-17T21:59:43.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.080+0000 7f71c7810640 1 -- 172.21.15.83:0/3717183594 wait complete. 2024-05-17T21:59:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 Processor -- start 2024-05-17T21:59:43.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 -- start start 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac003fe0 0x7f71c01f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c01ea140 0x7f71c01ea550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71c00b3d70 con 0x7f71c00d1560 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71c00b3bf0 con 0x7f71ac003fe0 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c7810640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71c00b3ef0 con 0x7f71c01ea140 2024-05-17T21:59:43.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39114/0 (socket says 172.21.15.83:39114) 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 -- 172.21.15.83:0/2713740670 learned_addr learned my addr 172.21.15.83:0/2713740670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c4dab640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c01ea140 0x7f71c01ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c55ac640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac003fe0 0x7f71c01f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 -- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c01ea140 msgr2=0x7f71c01ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c01ea140 0x7f71c01ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 -- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac003fe0 msgr2=0x7f71c01f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac003fe0 0x7f71c01f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 -- 172.21.15.83:0/2713740670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c00d1eb0 con 0x7f71c00d1560 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.084+0000 7f71c5dad640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f90 tx=0x7f71b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8004d70 con 0x7f71c00d1560 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b800bb20 con 0x7f71c00d1560 2024-05-17T21:59:43.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007070 con 0x7f71c00d1560 2024-05-17T21:59:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01e8d40 con 0x7f71c00d1560 2024-05-17T21:59:43.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01e9300 con 0x7f71c00d1560 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0160230 con 0x7f71c00d1560 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f71b801b020 con 0x7f71c00d1560 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 0x7f71a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f71b808f570 con 0x7f71c00d1560 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71c55ac640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 0x7f71a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.088+0000 7f71c55ac640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 0x7f71a4071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b3c0 tx=0x7f71bc00f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.092+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f71c00d1eb0 con 0x7f71c00d1560 2024-05-17T21:59:43.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.288+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71c01eabd0 con 0x7f71c00d1560 2024-05-17T21:59:43.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71b5ffb640 1 -- 172.21.15.83:0/2713740670 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f71b805c410 con 0x7f71c00d1560 2024-05-17T21:59:43.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:43.285 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:43.285 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 msgr2=0x7f71a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 0x7f71a4071910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b3c0 tx=0x7f71bc00f040 comp rx=0 tx=0).stop 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 msgr2=0x7f71c00d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71b8000f90 tx=0x7f71b800b4a0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 shutdown_connections 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c00d1560 0x7f71c00d1970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71c01ea140 0x7f71c01ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71a406f400 0x7f71a4071910 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 --2- 172.21.15.83:0/2713740670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71ac003fe0 0x7f71c01f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 >> 172.21.15.83:0/2713740670 conn(0x7f71c006cd90 msgr2=0x7f71c0073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 shutdown_connections 2024-05-17T21:59:43.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.292+0000 7f71c7810640 1 -- 172.21.15.83:0/2713740670 wait complete. 2024-05-17T21:59:43.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 Processor -- start 2024-05-17T21:59:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 -- start start 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 0x7f3bc415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc406e480 0x7f3bc41698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc40d55e0 con 0x7f3bc40d44a0 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc40b8d00 con 0x7f3bc406e480 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bca58e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc40d4a60 con 0x7f3bc40d0370 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc406e480 0x7f3bc41698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc406e480 0x7f3bc41698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39130/0 (socket says 172.21.15.83:39130) 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3442094978 learned_addr learned my addr 172.21.15.83:0/3442094978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc3fff640 1 --2- 172.21.15.83:0/3442094978 >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 0x7f3bc415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3442094978 >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 msgr2=0x7f3bc415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 --2- 172.21.15.83:0/3442094978 >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 0x7f3bc415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3442094978 >> v1:172.21.15.155:6789/0 conn(0x7f3bc40d44a0 legacy=0x7f3bc406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3442094978 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc40d4d40 con 0x7f3bc406e480 2024-05-17T21:59:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.464+0000 7f3bc3fff640 1 --2- 172.21.15.83:0/3442094978 >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 0x7f3bc415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bc8b2b640 1 --2- 172.21.15.83:0/3442094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc406e480 0x7f3bc41698f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b8410 tx=0x7f3bb401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a7ad5fee319f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bc2ffd640 1 -- 172.21.15.83:0/3442094978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40b0070 con 0x7f3bc406e480 2024-05-17T21:59:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bc2ffd640 1 -- 172.21.15.83:0/3442094978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb40ab600 con 0x7f3bc406e480 2024-05-17T21:59:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bc2ffd640 1 -- 172.21.15.83:0/3442094978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40ab9d0 con 0x7f3bc406e480 2024-05-17T21:59:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- 172.21.15.83:0/3442094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc406e480 msgr2=0x7f3bc41698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3442094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc406e480 0x7f3bc41698f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b8410 tx=0x7f3bb401c520 comp rx=0 tx=0).stop 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- 172.21.15.83:0/3442094978 shutdown_connections 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3442094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc406e480 0x7f3bc41698f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3442094978 >> v2:172.21.15.155:3300/0 conn(0x7f3bc40d0370 0x7f3bc415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- 172.21.15.83:0/3442094978 >> 172.21.15.83:0/3442094978 conn(0x7f3bc4074a40 msgr2=0x7f3bc40d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- 172.21.15.83:0/3442094978 shutdown_connections 2024-05-17T21:59:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- 172.21.15.83:0/3442094978 wait complete. 2024-05-17T21:59:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 Processor -- start 2024-05-17T21:59:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.468+0000 7f3bca58e640 1 -- start start 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bc40d0370 0x7f3bc4157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bc40d44a0 0x7f3bc4158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40bb300 con 0x7f3bc415ee70 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40bb180 con 0x7f3bc40d44a0 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc416c650 con 0x7f3bc40d0370 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39146/0 (socket says 172.21.15.83:39146) 2024-05-17T21:59:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3459123372 learned_addr learned my addr 172.21.15.83:0/3459123372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc37fe640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bc40d44a0 0x7f3bc4158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc3fff640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bc40d0370 0x7f3bc4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bc40d0370 msgr2=0x7f3bc4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bc40d0370 0x7f3bc4157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bc40d44a0 msgr2=0x7f3bc4158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bc40d44a0 0x7f3bc4158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 -- 172.21.15.83:0/3459123372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4156870 con 0x7f3bc415ee70 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc8b2b640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401abd0 tx=0x7f3bb40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40ba070 con 0x7f3bc415ee70 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb40b4620 con 0x7f3bc415ee70 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc4156b50 con 0x7f3bc415ee70 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40b0040 con 0x7f3bc415ee70 2024-05-17T21:59:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc40cb420 con 0x7f3bc415ee70 2024-05-17T21:59:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc415fd00 con 0x7f3bc415ee70 2024-05-17T21:59:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.472+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3bb40bd070 con 0x7f3bc415ee70 2024-05-17T21:59:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.476+0000 7f3bc0ff9640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 0x7f3ba4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.476+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f3bb4128590 con 0x7f3bc415ee70 2024-05-17T21:59:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.476+0000 7f3bc3fff640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 0x7f3ba4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.476+0000 7f3bc3fff640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 0x7f3ba4071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc006fd0 tx=0x7f3bbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.480+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3bb4016110 con 0x7f3bc415ee70 2024-05-17T21:59:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.668+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bc40cb650 con 0x7f3bc415ee70 2024-05-17T21:59:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.668+0000 7f3bc0ff9640 1 -- 172.21.15.83:0/3459123372 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f3bb40f4e80 con 0x7f3bc415ee70 2024-05-17T21:59:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:43.663 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 msgr2=0x7f3ba4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 0x7f3ba4071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc006fd0 tx=0x7f3bbc008040 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 msgr2=0x7f3bc4153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3bb401abd0 tx=0x7f3bb40192c0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 shutdown_connections 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc415ee70 0x7f3bc4153f10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3ba406f430 0x7f3ba4071940 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bc40d44a0 0x7f3bc4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 --2- 172.21.15.83:0/3459123372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bc40d0370 0x7f3bc4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 >> 172.21.15.83:0/3459123372 conn(0x7f3bc4074a40 msgr2=0x7f3bc41623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 shutdown_connections 2024-05-17T21:59:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.672+0000 7f3bca58e640 1 -- 172.21.15.83:0/3459123372 wait complete. 2024-05-17T21:59:43.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:43.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 Processor -- start 2024-05-17T21:59:43.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 -- start start 2024-05-17T21:59:43.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8144080 0x7fc0e8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc0e8073370 0x7fc0e806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e8073780 con 0x7fc0e80d1040 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e815e790 con 0x7fc0e8144080 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0f01eb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e815ea40 con 0x7fc0e8073370 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8144080 0x7fc0e8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e8144080 0x7fc0e8072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39154/0 (socket says 172.21.15.83:39154) 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 -- 172.21.15.83:0/960765777 learned_addr learned my addr 172.21.15.83:0/960765777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ee788640 1 --2- 172.21.15.83:0/960765777 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8073370 0x7fc0e806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 -- 172.21.15.83:0/960765777 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8073370 msgr2=0x7fc0e806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 --2- 172.21.15.83:0/960765777 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8073370 0x7fc0e806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 -- 172.21.15.83:0/960765777 >> v1:172.21.15.155:6789/0 conn(0x7fc0e80d1040 legacy=0x7fc0e815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:43.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.824+0000 7fc0ed786640 1 -- 172.21.15.83:0/960765777 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e815ed20 con 0x7fc0e8144080 2024-05-17T21:59:43.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0ed786640 1 --2- 172.21.15.83:0/960765777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8144080 0x7fc0e8072e30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8072c30 tx=0x7fc0d801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17308e770cf06cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0ecf85640 1 -- 172.21.15.83:0/960765777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b0070 con 0x7fc0e8144080 2024-05-17T21:59:43.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0ecf85640 1 -- 172.21.15.83:0/960765777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d80ab600 con 0x7fc0e8144080 2024-05-17T21:59:43.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0ecf85640 1 -- 172.21.15.83:0/960765777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80ab9d0 con 0x7fc0e8144080 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- 172.21.15.83:0/960765777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8144080 msgr2=0x7fc0e8072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/960765777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8144080 0x7fc0e8072e30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8072c30 tx=0x7fc0d801c310 comp rx=0 tx=0).stop 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- 172.21.15.83:0/960765777 shutdown_connections 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/960765777 >> v2:172.21.15.155:3300/0 conn(0x7fc0e8073370 0x7fc0e806f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/960765777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e8144080 0x7fc0e8072e30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- 172.21.15.83:0/960765777 >> 172.21.15.83:0/960765777 conn(0x7fc0e806cd90 msgr2=0x7fc0e80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- 172.21.15.83:0/960765777 shutdown_connections 2024-05-17T21:59:43.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- 172.21.15.83:0/960765777 wait complete. 2024-05-17T21:59:43.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 Processor -- start 2024-05-17T21:59:43.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.828+0000 7fc0f01eb640 1 -- start start 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e8073370 0x7fc0e815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e8144080 0x7fc0e815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8070480 con 0x7fc0e80d1040 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8144490 con 0x7fc0e8073370 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e8070600 con 0x7fc0e8144080 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39164/0 (socket says 172.21.15.83:39164) 2024-05-17T21:59:43.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 -- 172.21.15.83:0/4092184576 learned_addr learned my addr 172.21.15.83:0/4092184576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 -- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e8144080 msgr2=0x7fc0e815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ee788640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e8144080 0x7fc0e815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0edf87640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e8073370 0x7fc0e815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e8144080 0x7fc0e815c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 -- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e8073370 msgr2=0x7fc0e815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e8073370 0x7fc0e815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 -- 172.21.15.83:0/4092184576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e815ce50 con 0x7fc0e80d1040 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0ed786640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ae040 tx=0x7fc0d801ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b0040 con 0x7fc0e80d1040 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d80bc070 con 0x7fc0e80d1040 2024-05-17T21:59:43.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e815d130 con 0x7fc0e80d1040 2024-05-17T21:59:43.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b6880 con 0x7fc0e80d1040 2024-05-17T21:59:43.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e81420b0 con 0x7fc0e80d1040 2024-05-17T21:59:43.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e815cba0 con 0x7fc0e80d1040 2024-05-17T21:59:43.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc0d80ab4c0 con 0x7fc0e80d1040 2024-05-17T21:59:43.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 0x7fc0cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:43.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fc0d8129800 con 0x7fc0e80d1040 2024-05-17T21:59:43.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.832+0000 7fc0edf87640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 0x7fc0cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:43.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.836+0000 7fc0edf87640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 0x7fc0cc071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400a750 tx=0x7fc0e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:43.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.840+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc0e8073780 con 0x7fc0e80d1040 2024-05-17T21:59:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.988+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0e8070990 con 0x7fc0e80d1040 2024-05-17T21:59:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.988+0000 7fc0de7fc640 1 -- 172.21.15.83:0/4092184576 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16680 (secure 0 0 0) 0x7fc0d80f6740 con 0x7fc0e80d1040 2024-05-17T21:59:43.984 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:43.984 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:12.063712+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:57.908827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:59.153525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.32","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 msgr2=0x7fc0cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 0x7fc0cc071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400a750 tx=0x7fc0e4008040 comp rx=0 tx=0).stop 2024-05-17T21:59:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 msgr2=0x7fc0e815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ae040 tx=0x7fc0d801ab50 comp rx=0 tx=0).stop 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 shutdown_connections 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc0e8144080 0x7fc0e815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e80d1040 0x7fc0e815c090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc0cc06f400 0x7fc0cc071910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 --2- 172.21.15.83:0/4092184576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0e8073370 0x7fc0e815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 >> 172.21.15.83:0/4092184576 conn(0x7fc0e806cd90 msgr2=0x7fc0e80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:43.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 shutdown_connections 2024-05-17T21:59:43.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:43.992+0000 7fc0f01eb640 1 -- 172.21.15.83:0/4092184576 wait complete. 2024-05-17T21:59:44.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T21:59:44.004 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 Processor -- start 2024-05-17T21:59:44.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 -- start start 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a6806cea0 0x7f0a680cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6806f7e0 con 0x7f0a680d0150 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0a68160430 con 0x7f0a6806cea0 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6f5f6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680b19d0 con 0x7f0a6806d2b0 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6cb91640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a6806cea0 0x7f0a680cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6d392640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a680d0150 0x7f0a6806c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34098/0 (socket says 172.21.15.83:34098) 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6d392640 1 -- 172.21.15.83:0/2446303772 learned_addr learned my addr 172.21.15.83:0/2446303772 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6db93640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6cb91640 1 -- 172.21.15.83:0/2446303772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 msgr2=0x7f0a680cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.156+0000 7f0a6cb91640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.160+0000 7f0a6cb91640 1 -- 172.21.15.83:0/2446303772 >> v1:172.21.15.83:6789/0 conn(0x7f0a680d0150 legacy=0x7f0a6806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.160+0000 7f0a6cb91640 1 -- 172.21.15.83:0/2446303772 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680b1cb0 con 0x7f0a6806cea0 2024-05-17T21:59:44.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.160+0000 7f0a6db93640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:44.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.160+0000 7f0a6cb91640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6806cea0 0x7f0a680cd7e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097c0 tx=0x7f0a5801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a0831e45845a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a5ffff640 1 -- 172.21.15.83:0/2446303772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580b1070 con 0x7f0a6806cea0 2024-05-17T21:59:44.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a5ffff640 1 -- 172.21.15.83:0/2446303772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a580a8050 con 0x7f0a6806cea0 2024-05-17T21:59:44.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a5ffff640 1 -- 172.21.15.83:0/2446303772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a580ac860 con 0x7f0a6806cea0 2024-05-17T21:59:44.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/2446303772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6806cea0 msgr2=0x7f0a680cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6806cea0 0x7f0a680cd7e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0a580097c0 tx=0x7f0a5801c3e0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/2446303772 shutdown_connections 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/2446303772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a6806cea0 0x7f0a680cd7e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/2446303772 >> 172.21.15.83:0/2446303772 conn(0x7f0a680b4170 msgr2=0x7f0a680b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/2446303772 shutdown_connections 2024-05-17T21:59:44.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/2446303772 wait complete. 2024-05-17T21:59:44.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 Processor -- start 2024-05-17T21:59:44.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.164+0000 7f0a6f5f6640 1 -- start start 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6806cea0 0x7f0a681582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680d0560 con 0x7f0a680d0150 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680b3ea0 con 0x7f0a6806cea0 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680d06e0 con 0x7f0a6806d2b0 2024-05-17T21:59:44.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39184/0 (socket says 172.21.15.83:39184) 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 -- 172.21.15.83:0/3210284630 learned_addr learned my addr 172.21.15.83:0/3210284630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6d392640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6806cea0 0x7f0a681582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6cb91640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 -- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 msgr2=0x7f0a680b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 -- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6806cea0 msgr2=0x7f0a681582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6806cea0 0x7f0a681582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 -- 172.21.15.83:0/3210284630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680c7550 con 0x7f0a680d0150 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6db93640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000cde0 tx=0x7f0a6000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000f680 con 0x7f0a680d0150 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6000f840 con 0x7f0a680d0150 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000fc30 con 0x7f0a680d0150 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a680c64b0 con 0x7f0a680d0150 2024-05-17T21:59:44.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a680c6a70 con 0x7f0a680d0150 2024-05-17T21:59:44.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a680c6220 con 0x7f0a680d0150 2024-05-17T21:59:44.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a600213f0 con 0x7f0a680d0150 2024-05-17T21:59:44.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 0x7f0a4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a6d392640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 0x7f0a4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.168+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f0a600900b0 con 0x7f0a680d0150 2024-05-17T21:59:44.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.172+0000 7f0a6d392640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 0x7f0a4c071940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000f70 tx=0x7f0a64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.172+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0a60014030 con 0x7f0a680d0150 2024-05-17T21:59:44.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.368+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a680d1b00 con 0x7f0a680d0150 2024-05-17T21:59:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a5dffb640 1 -- 172.21.15.83:0/3210284630 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7f0a6005cf60 con 0x7f0a680d0150 2024-05-17T21:59:44.364 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:44.364 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:44.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 msgr2=0x7f0a4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 0x7f0a4c071940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a64000f70 tx=0x7f0a64009040 comp rx=0 tx=0).stop 2024-05-17T21:59:44.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 msgr2=0x7f0a680b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000cde0 tx=0x7f0a6000a650 comp rx=0 tx=0).stop 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 shutdown_connections 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a680d0150 0x7f0a680b27c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a6806d2b0 0x7f0a680b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a4c06f430 0x7f0a4c071940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 --2- 172.21.15.83:0/3210284630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6806cea0 0x7f0a681582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 >> 172.21.15.83:0/3210284630 conn(0x7f0a680b4170 msgr2=0x7f0a680b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 shutdown_connections 2024-05-17T21:59:44.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.372+0000 7f0a6f5f6640 1 -- 172.21.15.83:0/3210284630 wait complete. 2024-05-17T21:59:44.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 Processor -- start 2024-05-17T21:59:44.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 -- start start 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcd40cf2e0 0x7fbcd409b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbcd409c570 0x7fbcd409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd40d1950 con 0x7fbcd40b8de0 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd40d1670 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcda23d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd4071280 con 0x7fbcd409c570 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcd40cf2e0 0x7fbcd409b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcd40cf2e0 0x7fbcd409b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39192/0 (socket says 172.21.15.83:39192) 2024-05-17T21:59:44.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 -- 172.21.15.83:0/3442110419 learned_addr learned my addr 172.21.15.83:0/3442110419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd2ffd640 1 --2- 172.21.15.83:0/3442110419 >> v2:172.21.15.155:3300/0 conn(0x7fbcd409c570 0x7fbcd409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 -- 172.21.15.83:0/3442110419 >> v2:172.21.15.155:3300/0 conn(0x7fbcd409c570 msgr2=0x7fbcd409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 --2- 172.21.15.83:0/3442110419 >> v2:172.21.15.155:3300/0 conn(0x7fbcd409c570 0x7fbcd409bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 -- 172.21.15.83:0/3442110419 >> v1:172.21.15.155:6789/0 conn(0x7fbcd40b8de0 legacy=0x7fbcd40ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:44.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.540+0000 7fbcd37fe640 1 -- 172.21.15.83:0/3442110419 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd4071560 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcd37fe640 1 --2- 172.21.15.83:0/3442110419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd409b990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcd408cef0 tx=0x7fbcc401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede3e52fe22b9607 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcd27fc640 1 -- 172.21.15.83:0/3442110419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40b5070 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcd27fc640 1 -- 172.21.15.83:0/3442110419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc40a89d0 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcd27fc640 1 -- 172.21.15.83:0/3442110419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40a8da0 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 -- 172.21.15.83:0/3442110419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40cf2e0 msgr2=0x7fbcd409b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 --2- 172.21.15.83:0/3442110419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd409b990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcd408cef0 tx=0x7fbcc401ed10 comp rx=0 tx=0).stop 2024-05-17T21:59:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 -- 172.21.15.83:0/3442110419 shutdown_connections 2024-05-17T21:59:44.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 --2- 172.21.15.83:0/3442110419 >> v2:172.21.15.155:3300/0 conn(0x7fbcd409c570 0x7fbcd409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 --2- 172.21.15.83:0/3442110419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd409b990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 -- 172.21.15.83:0/3442110419 >> 172.21.15.83:0/3442110419 conn(0x7fbcd40b5120 msgr2=0x7fbcd40b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 -- 172.21.15.83:0/3442110419 shutdown_connections 2024-05-17T21:59:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.544+0000 7fbcda23d640 1 -- 172.21.15.83:0/3442110419 wait complete. 2024-05-17T21:59:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 Processor -- start 2024-05-17T21:59:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- start start 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcd409c570 0x7fbcd40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd41f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd40d1380 con 0x7fbcd40b8de0 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd40d1200 con 0x7fbcd40cf2e0 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd4072d70 con 0x7fbcd409c570 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39206/0 (socket says 172.21.15.83:39206) 2024-05-17T21:59:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 -- 172.21.15.83:0/1959707380 learned_addr learned my addr 172.21.15.83:0/1959707380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd37fe640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcd409c570 0x7fbcd40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 -- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcd409c570 msgr2=0x7fbcd40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd3fff640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd41f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcd409c570 0x7fbcd40cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 -- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 msgr2=0x7fbcd41f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd41f99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 -- 172.21.15.83:0/1959707380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd40b2c80 con 0x7fbcd40b8de0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd3fff640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd41f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcd2ffd640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ccd0 tx=0x7fbcbc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0039a0 con 0x7fbcd40b8de0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc003b40 con 0x7fbcd40b8de0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd406e690 con 0x7fbcd40b8de0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc004020 con 0x7fbcd40b8de0 2024-05-17T21:59:44.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd406ec50 con 0x7fbcd40b8de0 2024-05-17T21:59:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.548+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd406e400 con 0x7fbcd40b8de0 2024-05-17T21:59:44.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.552+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbcbc0041c0 con 0x7fbcd40b8de0 2024-05-17T21:59:44.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.552+0000 7fbccbfff640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 0x7fbcb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.552+0000 7fbcd37fe640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 0x7fbcb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.552+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fbcbc08f7c0 con 0x7fbcd40b8de0 2024-05-17T21:59:44.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.552+0000 7fbcd37fe640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 0x7fbcb4071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40b2000 tx=0x7fbcc40aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.556+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbcbc014030 con 0x7fbcd40b8de0 2024-05-17T21:59:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.744+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcd409b990 con 0x7fbcd40b8de0 2024-05-17T21:59:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbccbfff640 1 -- 172.21.15.83:0/1959707380 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5184 (secure 0 0 0) 0x7fbcbc05c670 con 0x7fbcd40b8de0 2024-05-17T21:59:44.740 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:44.740 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:00.239967+0000","modified":"2024-05-17T21:59:03.215524+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2531920228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2531920228},{"type":"v1","addr":"172.21.15.83:6837","nonce":2531920228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":3}]} 2024-05-17T21:59:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-05-17T21:59:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 msgr2=0x7fbcb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 0x7fbcb4071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40b2000 tx=0x7fbcc40aa070 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 msgr2=0x7fbcd40cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ccd0 tx=0x7fbcbc00a5c0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 shutdown_connections 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbcb406f400 0x7fbcb4071910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcd40cf2e0 0x7fbcd41f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcd40b8de0 0x7fbcd40cb610 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 --2- 172.21.15.83:0/1959707380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcd409c570 0x7fbcd40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 >> 172.21.15.83:0/1959707380 conn(0x7fbcd40b5120 msgr2=0x7fbcd40b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 shutdown_connections 2024-05-17T21:59:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.748+0000 7fbcda23d640 1 -- 172.21.15.83:0/1959707380 wait complete. 2024-05-17T21:59:44.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.900+0000 7febf859b640 1 Processor -- start 2024-05-17T21:59:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.900+0000 7febf859b640 1 -- start start 2024-05-17T21:59:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf859b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7febf0144080 0x7febf0072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf859b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf859b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf0073780 con 0x7febf00d1040 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf859b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7febf015e790 con 0x7febf0144080 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf859b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febf015ea40 con 0x7febf0073370 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7febf0144080 0x7febf0072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf6337640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7febf00d1040 0x7febf015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34114/0 (socket says 172.21.15.83:34114) 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf6337640 1 -- 172.21.15.83:0/1992434167 learned_addr learned my addr 172.21.15.83:0/1992434167 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf6b38640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 -- 172.21.15.83:0/1992434167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 msgr2=0x7febf006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 -- 172.21.15.83:0/1992434167 >> v1:172.21.15.83:6789/0 conn(0x7febf00d1040 legacy=0x7febf015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5335640 1 -- 172.21.15.83:0/1992434167 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363403130 0 0) 0x7febec001120 con 0x7febf00d1040 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 -- 172.21.15.83:0/1992434167 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf015ed20 con 0x7febf0144080 2024-05-17T21:59:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf6b38640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.904+0000 7febf5b36640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0144080 0x7febf0072e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7febe40097c0 tx=0x7febe401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8996d8b7f68574c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf5335640 1 -- 172.21.15.83:0/1992434167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40a7070 con 0x7febf0144080 2024-05-17T21:59:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf5335640 1 -- 172.21.15.83:0/1992434167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe40b0070 con 0x7febf0144080 2024-05-17T21:59:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf5335640 1 -- 172.21.15.83:0/1992434167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40ab890 con 0x7febf0144080 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- 172.21.15.83:0/1992434167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0144080 msgr2=0x7febf0072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0144080 0x7febf0072e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7febe40097c0 tx=0x7febe401c470 comp rx=0 tx=0).stop 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- 172.21.15.83:0/1992434167 shutdown_connections 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 --2- 172.21.15.83:0/1992434167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0144080 0x7febf0072e30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- 172.21.15.83:0/1992434167 >> 172.21.15.83:0/1992434167 conn(0x7febf006cd90 msgr2=0x7febf00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- 172.21.15.83:0/1992434167 shutdown_connections 2024-05-17T21:59:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- 172.21.15.83:0/1992434167 wait complete. 2024-05-17T21:59:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 Processor -- start 2024-05-17T21:59:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.908+0000 7febf859b640 1 -- start start 2024-05-17T21:59:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf00afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 0x7febf0166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febf0070480 con 0x7febf00d1040 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7febf0144490 con 0x7febf0144080 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7febf0070600 con 0x7febf0073370 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39218/0 (socket says 172.21.15.83:39218) 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 -- 172.21.15.83:0/900204504 learned_addr learned my addr 172.21.15.83:0/900204504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6337640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf00afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6b38640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 0x7febf0166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 -- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 msgr2=0x7febf00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf00afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 -- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 msgr2=0x7febf0166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 0x7febf0166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 -- 172.21.15.83:0/900204504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf012c9b0 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6337640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6b38640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 0x7febf0166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf5b36640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7febe401a0a0 tx=0x7febe401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40bf610 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe40bf7d0 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40bfbc0 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf012cc90 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf012d250 con 0x7febf00d1040 2024-05-17T21:59:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf006f730 con 0x7febf00d1040 2024-05-17T21:59:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7febe40bd020 con 0x7febf00d1040 2024-05-17T21:59:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 0x7febd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6337640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 0x7febd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7febe4129ce0 con 0x7febf00d1040 2024-05-17T21:59:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.912+0000 7febf6337640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 0x7febd4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febec007fd0 tx=0x7febec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:44.916+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7febe4016090 con 0x7febf00d1040 2024-05-17T21:59:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.076+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febf01435d0 con 0x7febf00d1040 2024-05-17T21:59:45.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.076+0000 7febe2ffd640 1 -- 172.21.15.83:0/900204504 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16680 (secure 0 0 0) 0x7febe40f6c20 con 0x7febf00d1040 2024-05-17T21:59:45.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:45.072 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:12.063712+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:58:57.908827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-17T21:58:59.153525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.32","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 msgr2=0x7febd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 0x7febd4071910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febec007fd0 tx=0x7febec009040 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 msgr2=0x7febf0166830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7febe401a0a0 tx=0x7febe401acb0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 shutdown_connections 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7febd406f400 0x7febd4071910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf0144080 0x7febf0166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d1040 0x7febf0166830 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 --2- 172.21.15.83:0/900204504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7febf0073370 0x7febf00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 >> 172.21.15.83:0/900204504 conn(0x7febf006cd90 msgr2=0x7febf01457b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 shutdown_connections 2024-05-17T21:59:45.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.080+0000 7febf859b640 1 -- 172.21.15.83:0/900204504 wait complete. 2024-05-17T21:59:45.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T21:59:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 Processor -- start 2024-05-17T21:59:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 -- start start 2024-05-17T21:59:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b5806c790 0x7f0b5806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b580b5670 0x7f0b580b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b580b5fc0 con 0x7f0b5815fcc0 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0b580711a0 con 0x7f0b580b5670 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5e50d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5806cc90 con 0x7f0b5806c790 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5caaa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b580b5670 0x7f0b580b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b577fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0b5815fcc0 0x7f0b58070a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34124/0 (socket says 172.21.15.83:34124) 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b577fe640 1 -- 172.21.15.83:0/303684730 learned_addr learned my addr 172.21.15.83:0/303684730 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b57fff640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b5806c790 0x7f0b5806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5caaa640 1 -- 172.21.15.83:0/303684730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b5806c790 msgr2=0x7f0b5806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5caaa640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b5806c790 0x7f0b5806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5caaa640 1 -- 172.21.15.83:0/303684730 >> v1:172.21.15.83:6789/0 conn(0x7f0b5815fcc0 legacy=0x7f0b58070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b5caaa640 1 -- 172.21.15.83:0/303684730 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5806cf70 con 0x7f0b580b5670 2024-05-17T21:59:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.244+0000 7f0b56ffd640 1 -- 172.21.15.83:0/303684730 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469270426 0 0) 0x7f0b4c001120 con 0x7f0b5815fcc0 2024-05-17T21:59:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5caaa640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b580b5670 0x7f0b580b5a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009d90 tx=0x7f0b5001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e9a153831706d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b56ffd640 1 -- 172.21.15.83:0/303684730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b500b2070 con 0x7f0b580b5670 2024-05-17T21:59:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b56ffd640 1 -- 172.21.15.83:0/303684730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b500a5a00 con 0x7f0b580b5670 2024-05-17T21:59:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b56ffd640 1 -- 172.21.15.83:0/303684730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b500a5dd0 con 0x7f0b580b5670 2024-05-17T21:59:45.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 -- 172.21.15.83:0/303684730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b580b5670 msgr2=0x7f0b580b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b580b5670 0x7f0b580b5a80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009d90 tx=0x7f0b5001c940 comp rx=0 tx=0).stop 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 -- 172.21.15.83:0/303684730 shutdown_connections 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b580b5670 0x7f0b580b5a80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/303684730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b5806c790 0x7f0b5806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 -- 172.21.15.83:0/303684730 >> 172.21.15.83:0/303684730 conn(0x7f0b580d20b0 msgr2=0x7f0b580d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 -- 172.21.15.83:0/303684730 shutdown_connections 2024-05-17T21:59:45.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.248+0000 7f0b5e50d640 1 -- 172.21.15.83:0/303684730 wait complete. 2024-05-17T21:59:45.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 Processor -- start 2024-05-17T21:59:45.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- start start 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 0x7f0b58142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5815fcc0 0x7f0b580b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b580af910 con 0x7f0b5806c790 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0b580af790 con 0x7f0b5815fcc0 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0b580afa90 con 0x7f0b580b5670 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39228/0 (socket says 172.21.15.83:39228) 2024-05-17T21:59:45.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 -- 172.21.15.83:0/3605382741 learned_addr learned my addr 172.21.15.83:0/3605382741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b577fe640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 0x7f0b58142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 -- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 msgr2=0x7f0b58142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 0x7f0b58142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 -- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5815fcc0 msgr2=0x7f0b580b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5815fcc0 0x7f0b580b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 -- 172.21.15.83:0/3605382741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b58159f70 con 0x7f0b5806c790 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b577fe640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 0x7f0b58142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b57fff640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d5f0 tx=0x7f0b4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480125f0 con 0x7f0b5806c790 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b48019070 con 0x7f0b5806c790 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5815a250 con 0x7f0b5806c790 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480129b0 con 0x7f0b5806c790 2024-05-17T21:59:45.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5815a810 con 0x7f0b5806c790 2024-05-17T21:59:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.252+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5806dd30 con 0x7f0b5806c790 2024-05-17T21:59:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.256+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0b48012b50 con 0x7f0b5806c790 2024-05-17T21:59:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.256+0000 7f0b54ff9640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 0x7f0b44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.256+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f0b48090050 con 0x7f0b5806c790 2024-05-17T21:59:45.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.256+0000 7f0b577fe640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 0x7f0b44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.256+0000 7f0b577fe640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 0x7f0b44071940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b581fca20 tx=0x7f0b4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.260+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0b4805d070 con 0x7f0b5806c790 2024-05-17T21:59:45.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.444+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0b580b36f0 con 0x7f0b5806c790 2024-05-17T21:59:45.437 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T21:59:45.444+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T21:59:45.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.444+0000 7f0b54ff9640 1 -- 172.21.15.83:0/3605382741 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7f0b4805c970 con 0x7f0b5806c790 2024-05-17T21:59:45.440 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T21:59:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 msgr2=0x7f0b44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 0x7f0b44071940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b581fca20 tx=0x7f0b4c009040 comp rx=0 tx=0).stop 2024-05-17T21:59:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 msgr2=0x7f0b580b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800d5f0 tx=0x7f0b4800dab0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 shutdown_connections 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0b4406f430 0x7f0b44071940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b5815fcc0 0x7f0b580b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b580b5670 0x7f0b58142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 --2- 172.21.15.83:0/3605382741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b5806c790 0x7f0b580b2180 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 >> 172.21.15.83:0/3605382741 conn(0x7f0b580d20b0 msgr2=0x7f0b5815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 shutdown_connections 2024-05-17T21:59:45.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.448+0000 7f0b5e50d640 1 -- 172.21.15.83:0/3605382741 wait complete. 2024-05-17T21:59:45.449 INFO:tasks.ceph.mds.a.smithi083.stderr: -4988> 2024-05-17T21:59:01.389+0000 7ff47d4ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:59:45.454 INFO:tasks.ceph.mds.a.smithi083.stderr: -3082> 2024-05-17T21:59:10.841+0000 7ff4794a4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5328=0-4194304@1} caps={5328=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5328 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x563b1b6bcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T21:59:45.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T21:59:45.527 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:59:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 Processor -- start 2024-05-17T21:59:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 -- start start 2024-05-17T21:59:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff8180b3d00 0x7ff8180d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff818143f20 0x7ff818072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8180733c0 0x7ff81806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff8180b4110 con 0x7ff8180733c0 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff818070770 con 0x7ff8180b3d00 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff81dd86640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff8180737d0 con 0x7ff818143f20 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8180733c0 0x7ff81806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8180733c0 0x7ff81806eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39266/0 (socket says 172.21.15.83:39266) 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 -- 172.21.15.83:0/3834871874 learned_addr learned my addr 172.21.15.83:0/3834871874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff8177fe640 1 --2- 172.21.15.83:0/3834871874 >> v2:172.21.15.155:3300/0 conn(0x7ff8180b3d00 0x7ff8180d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff816ffd640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff818143f20 0x7ff818072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 -- 172.21.15.83:0/3834871874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff818143f20 msgr2=0x7ff818072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff818143f20 0x7ff818072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 -- 172.21.15.83:0/3834871874 >> v2:172.21.15.155:3300/0 conn(0x7ff8180b3d00 msgr2=0x7ff8180d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:45.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 --2- 172.21.15.83:0/3834871874 >> v2:172.21.15.155:3300/0 conn(0x7ff8180b3d00 0x7ff8180d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.628+0000 7ff817fff640 1 -- 172.21.15.83:0/3834871874 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8180d10d0 con 0x7ff8180733c0 2024-05-17T21:59:45.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff817fff640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81806eec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff81806d6e0 tx=0x7ff81001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9374370197c00dba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff8167fc640 1 -- 172.21.15.83:0/3834871874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100a5840 con 0x7ff8180733c0 2024-05-17T21:59:45.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff8167fc640 1 -- 172.21.15.83:0/3834871874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8100a59e0 con 0x7ff8180733c0 2024-05-17T21:59:45.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff8167fc640 1 -- 172.21.15.83:0/3834871874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100a5db0 con 0x7ff8180733c0 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- 172.21.15.83:0/3834871874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 msgr2=0x7ff81806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81806eec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff81806d6e0 tx=0x7ff81001c9c0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- 172.21.15.83:0/3834871874 shutdown_connections 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81806eec0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 --2- 172.21.15.83:0/3834871874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff818143f20 0x7ff818072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 --2- 172.21.15.83:0/3834871874 >> v2:172.21.15.155:3300/0 conn(0x7ff8180b3d00 0x7ff8180d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- 172.21.15.83:0/3834871874 >> 172.21.15.83:0/3834871874 conn(0x7ff81806cd90 msgr2=0x7ff8180b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- 172.21.15.83:0/3834871874 shutdown_connections 2024-05-17T21:59:45.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- 172.21.15.83:0/3834871874 wait complete. 2024-05-17T21:59:45.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 Processor -- start 2024-05-17T21:59:45.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.632+0000 7ff81dd86640 1 -- start start 2024-05-17T21:59:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 0x7ff8180b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff818143f20 0x7ff8180b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff818070c00 con 0x7ff8180733c0 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff818144330 con 0x7ff818143f20 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff81815e9b0 con 0x7ff8180b3d00 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39274/0 (socket says 172.21.15.83:39274) 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 -- 172.21.15.83:0/4001142145 learned_addr learned my addr 172.21.15.83:0/4001142145 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff816ffd640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 0x7ff8180b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff817fff640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff818143f20 0x7ff8180b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 -- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 msgr2=0x7ff8180b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 0x7ff8180b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 -- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff818143f20 msgr2=0x7ff8180b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff818143f20 0x7ff8180b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 -- 172.21.15.83:0/4001142145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff818143820 con 0x7ff8180733c0 2024-05-17T21:59:45.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff816ffd640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 0x7ff8180b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff8177fe640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff80800d5f0 tx=0x7ff80800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8080125f0 con 0x7ff8180733c0 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff808019070 con 0x7ff8180733c0 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff818141fc0 con 0x7ff8180733c0 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8080129b0 con 0x7ff8180733c0 2024-05-17T21:59:45.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8180b5500 con 0x7ff8180733c0 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81806fc10 con 0x7ff8180733c0 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.636+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff808012b50 con 0x7ff8180733c0 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.640+0000 7ff7fffff640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 0x7ff804071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.640+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7ff80808f710 con 0x7ff8180733c0 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.640+0000 7ff816ffd640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 0x7ff804071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:45.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.640+0000 7ff816ffd640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 0x7ff804071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff818150ad0 tx=0x7ff80c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:45.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.644+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff818143820 con 0x7ff8180733c0 2024-05-17T21:59:45.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:45.824+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff81813c870 con 0x7ff8180733c0 2024-05-17T21:59:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.448+0000 7ff7fffff640 1 -- 172.21.15.83:0/4001142145 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7ff80805c600 con 0x7ff8180733c0 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 msgr2=0x7ff804071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 0x7ff804071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff818150ad0 tx=0x7ff80c008040 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 msgr2=0x7ff81815e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff80800d5f0 tx=0x7ff80800dab0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 shutdown_connections 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80406f430 0x7ff804071940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff818143f20 0x7ff8180b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8180b3d00 0x7ff8180b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 --2- 172.21.15.83:0/4001142145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8180733c0 0x7ff81815e140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 >> 172.21.15.83:0/4001142145 conn(0x7ff81806cd90 msgr2=0x7ff818145550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 shutdown_connections 2024-05-17T21:59:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.452+0000 7ff81dd86640 1 -- 172.21.15.83:0/4001142145 wait complete. 2024-05-17T21:59:46.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T21:59:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 Processor -- start 2024-05-17T21:59:46.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 -- start start 2024-05-17T21:59:46.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9a6c0b0bc0 0x7f9a6c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6c15ec60 con 0x7f9a6c073560 2024-05-17T21:59:46.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a6c0b5be0 con 0x7f9a6c0b66c0 2024-05-17T21:59:46.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a70bc3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9a6c0b6230 con 0x7f9a6c0b0bc0 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a69d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9a6c073560 0x7f9a6c15df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34144/0 (socket says 172.21.15.83:34144) 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a69d9b640 1 -- 172.21.15.83:0/848925088 learned_addr learned my addr 172.21.15.83:0/848925088 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a6ad9d640 1 --2- 172.21.15.83:0/848925088 >> v2:172.21.15.155:3300/0 conn(0x7f9a6c0b0bc0 0x7f9a6c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.636+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515453196 0 0) 0x7f9a6c15ec60 con 0x7f9a6c073560 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a54003530 con 0x7f9a6c073560 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2460856684 0 0) 0x7f9a54003530 con 0x7f9a6c073560 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 >> v2:172.21.15.155:3300/0 conn(0x7f9a6c0b0bc0 msgr2=0x7f9a6c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 --2- 172.21.15.83:0/848925088 >> v2:172.21.15.155:3300/0 conn(0x7f9a6c0b0bc0 0x7f9a6c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 >> v1:172.21.15.155:6789/0 conn(0x7f9a6c0b66c0 legacy=0x7f9a6c072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c0b5ec0 con 0x7f9a6c073560 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9a60001700 con 0x7f9a6c073560 2024-05-17T21:59:46.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 >> v1:172.21.15.83:6789/0 conn(0x7f9a6c073560 legacy=0x7f9a6c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a54003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004810 0x7f9a54004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a54005280 0x7f9a5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c0b6230 con 0x7f9a54003a60 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5400c210 con 0x7f9a54004810 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5400c410 con 0x7f9a54005280 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a54003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004810 0x7f9a54004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6ad9d640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a54005280 0x7f9a5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 -- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a54005280 msgr2=0x7f9a5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a54005280 0x7f9a5400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 -- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004810 msgr2=0x7f9a54004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:46.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004810 0x7f9a54004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6ad9d640 1 -- 172.21.15.83:0/848925088 reap_dead start 2024-05-17T21:59:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 -- 172.21.15.83:0/848925088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a54003530 con 0x7f9a54003a60 2024-05-17T21:59:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004810 0x7f9a54004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a54003ef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c001790 tx=0x7f9a5c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2146a5c3453f58cc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0013e0 con 0x7f9a54003a60 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5c0a4e50 con 0x7f9a54003a60 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.640+0000 7f9a6959a640 1 -- 172.21.15.83:0/848925088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c003600 con 0x7f9a54003a60 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 msgr2=0x7f9a54003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a54003ef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c001790 tx=0x7f9a5c01c3b0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- 172.21.15.83:0/848925088 shutdown_connections 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/848925088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a54003ef0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- 172.21.15.83:0/848925088 >> 172.21.15.83:0/848925088 conn(0x7f9a6c06cd90 msgr2=0x7f9a6c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- 172.21.15.83:0/848925088 shutdown_connections 2024-05-17T21:59:46.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- 172.21.15.83:0/848925088 wait complete. 2024-05-17T21:59:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 Processor -- start 2024-05-17T21:59:46.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- start start 2024-05-17T21:59:46.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 0x7f9a6c144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6c158560 0x7f9a6c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c0b44c0 con 0x7f9a54003a60 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6c15e950 con 0x7f9a6c13d290 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a70bc3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6c0b4640 con 0x7f9a6c158560 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39298/0 (socket says 172.21.15.83:39298) 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a6a59c640 1 -- 172.21.15.83:0/223456426 learned_addr learned my addr 172.21.15.83:0/223456426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 0x7f9a6c144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.644+0000 7f9a6ad9d640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6c158560 0x7f9a6c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 -- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6c158560 msgr2=0x7f9a6c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6c158560 0x7f9a6c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 -- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 msgr2=0x7f9a6c144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 0x7f9a6c144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 -- 172.21.15.83:0/223456426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6c143380 con 0x7f9a54003a60 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 0x7f9a6c144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a6a59c640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c01aab0 tx=0x7f9a54005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c003dc0 con 0x7f9a54003a60 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a5c0013c0 con 0x7f9a54003a60 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c007260 con 0x7f9a54003a60 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6c143660 con 0x7f9a54003a60 2024-05-17T21:59:46.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6c143c20 con 0x7f9a54003a60 2024-05-17T21:59:46.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6c0b49d0 con 0x7f9a54003a60 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9a5c007400 con 0x7f9a54003a60 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 0x7f9a4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 0x7f9a4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f9a5c129db0 con 0x7f9a54003a60 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.648+0000 7f9a69d9b640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 0x7f9a4c071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c1414d0 tx=0x7f9a60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:46.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.652+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9a5c0b8030 con 0x7f9a54003a60 2024-05-17T21:59:46.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:46.816+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9a6c0b3dd0 con 0x7f9a54003a60 2024-05-17T21:59:47.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.456+0000 7f9a5affd640 1 -- 172.21.15.83:0/223456426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7f9a5c016110 con 0x7f9a54003a60 2024-05-17T21:59:47.451 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T21:59:47.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 msgr2=0x7f9a4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 0x7f9a4c071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c1414d0 tx=0x7f9a60009040 comp rx=0 tx=0).stop 2024-05-17T21:59:47.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 msgr2=0x7f9a6c159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c01aab0 tx=0x7f9a54005840 comp rx=0 tx=0).stop 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 shutdown_connections 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6c158560 0x7f9a6c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f400 0x7f9a4c071910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a6c13d290 0x7f9a6c144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 --2- 172.21.15.83:0/223456426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a54003a60 0x7f9a6c159240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 >> 172.21.15.83:0/223456426 conn(0x7f9a6c06cd90 msgr2=0x7f9a6c15ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 shutdown_connections 2024-05-17T21:59:47.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.460+0000 7f9a70bc3640 1 -- 172.21.15.83:0/223456426 wait complete. 2024-05-17T21:59:47.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T21:59:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.644+0000 7f8257289640 1 Processor -- start 2024-05-17T21:59:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.644+0000 7f8257289640 1 -- start start 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8257289640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f825015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8257289640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825015ec60 con 0x7f8250073560 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8257289640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82500b5be0 con 0x7f82500b66c0 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8257289640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82500b6230 con 0x7f82500b0bc0 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8250073560 0x7f825015df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50336/0 (socket says 172.21.15.83:50336) 2024-05-17T21:59:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 -- 172.21.15.83:0/2834912106 learned_addr learned my addr 172.21.15.83:0/2834912106 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 126901349 0 0) 0x7f825015ec60 con 0x7f8250073560 2024-05-17T21:59:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f823c003530 con 0x7f8250073560 2024-05-17T21:59:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1990081638 0 0) 0x7f823c003530 con 0x7f8250073560 2024-05-17T21:59:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 msgr2=0x7f825015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f825015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 >> v1:172.21.15.155:6789/0 conn(0x7f82500b66c0 legacy=0x7f8250072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82500b5ec0 con 0x7f8250073560 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f8240001720 con 0x7f8250073560 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 >> v1:172.21.15.83:6789/0 conn(0x7f8250073560 legacy=0x7f825015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f823c0039f0 0x7f823c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f823c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f823c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82500b6230 con 0x7f823c004810 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0055f0 con 0x7f823c0039f0 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f823c00bf10 con 0x7f82500b0bc0 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f823c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8255025640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f823c0039f0 0x7f823c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8255826640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f823c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 -- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 msgr2=0x7f823c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f823c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 -- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f823c0039f0 msgr2=0x7f823c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f823c0039f0 0x7f823c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8254824640 1 -- 172.21.15.83:0/2834912106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c003530 con 0x7f823c004810 2024-05-17T21:59:47.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.648+0000 7f8255826640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82500b0bc0 0x7f823c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8254824640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f823c004c20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f82500b3a30 tx=0x7f824001f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766ca1f25247a951 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240007d70 con 0x7f823c004810 2024-05-17T21:59:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240005b00 con 0x7f823c004810 2024-05-17T21:59:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8247fff640 1 -- 172.21.15.83:0/2834912106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240007070 con 0x7f823c004810 2024-05-17T21:59:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 -- 172.21.15.83:0/2834912106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 msgr2=0x7f823c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 --2- 172.21.15.83:0/2834912106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f823c004c20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f82500b3a30 tx=0x7f824001f2f0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8255826640 1 -- 172.21.15.83:0/2834912106 reap_dead start 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 -- 172.21.15.83:0/2834912106 shutdown_connections 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 -- 172.21.15.83:0/2834912106 >> 172.21.15.83:0/2834912106 conn(0x7f825006cd90 msgr2=0x7f82500b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 -- 172.21.15.83:0/2834912106 shutdown_connections 2024-05-17T21:59:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.652+0000 7f8257289640 1 -- 172.21.15.83:0/2834912106 wait complete. 2024-05-17T21:59:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 Processor -- start 2024-05-17T21:59:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- start start 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 0x7f8250141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 0x7f82501f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f825015e9d0 con 0x7f823c004810 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f825015e850 con 0x7f82501f3a80 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f82500b44c0 con 0x7f825013bda0 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48894/0 (socket says 172.21.15.83:48894) 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 -- 172.21.15.83:0/4256128386 learned_addr learned my addr 172.21.15.83:0/4256128386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8254824640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 0x7f8250141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255826640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 0x7f82501f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 -- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 msgr2=0x7f8250141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 0x7f8250141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 -- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 msgr2=0x7f82501f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 0x7f82501f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 -- 172.21.15.83:0/4256128386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825013cb40 con 0x7f823c004810 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8254824640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 0x7f8250141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255826640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 0x7f82501f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8255025640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e4e0 tx=0x7f824c00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c0052e0 con 0x7f823c004810 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c018070 con 0x7f823c004810 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c0054a0 con 0x7f823c004810 2024-05-17T21:59:47.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825013cfb0 con 0x7f823c004810 2024-05-17T21:59:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825013d570 con 0x7f823c004810 2024-05-17T21:59:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.656+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825015df30 con 0x7f823c004810 2024-05-17T21:59:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.660+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f824c003840 con 0x7f823c004810 2024-05-17T21:59:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.660+0000 7f8245ffb640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 0x7f8234071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.660+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5415+0+0 (secure 0 0 0) 0x7f824c08eec0 con 0x7f823c004810 2024-05-17T21:59:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.660+0000 7f8254824640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 0x7f8234071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:47.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.660+0000 7f8254824640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 0x7f8234071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82400ae000 tx=0x7f82500b5db0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.664+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f824c05bed0 con 0x7f823c004810 2024-05-17T21:59:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:47.824+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82500b3dd0 con 0x7f823c004810 2024-05-17T21:59:48.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.460+0000 7f8245ffb640 1 -- 172.21.15.83:0/4256128386 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7f824c05b7d0 con 0x7f823c004810 2024-05-17T21:59:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 msgr2=0x7f8234071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 0x7f8234071940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82400ae000 tx=0x7f82500b5db0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 msgr2=0x7f825013e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e4e0 tx=0x7f824c00e9a0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 shutdown_connections 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f823406f430 0x7f8234071940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82501f3a80 0x7f82501f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f825013bda0 0x7f8250141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 --2- 172.21.15.83:0/4256128386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f823c004810 0x7f825013e050 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 >> 172.21.15.83:0/4256128386 conn(0x7f825006cd90 msgr2=0x7f82501614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 shutdown_connections 2024-05-17T21:59:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.464+0000 7f8257289640 1 -- 172.21.15.83:0/4256128386 wait complete. 2024-05-17T21:59:48.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T21:59:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.652+0000 7fda3ee02640 1 Processor -- start 2024-05-17T21:59:48.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.652+0000 7fda3ee02640 1 -- start start 2024-05-17T21:59:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3ee02640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 0x7fda3815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3ee02640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda3815d530 0x7fda381663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3ee02640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda380b5640 con 0x7fda380cf830 2024-05-17T21:59:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3ee02640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fda38070870 con 0x7fda3815d530 2024-05-17T21:59:48.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3ee02640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda380b58e0 con 0x7fda380cdf40 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda3815d530 0x7fda381663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fda3815d530 0x7fda381663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48898/0 (socket says 172.21.15.83:48898) 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 -- 172.21.15.83:0/887176356 learned_addr learned my addr 172.21.15.83:0/887176356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda37fff640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 0x7fda3815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 -- 172.21.15.83:0/887176356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 msgr2=0x7fda3815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 0x7fda3815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 -- 172.21.15.83:0/887176356 >> v1:172.21.15.155:6789/0 conn(0x7fda380cf830 legacy=0x7fda380b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda3d39f640 1 -- 172.21.15.83:0/887176356 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3806f060 con 0x7fda3815d530 2024-05-17T21:59:48.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.656+0000 7fda37fff640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 0x7fda3815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3d39f640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda381663d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fda38072e10 tx=0x7fda3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ccf76c97dfdd3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda377fe640 1 -- 172.21.15.83:0/887176356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300a5640 con 0x7fda3815d530 2024-05-17T21:59:48.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda377fe640 1 -- 172.21.15.83:0/887176356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda300a57e0 con 0x7fda3815d530 2024-05-17T21:59:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda377fe640 1 -- 172.21.15.83:0/887176356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300a5bb0 con 0x7fda3815d530 2024-05-17T21:59:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- 172.21.15.83:0/887176356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 msgr2=0x7fda381663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda381663d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fda38072e10 tx=0x7fda3001c9f0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- 172.21.15.83:0/887176356 shutdown_connections 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda381663d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 --2- 172.21.15.83:0/887176356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cdf40 0x7fda3815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- 172.21.15.83:0/887176356 >> 172.21.15.83:0/887176356 conn(0x7fda3806cd90 msgr2=0x7fda380b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- 172.21.15.83:0/887176356 shutdown_connections 2024-05-17T21:59:48.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- 172.21.15.83:0/887176356 wait complete. 2024-05-17T21:59:48.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 Processor -- start 2024-05-17T21:59:48.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.660+0000 7fda3ee02640 1 -- start start 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380cdf40 0x7fda38154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 0x7fda38157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda38073a20 con 0x7fda3815d530 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda380738a0 con 0x7fda380cdf40 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda38073ba0 con 0x7fda380cf830 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48908/0 (socket says 172.21.15.83:48908) 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 -- 172.21.15.83:0/1638835564 learned_addr learned my addr 172.21.15.83:0/1638835564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:48.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda37fff640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 0x7fda38157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3cb9e640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380cdf40 0x7fda38154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 -- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 msgr2=0x7fda38157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 0x7fda38157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 -- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380cdf40 msgr2=0x7fda38154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380cdf40 0x7fda38154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 -- 172.21.15.83:0/1638835564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda38155860 con 0x7fda3815d530 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda37fff640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 0x7fda38157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3d39f640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda3001d290 tx=0x7fda30019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30009510 con 0x7fda3815d530 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda300be070 con 0x7fda3815d530 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300b8ae0 con 0x7fda3815d530 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda38155580 con 0x7fda3815d530 2024-05-17T21:59:48.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381586c0 con 0x7fda3815d530 2024-05-17T21:59:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3815cff0 con 0x7fda3815d530 2024-05-17T21:59:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda300b8c80 con 0x7fda3815d530 2024-05-17T21:59:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.664+0000 7fda357fa640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 0x7fda18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.668+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fda3012ad60 con 0x7fda3815d530 2024-05-17T21:59:48.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.668+0000 7fda3cb9e640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 0x7fda18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:48.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.668+0000 7fda3cb9e640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 0x7fda18071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda2800a770 tx=0x7fda28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:48.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.668+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda300165e0 con 0x7fda3815d530 2024-05-17T21:59:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.836+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fda38155310 con 0x7fda3815d530 2024-05-17T21:59:48.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.840+0000 7fda357fa640 1 -- 172.21.15.83:0/1638835564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fda300f8070 con 0x7fda3815d530 2024-05-17T21:59:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 msgr2=0x7fda18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 0x7fda18071940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda2800a770 tx=0x7fda28008040 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 msgr2=0x7fda38140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fda3001d290 tx=0x7fda30019790 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 shutdown_connections 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda3815d530 0x7fda38140d80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda380cf830 0x7fda38157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda1806f430 0x7fda18071940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 --2- 172.21.15.83:0/1638835564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda380cdf40 0x7fda38154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 >> 172.21.15.83:0/1638835564 conn(0x7fda3806cd90 msgr2=0x7fda380b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 shutdown_connections 2024-05-17T21:59:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:48.844+0000 7fda3ee02640 1 -- 172.21.15.83:0/1638835564 wait complete. 2024-05-17T21:59:48.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-17T21:59:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.008+0000 7f46a6475640 1 Processor -- start 2024-05-17T21:59:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.008+0000 7f46a6475640 1 -- start start 2024-05-17T21:59:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a6475640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46a00d1040 0x7f46a0144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a6475640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46a0073540 0x7f46a006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a6475640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a015ea40 con 0x7f46a015e080 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a6475640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f46a015ed20 con 0x7f46a00d1040 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a6475640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46a015f000 con 0x7f46a0073540 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f46a015e080 0x7f46a0072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50344/0 (socket says 172.21.15.83:50344) 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469f7fe640 1 -- 172.21.15.83:0/1355434893 learned_addr learned my addr 172.21.15.83:0/1355434893 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 --2- 172.21.15.83:0/1355434893 >> v2:172.21.15.155:3300/0 conn(0x7f46a00d1040 0x7f46a0144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a4a12640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46a0073540 0x7f46a006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934446592 0 0) 0x7f46a015ea40 con 0x7f46a015e080 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4688003530 con 0x7f46a015e080 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 277672364 0 0) 0x7f4688003530 con 0x7f46a015e080 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46a0073540 msgr2=0x7f46a006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46a0073540 0x7f46a006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 >> v2:172.21.15.155:3300/0 conn(0x7f46a00d1040 msgr2=0x7f46a0144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 --2- 172.21.15.83:0/1355434893 >> v2:172.21.15.155:3300/0 conn(0x7f46a00d1040 0x7f46a0144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a006ef30 con 0x7f46a015e080 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f4694001700 con 0x7f46a015e080 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 >> v1:172.21.15.83:6789/0 conn(0x7f46a015e080 legacy=0x7f46a0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688003a10 0x7f4688003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688004810 0x7f4688004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4688005160 0x7f468800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a015ed20 con 0x7f4688003a10 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46a015f000 con 0x7f4688005160 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f468800bf10 con 0x7f4688004810 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688003a10 0x7f4688003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469f7fe640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688004810 0x7f4688004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a4a12640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4688005160 0x7f468800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 -- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688004810 msgr2=0x7f4688004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688004810 0x7f4688004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 -- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4688005160 msgr2=0x7f468800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4688005160 0x7f468800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f469ffff640 1 -- 172.21.15.83:0/1355434893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4688003530 con 0x7f4688003a10 2024-05-17T21:59:49.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.012+0000 7f46a4a12640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4688005160 0x7f468800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f469ffff640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688003a10 0x7f4688003e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4690009ae0 tx=0x7f469001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2b6f69f493ccf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900b2070 con 0x7f4688003a10 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46900a5590 con 0x7f4688003a10 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f469effd640 1 -- 172.21.15.83:0/1355434893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46900096b0 con 0x7f4688003a10 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 -- 172.21.15.83:0/1355434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688003a10 msgr2=0x7f4688003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 --2- 172.21.15.83:0/1355434893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688003a10 0x7f4688003e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4690009ae0 tx=0x7f469001cab0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a4a12640 1 -- 172.21.15.83:0/1355434893 reap_dead start 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 -- 172.21.15.83:0/1355434893 shutdown_connections 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 -- 172.21.15.83:0/1355434893 >> 172.21.15.83:0/1355434893 conn(0x7f46a006cd90 msgr2=0x7f46a00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 -- 172.21.15.83:0/1355434893 shutdown_connections 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.016+0000 7f46a6475640 1 -- 172.21.15.83:0/1355434893 wait complete. 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 Processor -- start 2024-05-17T21:59:49.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- start start 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 0x7f46a0163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a015af10 0x7f46a015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00702d0 con 0x7f46a015d700 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0070150 con 0x7f46a015af10 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f46a0070450 con 0x7f4688003a10 2024-05-17T21:59:49.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48918/0 (socket says 172.21.15.83:48918) 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 -- 172.21.15.83:0/1134826459 learned_addr learned my addr 172.21.15.83:0/1134826459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469ffff640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 0x7f46a0163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a4a12640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a015af10 0x7f46a015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 -- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 msgr2=0x7f46a0163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 0x7f46a0163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 -- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a015af10 msgr2=0x7f46a015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a015af10 0x7f46a015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 -- 172.21.15.83:0/1134826459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a0162e80 con 0x7f46a015d700 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469ffff640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 0x7f46a0163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469f7fe640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f469400e7e0 tx=0x7f469400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400c6c0 con 0x7f46a015d700 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4694018070 con 0x7f46a015d700 2024-05-17T21:59:49.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a0163160 con 0x7f46a015d700 2024-05-17T21:59:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400caa0 con 0x7f46a015d700 2024-05-17T21:59:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.020+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a0163720 con 0x7f46a015d700 2024-05-17T21:59:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a0070810 con 0x7f46a015d700 2024-05-17T21:59:49.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f469400cc40 con 0x7f46a015d700 2024-05-17T21:59:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f469cff9640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 0x7f468c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f469ffff640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 0x7f468c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f469408ec80 con 0x7f46a015d700 2024-05-17T21:59:49.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.024+0000 7f469ffff640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 0x7f468c071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f46900046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.028+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f469405bf90 con 0x7f46a015d700 2024-05-17T21:59:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.252+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f46a0162c10 con 0x7f46a015d700 2024-05-17T21:59:49.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f469cff9640 1 -- 172.21.15.83:0/1134826459 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v131) ==== 149+0+0 (secure 0 0 0) 0x7f469405b890 con 0x7f46a015d700 2024-05-17T21:59:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 msgr2=0x7f468c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 0x7f468c071910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f46900af000 tx=0x7f46900046d0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 msgr2=0x7f46a00c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f469400e7e0 tx=0x7f469400eca0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 shutdown_connections 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468c06f400 0x7f468c071910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a015af10 0x7f46a015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a015d700 0x7f46a00c8640 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 --2- 172.21.15.83:0/1134826459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4688003a10 0x7f46a0163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 >> 172.21.15.83:0/1134826459 conn(0x7f46a006cd90 msgr2=0x7f46a01446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 shutdown_connections 2024-05-17T21:59:49.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.464+0000 7f46a6475640 1 -- 172.21.15.83:0/1134826459 wait complete. 2024-05-17T21:59:49.480 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-17T21:59:49.480 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-17T21:59:49.480 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T21:59:49.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 Processor -- start 2024-05-17T21:59:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 -- start start 2024-05-17T21:59:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280d0d10 0x7f80280d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80280b2030 0x7f80280b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f80280b2980 0x7f8028160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f80280b8280 con 0x7f80280b2030 2024-05-17T21:59:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f80280b6590 con 0x7f80280b2980 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802e671640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80280b5f80 con 0x7f80280d0d10 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80280b2030 0x7f80280b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f80280b2030 0x7f80280b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48926/0 (socket says 172.21.15.83:48926) 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 -- 172.21.15.83:0/3713549597 learned_addr learned my addr 172.21.15.83:0/3713549597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f802cc0e640 1 --2- 172.21.15.83:0/3713549597 >> v2:172.21.15.155:3300/0 conn(0x7f80280b2980 0x7f8028160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f8027fff640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280d0d10 0x7f80280d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 -- 172.21.15.83:0/3713549597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280d0d10 msgr2=0x7f80280d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280d0d10 0x7f80280d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 -- 172.21.15.83:0/3713549597 >> v2:172.21.15.155:3300/0 conn(0x7f80280b2980 msgr2=0x7f8028160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 --2- 172.21.15.83:0/3713549597 >> v2:172.21.15.155:3300/0 conn(0x7f80280b2980 0x7f8028160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.636+0000 7f801ffff640 1 -- 172.21.15.83:0/3713549597 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80280b3c70 con 0x7f80280b2030 2024-05-17T21:59:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f801ffff640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280b2030 0x7f80280b2440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f80280c96e0 tx=0x7f801001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68b76d362220032 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f80277fe640 1 -- 172.21.15.83:0/3713549597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100b0070 con 0x7f80280b2030 2024-05-17T21:59:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f80277fe640 1 -- 172.21.15.83:0/3713549597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80100ab600 con 0x7f80280b2030 2024-05-17T21:59:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f80277fe640 1 -- 172.21.15.83:0/3713549597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100ab9d0 con 0x7f80280b2030 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 -- 172.21.15.83:0/3713549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280b2030 msgr2=0x7f80280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280b2030 0x7f80280b2440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f80280c96e0 tx=0x7f801001c230 comp rx=0 tx=0).stop 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 -- 172.21.15.83:0/3713549597 shutdown_connections 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 --2- 172.21.15.83:0/3713549597 >> v2:172.21.15.155:3300/0 conn(0x7f80280b2980 0x7f8028160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280b2030 0x7f80280b2440 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 --2- 172.21.15.83:0/3713549597 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280d0d10 0x7f80280d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 -- 172.21.15.83:0/3713549597 >> 172.21.15.83:0/3713549597 conn(0x7f80280b4270 msgr2=0x7f80280b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 -- 172.21.15.83:0/3713549597 shutdown_connections 2024-05-17T21:59:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.640+0000 7f802e671640 1 -- 172.21.15.83:0/3713549597 wait complete. 2024-05-17T21:59:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 Processor -- start 2024-05-17T21:59:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- start start 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 0x7f802814ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80280b2980 0x7f80281594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80280b3a50 con 0x7f80280d0d10 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80280b38d0 con 0x7f80280b2980 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80280b3410 con 0x7f80280b2030 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48932/0 (socket says 172.21.15.83:48932) 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 -- 172.21.15.83:0/894990126 learned_addr learned my addr 172.21.15.83:0/894990126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f801ffff640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80280b2980 0x7f80281594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f8027fff640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 0x7f802814ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 -- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 msgr2=0x7f802814ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 0x7f802814ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 -- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80280b2980 msgr2=0x7f80281594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80280b2980 0x7f80281594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 -- 172.21.15.83:0/894990126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80280cdb80 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f8027fff640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 0x7f802814ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802cc0e640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8020007d10 tx=0x7f802000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020014420 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8020019070 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020014800 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80280cddf0 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.644+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281f62d0 con 0x7f80280d0d10 2024-05-17T21:59:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80280af4e0 con 0x7f80280d0d10 2024-05-17T21:59:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f80200149a0 con 0x7f80280d0d10 2024-05-17T21:59:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f80257fa640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 0x7f8008071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f8027fff640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 0x7f8008071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f80200900b0 con 0x7f80280d0d10 2024-05-17T21:59:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.648+0000 7f8027fff640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 0x7f8008071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80180097a0 tx=0x7f8018008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.652+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f802005d4a0 con 0x7f80280d0d10 2024-05-17T21:59:49.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.812+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80281f6500 con 0x7f80280d0d10 2024-05-17T21:59:49.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f80257fa640 1 -- 172.21.15.83:0/894990126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+12996 (secure 0 0 0) 0x7f802005cda0 con 0x7f80280d0d10 2024-05-17T21:59:49.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:49.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:48.461021+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/2531920228":"2024-05-18T21:59:45.447384+0000","172.21.15.83:6835/2531920228":"2024-05-18T21:59:45.447384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 msgr2=0x7f8008071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 0x7f8008071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80180097a0 tx=0x7f8018008040 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 msgr2=0x7f8028150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8020007d10 tx=0x7f802000e4b0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 shutdown_connections 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80280d0d10 0x7f8028150a80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f800806f400 0x7f8008071910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80280b2980 0x7f80281594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 --2- 172.21.15.83:0/894990126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80280b2030 0x7f802814ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.816+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 >> 172.21.15.83:0/894990126 conn(0x7f80280b4270 msgr2=0x7f8028076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.820+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 shutdown_connections 2024-05-17T21:59:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.820+0000 7f802e671640 1 -- 172.21.15.83:0/894990126 wait complete. 2024-05-17T21:59:49.831 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T21:59:49.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T21:59:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 Processor -- start 2024-05-17T21:59:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 -- start start 2024-05-17T21:59:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c0b4810 0x7fad2c0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 0x7fad2c15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fad2c0d4170 con 0x7fad2c0b4810 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fad2c0d4450 con 0x7fad2c0d0290 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad33744640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0d4730 con 0x7fad2c0d2000 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c0b4810 0x7fad2c0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad2c0b4810 0x7fad2c0d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48938/0 (socket says 172.21.15.83:48938) 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 -- 172.21.15.83:0/2436944907 learned_addr learned my addr 172.21.15.83:0/2436944907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad30cdf640 1 --2- 172.21.15.83:0/2436944907 >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 0x7fad2c15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad31ce1640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 -- 172.21.15.83:0/2436944907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 msgr2=0x7fad2c0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 -- 172.21.15.83:0/2436944907 >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 msgr2=0x7fad2c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 --2- 172.21.15.83:0/2436944907 >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 0x7fad2c15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad314e0640 1 -- 172.21.15.83:0/2436944907 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c0d4a10 con 0x7fad2c0b4810 2024-05-17T21:59:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.976+0000 7fad30cdf640 1 --2- 172.21.15.83:0/2436944907 >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 0x7fad2c15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.980+0000 7fad314e0640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c0d5210 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fad2c08e020 tx=0x7fad2801cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8a862dccb5a96f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.980+0000 7fad23fff640 1 -- 172.21.15.83:0/2436944907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280aa660 con 0x7fad2c0b4810 2024-05-17T21:59:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.980+0000 7fad23fff640 1 -- 172.21.15.83:0/2436944907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad280af070 con 0x7fad2c0b4810 2024-05-17T21:59:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.980+0000 7fad23fff640 1 -- 172.21.15.83:0/2436944907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280aaa20 con 0x7fad2c0b4810 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- 172.21.15.83:0/2436944907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 msgr2=0x7fad2c0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c0d5210 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fad2c08e020 tx=0x7fad2801cf80 comp rx=0 tx=0).stop 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- 172.21.15.83:0/2436944907 shutdown_connections 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- 172.21.15.83:0/2436944907 >> v2:172.21.15.155:3300/0 conn(0x7fad2c0d0290 0x7fad2c15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- 172.21.15.83:0/2436944907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c0d5210 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- 172.21.15.83:0/2436944907 >> 172.21.15.83:0/2436944907 conn(0x7fad2c06e010 msgr2=0x7fad2c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- 172.21.15.83:0/2436944907 shutdown_connections 2024-05-17T21:59:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- 172.21.15.83:0/2436944907 wait complete. 2024-05-17T21:59:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 Processor -- start 2024-05-17T21:59:49.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- start start 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad2c0d0290 0x7fad2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0b7aa0 con 0x7fad2c0b4810 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c160050 con 0x7fad2c0d0290 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad33744640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0b7c20 con 0x7fad2c0d2000 2024-05-17T21:59:49.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48946/0 (socket says 172.21.15.83:48946) 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 -- 172.21.15.83:0/2772831867 learned_addr learned my addr 172.21.15.83:0/2772831867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 -- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 msgr2=0x7fad2c15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 -- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad2c0d0290 msgr2=0x7fad2c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad2c0d0290 0x7fad2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 -- 172.21.15.83:0/2772831867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c0b6a10 con 0x7fad2c0b4810 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.984+0000 7fad314e0640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad2801cdc0 tx=0x7fad2801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280bd5f0 con 0x7fad2c0b4810 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad280bd7b0 con 0x7fad2c0b4810 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c1598d0 con 0x7fad2c0b4810 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad280bdba0 con 0x7fad2c0b4810 2024-05-17T21:59:49.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c159e90 con 0x7fad2c0b4810 2024-05-17T21:59:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2c0b7fb0 con 0x7fad2c0b4810 2024-05-17T21:59:49.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad280b33f0 con 0x7fad2c0b4810 2024-05-17T21:59:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 0x7fad10071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fad28129710 con 0x7fad2c0b4810 2024-05-17T21:59:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad30cdf640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 0x7fad10071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.988+0000 7fad30cdf640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 0x7fad10071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad1c006e90 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:49.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:49.992+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fad280c3060 con 0x7fad2c0b4810 2024-05-17T21:59:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.152+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad2c069260 con 0x7fad2c0b4810 2024-05-17T21:59:50.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad21ffb640 1 -- 172.21.15.83:0/2772831867 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7fad2c069260 con 0x7fad2c0b4810 2024-05-17T21:59:50.148 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T21:59:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 msgr2=0x7fad10071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 0x7fad10071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad1c006e90 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-05-17T21:59:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 msgr2=0x7fad2c15a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad2801cdc0 tx=0x7fad2801ad60 comp rx=0 tx=0).stop 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 shutdown_connections 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad2c0d2000 0x7fad2c15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad1006f430 0x7fad10071940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad2c0d0290 0x7fad2c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 --2- 172.21.15.83:0/2772831867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad2c0b4810 0x7fad2c15a690 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 >> 172.21.15.83:0/2772831867 conn(0x7fad2c06e010 msgr2=0x7fad2c0b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 shutdown_connections 2024-05-17T21:59:50.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.156+0000 7fad33744640 1 -- 172.21.15.83:0/2772831867 wait complete. 2024-05-17T21:59:50.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-17T21:59:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 Processor -- start 2024-05-17T21:59:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 -- start start 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c041461c0 0x7f3c0415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c0415e6b0 0x7f3c0406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c041465d0 con 0x7f3c04070510 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0406ef30 con 0x7f3c041461c0 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c0b42b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0406f210 con 0x7f3c0415e6b0 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c091c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c041461c0 0x7f3c0415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c091c7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3c041461c0 0x7f3c0415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48948/0 (socket says 172.21.15.83:48948) 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c099c8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3c04070510 0x7f3c041605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50360/0 (socket says 172.21.15.83:50360) 2024-05-17T21:59:50.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.312+0000 7f3c091c7640 1 -- 172.21.15.83:0/146285391 learned_addr learned my addr 172.21.15.83:0/146285391 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:50.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c089c6640 1 --2- 172.21.15.83:0/146285391 >> v2:172.21.15.155:3300/0 conn(0x7f3c0415e6b0 0x7f3c0406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c091c7640 1 -- 172.21.15.83:0/146285391 >> v2:172.21.15.155:3300/0 conn(0x7f3c0415e6b0 msgr2=0x7f3c0406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:50.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c091c7640 1 --2- 172.21.15.83:0/146285391 >> v2:172.21.15.155:3300/0 conn(0x7f3c0415e6b0 0x7f3c0406ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c091c7640 1 -- 172.21.15.83:0/146285391 >> v1:172.21.15.83:6789/0 conn(0x7f3c04070510 legacy=0x7f3c041605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c091c7640 1 -- 172.21.15.83:0/146285391 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c04070920 con 0x7f3c041461c0 2024-05-17T21:59:50.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c091c7640 1 --2- 172.21.15.83:0/146285391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c041461c0 0x7f3c0415e170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c04071230 tx=0x7f3c0001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62226d42fbde8b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:50.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3bfbfff640 1 -- 172.21.15.83:0/146285391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000b0070 con 0x7f3c041461c0 2024-05-17T21:59:50.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3bfbfff640 1 -- 172.21.15.83:0/146285391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c00004230 con 0x7f3c041461c0 2024-05-17T21:59:50.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3bfbfff640 1 -- 172.21.15.83:0/146285391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00003620 con 0x7f3c041461c0 2024-05-17T21:59:50.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 -- 172.21.15.83:0/146285391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c041461c0 msgr2=0x7f3c0415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/146285391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c041461c0 0x7f3c0415e170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3c04071230 tx=0x7f3c0001c3b0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 -- 172.21.15.83:0/146285391 shutdown_connections 2024-05-17T21:59:50.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/146285391 >> v2:172.21.15.155:3300/0 conn(0x7f3c0415e6b0 0x7f3c0406ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/146285391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c041461c0 0x7f3c0415e170 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 -- 172.21.15.83:0/146285391 >> 172.21.15.83:0/146285391 conn(0x7f3c0406cd90 msgr2=0x7f3c040b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 -- 172.21.15.83:0/146285391 shutdown_connections 2024-05-17T21:59:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.316+0000 7f3c0b42b640 1 -- 172.21.15.83:0/146285391 wait complete. 2024-05-17T21:59:50.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 Processor -- start 2024-05-17T21:59:50.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 -- start start 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c04070510 0x7f3c040b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c041461c0 0x7f3c04136e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406fab0 con 0x7f3c0415e6b0 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406f7d0 con 0x7f3c041461c0 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3c04160cb0 con 0x7f3c04070510 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48956/0 (socket says 172.21.15.83:48956) 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 -- 172.21.15.83:0/2621980485 learned_addr learned my addr 172.21.15.83:0/2621980485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c089c6640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c041461c0 0x7f3c04136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c091c7640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c04070510 0x7f3c040b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 -- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c04070510 msgr2=0x7f3c040b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c04070510 0x7f3c040b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 -- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c041461c0 msgr2=0x7f3c04136e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c041461c0 0x7f3c04136e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 -- 172.21.15.83:0/2621980485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c04141350 con 0x7f3c0415e6b0 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c099c8640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0058b0 tx=0x7f3bfc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc01b070 con 0x7f3c0415e6b0 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc00db40 con 0x7f3c0415e6b0 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00a0d0 con 0x7f3c0415e6b0 2024-05-17T21:59:50.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.320+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c040abe40 con 0x7f3c0415e6b0 2024-05-17T21:59:50.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c040ac400 con 0x7f3c0415e6b0 2024-05-17T21:59:50.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0415e170 con 0x7f3c0415e6b0 2024-05-17T21:59:50.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3bfc00a270 con 0x7f3c0415e6b0 2024-05-17T21:59:50.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3bf9ffb640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 0x7f3be8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:50.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f3bfc091ae0 con 0x7f3c0415e6b0 2024-05-17T21:59:50.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3c091c7640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 0x7f3be8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:50.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.324+0000 7f3c091c7640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 0x7f3be8071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c000ad000 tx=0x7f3c000a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.328+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3c04141350 con 0x7f3c0415e6b0 2024-05-17T21:59:50.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:50.544+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f3c040b41a0 con 0x7f3c0415e6b0 2024-05-17T21:59:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.476+0000 7f3bf9ffb640 1 -- 172.21.15.83:0/2621980485 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v133) ==== 172+0+0 (secure 0 0 0) 0x7f3bfc05e680 con 0x7f3c0415e6b0 2024-05-17T21:59:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 msgr2=0x7f3be8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 0x7f3be8071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c000ad000 tx=0x7f3c000a6000 comp rx=0 tx=0).stop 2024-05-17T21:59:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 msgr2=0x7f3c040b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0058b0 tx=0x7f3bfc00d430 comp rx=0 tx=0).stop 2024-05-17T21:59:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 shutdown_connections 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c0415e6b0 0x7f3c040b2f50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3be806f400 0x7f3be8071910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c041461c0 0x7f3c04136e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 --2- 172.21.15.83:0/2621980485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c04070510 0x7f3c040b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 >> 172.21.15.83:0/2621980485 conn(0x7f3c0406cd90 msgr2=0x7f3c0415eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 shutdown_connections 2024-05-17T21:59:51.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.480+0000 7f3c0b42b640 1 -- 172.21.15.83:0/2621980485 wait complete. 2024-05-17T21:59:51.495 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T21:59:51.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 Processor -- start 2024-05-17T21:59:51.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 -- start start 2024-05-17T21:59:51.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2da0073590 0x7f2da00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2da01447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da00d10d0 con 0x7f2da00b3d00 2024-05-17T21:59:51.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2da00b4110 con 0x7f2da0073590 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da7065640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2da0070770 con 0x7f2da0143f20 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da4e01640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2da00b3d00 0x7f2da0072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50372/0 (socket says 172.21.15.83:50372) 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da4e01640 1 -- 172.21.15.83:0/1010276821 learned_addr learned my addr 172.21.15.83:0/1010276821 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/1010276821 >> v2:172.21.15.155:3300/0 conn(0x7f2da0073590 0x7f2da00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2da5602640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2da01447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151614390 0 0) 0x7f2da00d10d0 con 0x7f2da00b3d00 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d88003530 con 0x7f2da00b3d00 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 265160286 0 0) 0x7f2d88003530 con 0x7f2da00b3d00 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 msgr2=0x7f2da01447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2da01447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 >> v2:172.21.15.155:3300/0 conn(0x7f2da0073590 msgr2=0x7f2da00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 --2- 172.21.15.83:0/1010276821 >> v2:172.21.15.155:3300/0 conn(0x7f2da0073590 0x7f2da00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da0070120 con 0x7f2da00b3d00 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2d90001700 con 0x7f2da00b3d00 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 >> v1:172.21.15.83:6789/0 conn(0x7f2da00b3d00 legacy=0x7f2da0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d88003a10 0x7f2d88003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88004810 0x7f2d88004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2d88005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2da00b4110 con 0x7f2d88004810 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0070770 con 0x7f2d88003a10 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8800ba80 con 0x7f2da0143f20 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.660+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88004810 0x7f2d88004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da5602640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2d88005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da4e01640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d88003a10 0x7f2d88003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 -- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 msgr2=0x7f2d88005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2d88005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 -- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d88003a10 msgr2=0x7f2d88003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:51.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d88003a10 0x7f2d88003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 -- 172.21.15.83:0/1010276821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d88003530 con 0x7f2d88004810 2024-05-17T21:59:51.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da5602640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2da0143f20 0x7f2d88005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88004810 0x7f2d88004c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009ae0 tx=0x7f2d9401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbc63663f53f859 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940b2070 con 0x7f2d88004810 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d940a5590 con 0x7f2d88004810 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2d9f7fe640 1 -- 172.21.15.83:0/1010276821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940096b0 con 0x7f2d88004810 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 -- 172.21.15.83:0/1010276821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88004810 msgr2=0x7f2d88004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 --2- 172.21.15.83:0/1010276821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88004810 0x7f2d88004c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009ae0 tx=0x7f2d9401cab0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da5602640 1 -- 172.21.15.83:0/1010276821 reap_dead start 2024-05-17T21:59:51.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 -- 172.21.15.83:0/1010276821 shutdown_connections 2024-05-17T21:59:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 -- 172.21.15.83:0/1010276821 >> 172.21.15.83:0/1010276821 conn(0x7f2da006cd90 msgr2=0x7f2da00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 -- 172.21.15.83:0/1010276821 shutdown_connections 2024-05-17T21:59:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.664+0000 7f2da7065640 1 -- 172.21.15.83:0/1010276821 wait complete. 2024-05-17T21:59:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 Processor -- start 2024-05-17T21:59:51.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- start start 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d88004810 0x7f2da013e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da00ab160 0x7f2da00ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0147600 con 0x7f2da013ecc0 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2da006fed0 con 0x7f2da00ab160 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2da0144330 con 0x7f2d88004810 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48974/0 (socket says 172.21.15.83:48974) 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 -- 172.21.15.83:0/4037664642 learned_addr learned my addr 172.21.15.83:0/4037664642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da4e01640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d88004810 0x7f2da013e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da5602640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da00ab160 0x7f2da00ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 -- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d88004810 msgr2=0x7f2da013e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d88004810 0x7f2da013e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 -- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da00ab160 msgr2=0x7f2da00ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da00ab160 0x7f2da00ab570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 -- 172.21.15.83:0/4037664642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da00abab0 con 0x7f2da013ecc0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9ffff640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d940b0040 tx=0x7f2d88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940bd020 con 0x7f2da013ecc0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d94004020 con 0x7f2da013ecc0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da013f420 con 0x7f2da013ecc0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940b2040 con 0x7f2da013ecc0 2024-05-17T21:59:51.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.668+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da013f9e0 con 0x7f2da013ecc0 2024-05-17T21:59:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da013f180 con 0x7f2da013ecc0 2024-05-17T21:59:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2d940c0070 con 0x7f2da013ecc0 2024-05-17T21:59:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2d9d7fa640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 0x7f2d8c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2da4e01640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 0x7f2d8c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:51.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f2d94128c10 con 0x7f2da013ecc0 2024-05-17T21:59:51.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.672+0000 7f2da4e01640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 0x7f2d8c071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d90004880 tx=0x7f2d900047f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:51.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.676+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2da013f180 con 0x7f2da013ecc0 2024-05-17T21:59:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.836+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2da0070c00 con 0x7f2da013ecc0 2024-05-17T21:59:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.840+0000 7f2d9d7fa640 1 -- 172.21.15.83:0/4037664642 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2d940167c0 con 0x7f2da013ecc0 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 msgr2=0x7f2d8c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 0x7f2d8c071940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d90004880 tx=0x7f2d900047f0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 msgr2=0x7f2da00aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2d940b0040 tx=0x7f2d88005840 comp rx=0 tx=0).stop 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 shutdown_connections 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2d8c06f430 0x7f2d8c071940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2da00ab160 0x7f2da00ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da013ecc0 0x7f2da00aac20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 --2- 172.21.15.83:0/4037664642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d88004810 0x7f2da013e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 >> 172.21.15.83:0/4037664642 conn(0x7f2da006cd90 msgr2=0x7f2da015d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 shutdown_connections 2024-05-17T21:59:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:51.844+0000 7f2da7065640 1 -- 172.21.15.83:0/4037664642 wait complete. 2024-05-17T21:59:51.857 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T21:59:51.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:59:51.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T21:59:51.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T21:59:51.960 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:59:51.960 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:51.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:59:51.967 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:59:51.967 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:51.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:59:51.974 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T21:59:51.974 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:52.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T21:59:52.017 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T21:59:52.017 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T21:59:52.023 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T21:59:52.023 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T21:59:52.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T21:59:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 Processor -- start 2024-05-17T21:59:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 -- start start 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93680d1040 0x7f9368144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9368073540 0x7f936806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f936815ea40 con 0x7f936815e080 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f936815ed20 con 0x7f9368073540 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f937006f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f936815f000 con 0x7f93680d1040 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9368073540 0x7f936806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9368073540 0x7f936806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48990/0 (socket says 172.21.15.83:48990) 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 -- 172.21.15.83:0/3699915883 learned_addr learned my addr 172.21.15.83:0/3699915883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936d60a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f936815e080 0x7f9368072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50382/0 (socket says 172.21.15.83:50382) 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936de0b640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93680d1040 0x7f9368144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 -- 172.21.15.83:0/3699915883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93680d1040 msgr2=0x7f9368144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93680d1040 0x7f9368144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 -- 172.21.15.83:0/3699915883 >> v1:172.21.15.83:6789/0 conn(0x7f936815e080 legacy=0x7f9368072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.224+0000 7f936e60c640 1 -- 172.21.15.83:0/3699915883 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936806ef30 con 0x7f9368073540 2024-05-17T21:59:52.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f936e60c640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368073540 0x7f936806f730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9360009e50 tx=0x7f936001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c6645f24b69e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f936ce09640 1 -- 172.21.15.83:0/3699915883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600a5810 con 0x7f9368073540 2024-05-17T21:59:52.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f936ce09640 1 -- 172.21.15.83:0/3699915883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93600a59b0 con 0x7f9368073540 2024-05-17T21:59:52.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f936ce09640 1 -- 172.21.15.83:0/3699915883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600a5d80 con 0x7f9368073540 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 -- 172.21.15.83:0/3699915883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368073540 msgr2=0x7f936806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368073540 0x7f936806f730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9360009e50 tx=0x7f936001ca30 comp rx=0 tx=0).stop 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 -- 172.21.15.83:0/3699915883 shutdown_connections 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368073540 0x7f936806f730 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 --2- 172.21.15.83:0/3699915883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93680d1040 0x7f9368144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 -- 172.21.15.83:0/3699915883 >> 172.21.15.83:0/3699915883 conn(0x7f936806cd90 msgr2=0x7f93680b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 -- 172.21.15.83:0/3699915883 shutdown_connections 2024-05-17T21:59:52.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.228+0000 7f937006f640 1 -- 172.21.15.83:0/3699915883 wait complete. 2024-05-17T21:59:52.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 Processor -- start 2024-05-17T21:59:52.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- start start 2024-05-17T21:59:52.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680d1040 0x7f936815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f936815e080 0x7f936815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9368070810 con 0x7f9368162200 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9368070690 con 0x7f93680d1040 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9368070990 con 0x7f936815e080 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49002/0 (socket says 172.21.15.83:49002) 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 -- 172.21.15.83:0/1292613140 learned_addr learned my addr 172.21.15.83:0/1292613140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936d60a640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f936815e080 0x7f936815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936de0b640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680d1040 0x7f936815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 -- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f936815e080 msgr2=0x7f936815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f936815e080 0x7f936815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 -- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680d1040 msgr2=0x7f936815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680d1040 0x7f936815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 -- 172.21.15.83:0/1292613140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9368160760 con 0x7f9368162200 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f936e60c640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93600af040 tx=0x7f93600197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600b8880 con 0x7f9368162200 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93600be070 con 0x7f9368162200 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9368141d10 con 0x7f9368162200 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600b8c60 con 0x7f9368162200 2024-05-17T21:59:52.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93681422d0 con 0x7f9368162200 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.232+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936806f730 con 0x7f9368162200 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.236+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93600c3460 con 0x7f9368162200 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.236+0000 7f935e7fc640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 0x7f934c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.236+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f936012a400 con 0x7f9368162200 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.236+0000 7f936de0b640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 0x7f934c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.236+0000 7f936de0b640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 0x7f934c071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f936400b930 tx=0x7f9364008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.240+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9360016620 con 0x7f9368162200 2024-05-17T21:59:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9368146410 con 0x7f9368162200 2024-05-17T21:59:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f935e7fc640 1 -- 172.21.15.83:0/1292613140 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3869 (secure 0 0 0) 0x7f93600f7600 con 0x7f9368162200 2024-05-17T21:59:52.425 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:52.425 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3931413506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3931413506},{"type":"v1","addr":"172.21.15.155:6835","nonce":3931413506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/887119595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":887119595},{"type":"v1","addr":"172.21.15.155:6834","nonce":887119595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5154,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1716064479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1716064479},{"type":"v1","addr":"172.21.15.83:6836","nonce":1716064479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/506030559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":506030559},{"type":"v1","addr":"172.21.15.83:6837","nonce":506030559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-05-17T21:59:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 msgr2=0x7f934c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 0x7f934c071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f936400b930 tx=0x7f9364008040 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 msgr2=0x7f9368146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f93600af040 tx=0x7f93600197d0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 shutdown_connections 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9368162200 0x7f9368146930 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f936815e080 0x7f936815fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f934c06f400 0x7f934c071910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 --2- 172.21.15.83:0/1292613140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680d1040 0x7f936815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 >> 172.21.15.83:0/1292613140 conn(0x7f936806cd90 msgr2=0x7f936813d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 shutdown_connections 2024-05-17T21:59:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.432+0000 7f937006f640 1 -- 172.21.15.83:0/1292613140 wait complete. 2024-05-17T21:59:52.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T21:59:52.446 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T21:59:52.446 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T21:59:52.446 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:59:52.446 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T21:59:52.447 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T21:59:52.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T21:59:52.447 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:59:52.447 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T21:59:52.447 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T21:59:52.447 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T21:59:52.447 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:59:52.448 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T21:59:52.448 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T21:59:52.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T21:59:52.448 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T21:59:52.460 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T21:59:52.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T21:59:52.461 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T21:59:52.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T21:59:52.464 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T21:59:52.465 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T21:59:52.472 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T21:59:52.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T21:59:52.504 INFO:tasks.ceph.mds.a:Started 2024-05-17T21:59:52.505 DEBUG:teuthology.parallel:result is None 2024-05-17T21:59:52.505 INFO:tasks.ceph.mds.c:Started 2024-05-17T21:59:52.506 DEBUG:teuthology.parallel:result is None 2024-05-17T21:59:52.514 INFO:tasks.ceph.mds.b:Started 2024-05-17T21:59:52.514 DEBUG:teuthology.parallel:result is None 2024-05-17T21:59:52.515 INFO:tasks.ceph.mds.d:Started 2024-05-17T21:59:52.515 DEBUG:teuthology.parallel:result is None 2024-05-17T21:59:52.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T21:59:52.590 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T21:59:52.596 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T21:59:52.611 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T21:59:52.615 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T21:59:52.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 Processor -- start 2024-05-17T21:59:52.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 -- start start 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc93c0b4810 0x7fc93c0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc93c0d2000 0x7fc93c0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93c0d4170 con 0x7fc93c0d0290 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc93c0d4450 con 0x7fc93c0b4810 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc943ae6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc93c0d4730 con 0x7fc93c0d2000 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc93c0b4810 0x7fc93c0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc93c0b4810 0x7fc93c0d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49052/0 (socket says 172.21.15.83:49052) 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 -- 172.21.15.83:0/2647241048 learned_addr learned my addr 172.21.15.83:0/2647241048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 -- 172.21.15.83:0/2647241048 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0d2000 msgr2=0x7fc93c0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc942083640 1 --2- 172.21.15.83:0/2647241048 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0d2000 0x7fc93c0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 --2- 172.21.15.83:0/2647241048 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0d2000 0x7fc93c0d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 -- 172.21.15.83:0/2647241048 >> v1:172.21.15.155:6789/0 conn(0x7fc93c0d0290 legacy=0x7fc93c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.664+0000 7fc941882640 1 -- 172.21.15.83:0/2647241048 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c0d4a10 con 0x7fc93c0b4810 2024-05-17T21:59:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc941882640 1 --2- 172.21.15.83:0/2647241048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0b4810 0x7fc93c0d5210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0d3120 tx=0x7fc93801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d52a4462430644 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc940880640 1 -- 172.21.15.83:0/2647241048 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380aa660 con 0x7fc93c0b4810 2024-05-17T21:59:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc940880640 1 -- 172.21.15.83:0/2647241048 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9380af070 con 0x7fc93c0b4810 2024-05-17T21:59:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc940880640 1 -- 172.21.15.83:0/2647241048 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380aaa20 con 0x7fc93c0b4810 2024-05-17T21:59:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 -- 172.21.15.83:0/2647241048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0b4810 msgr2=0x7fc93c0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 --2- 172.21.15.83:0/2647241048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0b4810 0x7fc93c0d5210 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0d3120 tx=0x7fc93801c520 comp rx=0 tx=0).stop 2024-05-17T21:59:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 -- 172.21.15.83:0/2647241048 shutdown_connections 2024-05-17T21:59:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 --2- 172.21.15.83:0/2647241048 >> v2:172.21.15.155:3300/0 conn(0x7fc93c0d2000 0x7fc93c0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 --2- 172.21.15.83:0/2647241048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0b4810 0x7fc93c0d5210 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 -- 172.21.15.83:0/2647241048 >> 172.21.15.83:0/2647241048 conn(0x7fc93c06e010 msgr2=0x7fc93c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 -- 172.21.15.83:0/2647241048 shutdown_connections 2024-05-17T21:59:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.668+0000 7fc943ae6640 1 -- 172.21.15.83:0/2647241048 wait complete. 2024-05-17T21:59:52.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 Processor -- start 2024-05-17T21:59:52.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- start start 2024-05-17T21:59:52.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0b4810 0x7fc93c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d0290 0x7fc93c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c0b9670 con 0x7fc93c0d2000 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c0b94f0 con 0x7fc93c0b4810 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc93c0b97f0 con 0x7fc93c0d0290 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49066/0 (socket says 172.21.15.83:49066) 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 -- 172.21.15.83:0/3581739855 learned_addr learned my addr 172.21.15.83:0/3581739855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc941081640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d0290 0x7fc93c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc941882640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0b4810 0x7fc93c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 -- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d0290 msgr2=0x7fc93c0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d0290 0x7fc93c0cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 -- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0b4810 msgr2=0x7fc93c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:52.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0b4810 0x7fc93c154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 -- 172.21.15.83:0/3581739855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c0b5ba0 con 0x7fc93c0d2000 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc942083640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc93400cc30 tx=0x7fc93400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93400ce40 con 0x7fc93c0d2000 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9340039a0 con 0x7fc93c0d2000 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c0b5e80 con 0x7fc93c0d2000 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934003d50 con 0x7fc93c0d2000 2024-05-17T21:59:52.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c1590c0 con 0x7fc93c0d2000 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.672+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c0b5910 con 0x7fc93c0d2000 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.676+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc934003ef0 con 0x7fc93c0d2000 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.676+0000 7fc9327fc640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.676+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fc93408f750 con 0x7fc93c0d2000 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.676+0000 7fc941882640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:52.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.676+0000 7fc941882640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 0x7fc920071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ae000 tx=0x7fc9380a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:52.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.680+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc934014030 con 0x7fc93c0d2000 2024-05-17T21:59:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:52.848+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc93c0d5210 con 0x7fc93c0d2000 2024-05-17T21:59:53.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.500+0000 7fc9327fc640 1 -- 172.21.15.83:0/3581739855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) ==== 97+0+0 (secure 0 0 0) 0x7fc93405cab0 con 0x7fc93c0d2000 2024-05-17T21:59:53.494 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 msgr2=0x7fc920071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 0x7fc920071910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ae000 tx=0x7fc9380a7000 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 msgr2=0x7fc93c0b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc93400cc30 tx=0x7fc93400a5c0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 shutdown_connections 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc93c0d2000 0x7fc93c0b6430 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc93c0d0290 0x7fc93c0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92006f400 0x7fc920071910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 --2- 172.21.15.83:0/3581739855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc93c0b4810 0x7fc93c154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 >> 172.21.15.83:0/3581739855 conn(0x7fc93c06e010 msgr2=0x7fc93c15eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:53.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 shutdown_connections 2024-05-17T21:59:53.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.504+0000 7fc943ae6640 1 -- 172.21.15.83:0/3581739855 wait complete. 2024-05-17T21:59:53.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T21:59:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 Processor -- start 2024-05-17T21:59:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 -- start start 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc15fef0 0x7ff7dc06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7dc0b5a50 0x7ff7dc0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7dc072e30 0x7ff7dc073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc073780 con 0x7ff7dc072e30 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc0b49b0 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e337d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc0b4c90 con 0x7ff7dc15fef0 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7dc072e30 0x7ff7dc073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7dc072e30 0x7ff7dc073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49076/0 (socket says 172.21.15.83:49076) 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 -- 172.21.15.83:0/1396508113 learned_addr learned my addr 172.21.15.83:0/1396508113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e1119640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc15fef0 0x7ff7dc06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e0918640 1 --2- 172.21.15.83:0/1396508113 >> v2:172.21.15.155:3300/0 conn(0x7ff7dc0b5a50 0x7ff7dc0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 -- 172.21.15.83:0/1396508113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc15fef0 msgr2=0x7ff7dc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc15fef0 0x7ff7dc06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 -- 172.21.15.83:0/1396508113 >> v2:172.21.15.155:3300/0 conn(0x7ff7dc0b5a50 msgr2=0x7ff7dc0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 --2- 172.21.15.83:0/1396508113 >> v2:172.21.15.155:3300/0 conn(0x7ff7dc0b5a50 0x7ff7dc0b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.676+0000 7ff7e191a640 1 -- 172.21.15.83:0/1396508113 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc0b4f70 con 0x7ff7dc072e30 2024-05-17T21:59:53.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e191a640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc072e30 0x7ff7dc073240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0712c0 tx=0x7ff7d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e875d03ffba2b315 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:53.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7cffff640 1 -- 172.21.15.83:0/1396508113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40a5760 con 0x7ff7dc072e30 2024-05-17T21:59:53.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7cffff640 1 -- 172.21.15.83:0/1396508113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d40a5900 con 0x7ff7dc072e30 2024-05-17T21:59:53.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7cffff640 1 -- 172.21.15.83:0/1396508113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40a5cd0 con 0x7ff7dc072e30 2024-05-17T21:59:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 -- 172.21.15.83:0/1396508113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc072e30 msgr2=0x7ff7dc073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc072e30 0x7ff7dc073240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0712c0 tx=0x7ff7d401c9f0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 -- 172.21.15.83:0/1396508113 shutdown_connections 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc072e30 0x7ff7dc073240 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 --2- 172.21.15.83:0/1396508113 >> v2:172.21.15.155:3300/0 conn(0x7ff7dc0b5a50 0x7ff7dc0b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 --2- 172.21.15.83:0/1396508113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc15fef0 0x7ff7dc06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 -- 172.21.15.83:0/1396508113 >> 172.21.15.83:0/1396508113 conn(0x7ff7dc06cd90 msgr2=0x7ff7dc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 -- 172.21.15.83:0/1396508113 shutdown_connections 2024-05-17T21:59:53.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.680+0000 7ff7e337d640 1 -- 172.21.15.83:0/1396508113 wait complete. 2024-05-17T21:59:53.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 Processor -- start 2024-05-17T21:59:53.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- start start 2024-05-17T21:59:53.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc072e30 0x7ff7dc14a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc15fef0 0x7ff7dc0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc06ff10 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc06fd90 con 0x7ff7dc15fef0 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc0af9b0 con 0x7ff7dc072e30 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49084/0 (socket says 172.21.15.83:49084) 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 -- 172.21.15.83:0/2309333854 learned_addr learned my addr 172.21.15.83:0/2309333854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e1119640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc072e30 0x7ff7dc14a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e191a640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc15fef0 0x7ff7dc0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 -- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc072e30 msgr2=0x7ff7dc14a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc072e30 0x7ff7dc14a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 -- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc15fef0 msgr2=0x7ff7dc0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc15fef0 0x7ff7dc0b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 -- 172.21.15.83:0/2309333854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc14ab90 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e0918640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d5b0 tx=0x7ff7d000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0018020 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d0014620 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc157990 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00149d0 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc152550 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.684+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7dc0b5e60 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.688+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff7d001b020 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.688+0000 7ff7cdffb640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 0x7ff7c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.688+0000 7ff7e1119640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 0x7ff7c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.688+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7ff7d008f460 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.688+0000 7ff7e1119640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 0x7ff7c0071940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80096f0 tx=0x7ff7d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.692+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff7d0022020 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.896+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff7dc164720 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.896+0000 7ff7cdffb640 1 -- 172.21.15.83:0/2309333854 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7ff7d005c600 con 0x7ff7dc0b5a50 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T21:59:53.890 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:59:53.891 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.892 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.893 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T21:59:53.894 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T21:59:53.896 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.899 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T21:59:53.900 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.901 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T21:59:53.902 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T21:59:53.903 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 msgr2=0x7ff7c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 0x7ff7c0071940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80096f0 tx=0x7ff7d8008040 comp rx=0 tx=0).stop 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 msgr2=0x7ff7dc0b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d5b0 tx=0x7ff7d000da70 comp rx=0 tx=0).stop 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 shutdown_connections 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7c006f430 0x7ff7c0071940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7dc15fef0 0x7ff7dc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7dc0b5a50 0x7ff7dc0b0c30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 --2- 172.21.15.83:0/2309333854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7dc072e30 0x7ff7dc14a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 >> 172.21.15.83:0/2309333854 conn(0x7ff7dc06cd90 msgr2=0x7ff7dc15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 shutdown_connections 2024-05-17T21:59:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:53.900+0000 7ff7e337d640 1 -- 172.21.15.83:0/2309333854 wait complete. 2024-05-17T21:59:53.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T21:59:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.064+0000 7fb55e9b1640 1 Processor -- start 2024-05-17T21:59:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.064+0000 7fb55e9b1640 1 -- start start 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55e9b1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb55806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55e9b1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb558073540 0x7fb5580b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55e9b1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5580b49b0 con 0x7fb5580b5a50 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55e9b1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb5580b4c90 con 0x7fb558073540 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55e9b1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb5580b4f70 con 0x7fb55815fef0 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb558073540 0x7fb5580b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb558073540 0x7fb5580b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49088/0 (socket says 172.21.15.83:49088) 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3329999383 learned_addr learned my addr 172.21.15.83:0/3329999383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb557fff640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb55806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3329999383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 msgr2=0x7fb55806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb55806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3329999383 >> v1:172.21.15.155:6789/0 conn(0x7fb5580b5a50 legacy=0x7fb558072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:54.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.068+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3329999383 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5580b5250 con 0x7fb558073540 2024-05-17T21:59:54.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55cf4e640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558073540 0x7fb5580b5e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb550009870 tx=0x7fb55001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0010686e6a5b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb556ffd640 1 -- 172.21.15.83:0/3329999383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500ab5f0 con 0x7fb558073540 2024-05-17T21:59:54.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb556ffd640 1 -- 172.21.15.83:0/3329999383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5500b0070 con 0x7fb558073540 2024-05-17T21:59:54.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb556ffd640 1 -- 172.21.15.83:0/3329999383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500ab9b0 con 0x7fb558073540 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3329999383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558073540 msgr2=0x7fb5580b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558073540 0x7fb5580b5e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb550009870 tx=0x7fb55001c560 comp rx=0 tx=0).stop 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3329999383 shutdown_connections 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558073540 0x7fb5580b5e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3329999383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb55806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3329999383 >> 172.21.15.83:0/3329999383 conn(0x7fb55806cd90 msgr2=0x7fb5580b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3329999383 shutdown_connections 2024-05-17T21:59:54.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.072+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3329999383 wait complete. 2024-05-17T21:59:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 Processor -- start 2024-05-17T21:59:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- start start 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580b5a50 0x7fb558149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb5581582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5580af9b0 con 0x7fb558158ca0 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb55806ffa0 con 0x7fb5580b5a50 2024-05-17T21:59:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb5580afb30 con 0x7fb55815fef0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49094/0 (socket says 172.21.15.83:49094) 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3460222585 learned_addr learned my addr 172.21.15.83:0/3460222585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb5577fe640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb5581582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb557fff640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580b5a50 0x7fb558149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 msgr2=0x7fb5581582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb5581582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580b5a50 msgr2=0x7fb558149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580b5a50 0x7fb558149b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 -- 172.21.15.83:0/3460222585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5580cf270 con 0x7fb558158ca0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb5577fe640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb5581582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55cf4e640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb55001ce00 tx=0x7fb55001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500be5f0 con 0x7fb558158ca0 2024-05-17T21:59:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5580cf550 con 0x7fb558158ca0 2024-05-17T21:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5580cfb10 con 0x7fb558158ca0 2024-05-17T21:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5500be790 con 0x7fb558158ca0 2024-05-17T21:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.076+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500beb80 con 0x7fb558158ca0 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5380052a0 con 0x7fb558158ca0 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5500b43f0 con 0x7fb558158ca0 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb554ff9640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 0x7fb544071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7fb550128d70 con 0x7fb558158ca0 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb557fff640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 0x7fb544071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.080+0000 7fb557fff640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 0x7fb544071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5480096f0 tx=0x7fb548008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.084+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb5500c6020 con 0x7fb558158ca0 2024-05-17T21:59:54.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.232+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb538005860 con 0x7fb558158ca0 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.232+0000 7fb554ff9640 1 -- 172.21.15.83:0/3460222585 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) ==== 84+0+65 (secure 0 0 0) 0x7fb550016150 con 0x7fb558158ca0 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T21:59:54.227 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 msgr2=0x7fb544071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 0x7fb544071910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb5480096f0 tx=0x7fb548008040 comp rx=0 tx=0).stop 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 msgr2=0x7fb55815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb55001ce00 tx=0x7fb55001ada0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 shutdown_connections 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb558158ca0 0x7fb55815b570 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb55815fef0 0x7fb5581582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb54406f400 0x7fb544071910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 --2- 172.21.15.83:0/3460222585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580b5a50 0x7fb558149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 >> 172.21.15.83:0/3460222585 conn(0x7fb55806cd90 msgr2=0x7fb55815d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 shutdown_connections 2024-05-17T21:59:54.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.236+0000 7fb55e9b1640 1 -- 172.21.15.83:0/3460222585 wait complete. 2024-05-17T21:59:54.248 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T21:59:54.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T21:59:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 Processor -- start 2024-05-17T21:59:54.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 -- start start 2024-05-17T21:59:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04c00ce0f0 0x7f04c00b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04c00b2610 0x7f04c015d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c00c8ea0 con 0x7f04c00b51e0 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f04c00c91d0 con 0x7f04c00b2610 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c7057640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f04c015ddd0 con 0x7f04c00ce0f0 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c55f4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04c00b51e0 0x7f04c00b55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50408/0 (socket says 172.21.15.83:50408) 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04c00b2610 0x7f04c015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c55f4640 1 -- 172.21.15.83:0/1104520446 learned_addr learned my addr 172.21.15.83:0/1104520446 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04c4df3640 1 --2- 172.21.15.83:0/1104520446 >> v2:172.21.15.155:3300/0 conn(0x7f04c00ce0f0 0x7f04c00b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bffff640 1 -- 172.21.15.83:0/1104520446 >> v2:172.21.15.155:3300/0 conn(0x7f04c00ce0f0 msgr2=0x7f04c00b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bffff640 1 --2- 172.21.15.83:0/1104520446 >> v2:172.21.15.155:3300/0 conn(0x7f04c00ce0f0 0x7f04c00b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bffff640 1 -- 172.21.15.83:0/1104520446 >> v1:172.21.15.83:6789/0 conn(0x7f04c00b51e0 legacy=0x7f04c00b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bffff640 1 -- 172.21.15.83:0/1104520446 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c00c9420 con 0x7f04c00b2610 2024-05-17T21:59:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.408+0000 7f04bf7fe640 1 -- 172.21.15.83:0/1104520446 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117147247 0 0) 0x7f04c00c8ea0 con 0x7f04c00b51e0 2024-05-17T21:59:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04bffff640 1 --2- 172.21.15.83:0/1104520446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00b2610 0x7f04c015d890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04c00d10c0 tx=0x7f04b401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511de9e341566011 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04bf7fe640 1 -- 172.21.15.83:0/1104520446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40b1070 con 0x7f04c00b2610 2024-05-17T21:59:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04bf7fe640 1 -- 172.21.15.83:0/1104520446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b40ac620 con 0x7f04c00b2610 2024-05-17T21:59:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04bf7fe640 1 -- 172.21.15.83:0/1104520446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40ac9f0 con 0x7f04c00b2610 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 -- 172.21.15.83:0/1104520446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00b2610 msgr2=0x7f04c015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 --2- 172.21.15.83:0/1104520446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00b2610 0x7f04c015d890 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f04c00d10c0 tx=0x7f04b401c3c0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 -- 172.21.15.83:0/1104520446 shutdown_connections 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 --2- 172.21.15.83:0/1104520446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00b2610 0x7f04c015d890 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 --2- 172.21.15.83:0/1104520446 >> v2:172.21.15.155:3300/0 conn(0x7f04c00ce0f0 0x7f04c00b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 -- 172.21.15.83:0/1104520446 >> 172.21.15.83:0/1104520446 conn(0x7f04c00d3600 msgr2=0x7f04c00d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 -- 172.21.15.83:0/1104520446 shutdown_connections 2024-05-17T21:59:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.412+0000 7f04c7057640 1 -- 172.21.15.83:0/1104520446 wait complete. 2024-05-17T21:59:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 Processor -- start 2024-05-17T21:59:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- start start 2024-05-17T21:59:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c00b2610 0x7f04c00b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04c00b51e0 0x7f04c0160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0161f10 con 0x7f04c00ce0f0 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0161d90 con 0x7f04c00b2610 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04c0162090 con 0x7f04c00b51e0 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49108/0 (socket says 172.21.15.83:49108) 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 -- 172.21.15.83:0/1964472045 learned_addr learned my addr 172.21.15.83:0/1964472045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c4df3640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c00b2610 0x7f04c00b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04bffff640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04c00b51e0 0x7f04c0160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 -- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04c00b51e0 msgr2=0x7f04c0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04c00b51e0 0x7f04c0160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 -- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c00b2610 msgr2=0x7f04c00b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c00b2610 0x7f04c00b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 -- 172.21.15.83:0/1964472045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c00b6e90 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c55f4640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c6b0 tx=0x7f04b800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8018070 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b8004db0 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b800a880 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c00b8810 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.416+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c00b8dd0 con 0x7f04c00ce0f0 2024-05-17T21:59:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c00b8580 con 0x7f04c00ce0f0 2024-05-17T21:59:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04b801b070 con 0x7f04c00ce0f0 2024-05-17T21:59:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04bd7fa640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 0x7f04ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4857+0+0 (secure 0 0 0) 0x7f04b808e510 con 0x7f04c00ce0f0 2024-05-17T21:59:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04c4df3640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 0x7f04ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.420+0000 7f04c4df3640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 0x7f04ac071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04b00096f0 tx=0x7f04b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.424+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f04b805b7e0 con 0x7f04c00ce0f0 2024-05-17T21:59:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:54.580+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f04c00b8330 con 0x7f04c00ce0f0 2024-05-17T21:59:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.508+0000 7f04bd7fa640 1 -- 172.21.15.83:0/1964472045 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v53) ==== 138+0+0 (secure 0 0 0) 0x7f04c00b8330 con 0x7f04c00ce0f0 2024-05-17T21:59:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T21:59:55.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 msgr2=0x7f04ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 0x7f04ac071910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f04b00096f0 tx=0x7f04b0008040 comp rx=0 tx=0).stop 2024-05-17T21:59:55.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 msgr2=0x7f04c014fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c6b0 tx=0x7f04b800cb70 comp rx=0 tx=0).stop 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 shutdown_connections 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04c00ce0f0 0x7f04c014fc40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04c00b51e0 0x7f04c0160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04ac06f400 0x7f04ac071910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 --2- 172.21.15.83:0/1964472045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04c00b2610 0x7f04c00b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 >> 172.21.15.83:0/1964472045 conn(0x7f04c00d3600 msgr2=0x7f04c00ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 shutdown_connections 2024-05-17T21:59:55.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.512+0000 7f04c7057640 1 -- 172.21.15.83:0/1964472045 wait complete. 2024-05-17T21:59:55.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T21:59:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.700+0000 7f12720d3640 1 Processor -- start 2024-05-17T21:59:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.700+0000 7f12720d3640 1 -- start start 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c0b3d00 0x7f126c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f126c073590 0x7f126c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f126c0d10d0 con 0x7f126c143f20 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f126c0b4110 con 0x7f126c073590 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f126c070770 con 0x7f126c0b3d00 2024-05-17T21:59:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f126c073590 0x7f126c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f126c143f20 0x7f126c072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50410/0 (socket says 172.21.15.83:50410) 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f126c073590 0x7f126c1447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49112/0 (socket says 172.21.15.83:49112) 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 -- 172.21.15.83:0/481966570 learned_addr learned my addr 172.21.15.83:0/481966570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126b7fe640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c0b3d00 0x7f126c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 -- 172.21.15.83:0/481966570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c0b3d00 msgr2=0x7f126c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c0b3d00 0x7f126c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 -- 172.21.15.83:0/481966570 >> v1:172.21.15.83:6789/0 conn(0x7f126c143f20 legacy=0x7f126c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 -- 172.21.15.83:0/481966570 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c070120 con 0x7f126c073590 2024-05-17T21:59:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126bfff640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c1447f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f126401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44181e228665a9eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126a7fc640 1 -- 172.21.15.83:0/481966570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640a5790 con 0x7f126c073590 2024-05-17T21:59:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126a7fc640 1 -- 172.21.15.83:0/481966570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12640a5930 con 0x7f126c073590 2024-05-17T21:59:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f126a7fc640 1 -- 172.21.15.83:0/481966570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640a5d00 con 0x7f126c073590 2024-05-17T21:59:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- 172.21.15.83:0/481966570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 msgr2=0x7f126c1447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c1447f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1264009e50 tx=0x7f126401ca30 comp rx=0 tx=0).stop 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- 172.21.15.83:0/481966570 shutdown_connections 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c1447f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 --2- 172.21.15.83:0/481966570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c0b3d00 0x7f126c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- 172.21.15.83:0/481966570 >> 172.21.15.83:0/481966570 conn(0x7f126c06cd90 msgr2=0x7f126c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- 172.21.15.83:0/481966570 shutdown_connections 2024-05-17T21:59:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.704+0000 7f12720d3640 1 -- 172.21.15.83:0/481966570 wait complete. 2024-05-17T21:59:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 Processor -- start 2024-05-17T21:59:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 -- start start 2024-05-17T21:59:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 0x7f126c1fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c143f20 0x7f126c13bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f126c070c00 con 0x7f126c073590 2024-05-17T21:59:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f126c145540 con 0x7f126c0b3d00 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f12720d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f126c15b340 con 0x7f126c143f20 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49114/0 (socket says 172.21.15.83:49114) 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 -- 172.21.15.83:0/167514907 learned_addr learned my addr 172.21.15.83:0/167514907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126affd640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 0x7f126c1fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126bfff640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c143f20 0x7f126c13bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 -- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c143f20 msgr2=0x7f126c13bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c143f20 0x7f126c13bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 -- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 msgr2=0x7f126c1fa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 0x7f126c1fa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 -- 172.21.15.83:0/167514907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1fad60 con 0x7f126c073590 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126affd640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 0x7f126c1fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.708+0000 7f126b7fe640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d620 tx=0x7f125c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c00b620 con 0x7f126c073590 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c019070 con 0x7f126c073590 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c15bb20 con 0x7f126c073590 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c00b9e0 con 0x7f126c073590 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c15c0e0 con 0x7f126c073590 2024-05-17T21:59:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c1447f0 con 0x7f126c073590 2024-05-17T21:59:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f125c00bb80 con 0x7f126c073590 2024-05-17T21:59:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 0x7f1258071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5227+0+0 (secure 0 0 0) 0x7f125c08feb0 con 0x7f126c073590 2024-05-17T21:59:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f126affd640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 0x7f1258071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:55.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.712+0000 7f126affd640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 0x7f1258071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f126c141830 tx=0x7f1260009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:55.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.716+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f125c05d0f0 con 0x7f126c073590 2024-05-17T21:59:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:55.876+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f126c0711a0 con 0x7f126c073590 2024-05-17T21:59:56.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.516+0000 7f1253fff640 1 -- 172.21.15.83:0/167514907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v54) ==== 170+0+0 (secure 0 0 0) 0x7f125c05c9f0 con 0x7f126c073590 2024-05-17T21:59:56.508 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T21:59:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 msgr2=0x7f1258071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 0x7f1258071910 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f126c141830 tx=0x7f1260009040 comp rx=0 tx=0).stop 2024-05-17T21:59:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 msgr2=0x7f126c139eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f125c00d620 tx=0x7f125c00dae0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 shutdown_connections 2024-05-17T21:59:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f126c143f20 0x7f126c13bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f125806f400 0x7f1258071910 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f126c0b3d00 0x7f126c1fa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 --2- 172.21.15.83:0/167514907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c073590 0x7f126c139eb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 >> 172.21.15.83:0/167514907 conn(0x7f126c06cd90 msgr2=0x7f126c140c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.520+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 shutdown_connections 2024-05-17T21:59:56.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.524+0000 7f12720d3640 1 -- 172.21.15.83:0/167514907 wait complete. 2024-05-17T21:59:56.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T21:59:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.704+0000 7f20ed325640 1 Processor -- start 2024-05-17T21:59:56.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.704+0000 7f20ed325640 1 -- start start 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20ed325640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f20e80b7570 0x7f20e80cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20ed325640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20e80cef60 0x7f20e81665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20ed325640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20e80d0c20 con 0x7f20e80ce440 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20ed325640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80cda70 con 0x7f20e80cef60 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20ed325640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80d0f00 con 0x7f20e80b7570 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20de7fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f20e80ce440 0x7f20e80ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50418/0 (socket says 172.21.15.83:50418) 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20de7fc640 1 -- 172.21.15.83:0/3496669539 learned_addr learned my addr 172.21.15.83:0/3496669539 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e77fe640 1 --2- 172.21.15.83:0/3496669539 >> v2:172.21.15.83:3300/0 conn(0x7f20e80cef60 0x7f20e81665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e6ffd640 1 --2- 172.21.15.83:0/3496669539 >> v2:172.21.15.155:3300/0 conn(0x7f20e80b7570 0x7f20e80cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e67fc640 1 -- 172.21.15.83:0/3496669539 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909680047 0 0) 0x7f20e80d0c20 con 0x7f20e80ce440 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e67fc640 1 -- 172.21.15.83:0/3496669539 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d4003530 con 0x7f20e80ce440 2024-05-17T21:59:56.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e77fe640 1 -- 172.21.15.83:0/3496669539 >> v2:172.21.15.155:3300/0 conn(0x7f20e80b7570 msgr2=0x7f20e80cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e77fe640 1 --2- 172.21.15.83:0/3496669539 >> v2:172.21.15.155:3300/0 conn(0x7f20e80b7570 0x7f20e80cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e77fe640 1 -- 172.21.15.83:0/3496669539 >> v1:172.21.15.83:6789/0 conn(0x7f20e80ce440 legacy=0x7f20e80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.708+0000 7f20e77fe640 1 -- 172.21.15.83:0/3496669539 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e80d11e0 con 0x7f20e80cef60 2024-05-17T21:59:56.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20e77fe640 1 --2- 172.21.15.83:0/3496669539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80cef60 0x7f20e81665f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20e80cdbf0 tx=0x7f20e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa9e2e9d63b20d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:56.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20e67fc640 1 -- 172.21.15.83:0/3496669539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00b2070 con 0x7f20e80cef60 2024-05-17T21:59:56.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20e67fc640 1 -- 172.21.15.83:0/3496669539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e00a5820 con 0x7f20e80cef60 2024-05-17T21:59:56.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20e67fc640 1 -- 172.21.15.83:0/3496669539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00a5bf0 con 0x7f20e80cef60 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- 172.21.15.83:0/3496669539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80cef60 msgr2=0x7f20e81665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 --2- 172.21.15.83:0/3496669539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80cef60 0x7f20e81665f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20e80cdbf0 tx=0x7f20e001c930 comp rx=0 tx=0).stop 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- 172.21.15.83:0/3496669539 shutdown_connections 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 --2- 172.21.15.83:0/3496669539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80cef60 0x7f20e81665f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 --2- 172.21.15.83:0/3496669539 >> v2:172.21.15.155:3300/0 conn(0x7f20e80b7570 0x7f20e80cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- 172.21.15.83:0/3496669539 >> 172.21.15.83:0/3496669539 conn(0x7f20e80b4170 msgr2=0x7f20e80b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- 172.21.15.83:0/3496669539 shutdown_connections 2024-05-17T21:59:56.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- 172.21.15.83:0/3496669539 wait complete. 2024-05-17T21:59:56.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 Processor -- start 2024-05-17T21:59:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.712+0000 7f20ed325640 1 -- start start 2024-05-17T21:59:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 0x7f20e80b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e80cef60 0x7f20e80b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81694d0 con 0x7f20e80b7570 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8169350 con 0x7f20e80ce440 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20e8169650 con 0x7f20e80cef60 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49132/0 (socket says 172.21.15.83:49132) 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 -- 172.21.15.83:0/655507264 learned_addr learned my addr 172.21.15.83:0/655507264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 -- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e80cef60 msgr2=0x7f20e80b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e80cef60 0x7f20e80b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20de7fc640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 0x7f20e80b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 -- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 msgr2=0x7f20e80b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 0x7f20e80b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 -- 172.21.15.83:0/655507264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e80a0240 con 0x7f20e80b7570 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20de7fc640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 0x7f20e80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20e6ffd640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d5f0 tx=0x7f20d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80125f0 con 0x7f20e80b7570 2024-05-17T21:59:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d8019070 con 0x7f20e80b7570 2024-05-17T21:59:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e809c600 con 0x7f20e80b7570 2024-05-17T21:59:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80129b0 con 0x7f20e80b7570 2024-05-17T21:59:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e809cbc0 con 0x7f20e80b7570 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e80cdf00 con 0x7f20e80b7570 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f20d8012b50 con 0x7f20e80b7570 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 0x7f20c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5609+0+0 (secure 0 0 0) 0x7f20d808f690 con 0x7f20e80b7570 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.716+0000 7f20de7fc640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 0x7f20c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:56.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.720+0000 7f20de7fc640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 0x7f20c8071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f20e8072000 tx=0x7f20d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:56.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.724+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f20d805c5e0 con 0x7f20e80b7570 2024-05-17T21:59:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:56.924+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f20e806f000 con 0x7f20e80b7570 2024-05-17T21:59:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.544+0000 7f20dffff640 1 -- 172.21.15.83:0/655507264 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) ==== 172+0+0 (secure 0 0 0) 0x7f20d805bee0 con 0x7f20e80b7570 2024-05-17T21:59:57.539 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 8 and data pool 9 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 msgr2=0x7f20c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 0x7f20c8071940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f20e8072000 tx=0x7f20d0009040 comp rx=0 tx=0).stop 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 msgr2=0x7f20e815fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f20d800d5f0 tx=0x7f20d800dab0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 shutdown_connections 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e80cef60 0x7f20e80b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f430 0x7f20c8071940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e80ce440 0x7f20e80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 --2- 172.21.15.83:0/655507264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e80b7570 0x7f20e815fce0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 >> 172.21.15.83:0/655507264 conn(0x7f20e80b4170 msgr2=0x7f20e80b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 shutdown_connections 2024-05-17T21:59:57.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.548+0000 7f20ed325640 1 -- 172.21.15.83:0/655507264 wait complete. 2024-05-17T21:59:57.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 Processor -- start 2024-05-17T21:59:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 -- start start 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdfa8073540 0x7fdfa8168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa815e790 con 0x7fdfa8144080 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa815ea40 con 0x7fdfa80d1040 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfac9b5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa815ed20 con 0x7fdfa8073540 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdfa8144080 0x7fdfa8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50224/0 (socket says 172.21.15.83:50224) 2024-05-17T21:59:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa5d9b640 1 -- 172.21.15.83:0/2168297303 learned_addr learned my addr 172.21.15.83:0/2168297303 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa6d9d640 1 --2- 172.21.15.83:0/2168297303 >> v2:172.21.15.155:3300/0 conn(0x7fdfa8073540 0x7fdfa8168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 926228678 0 0) 0x7fdfa815e790 con 0x7fdfa8144080 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf90003530 con 0x7fdfa8144080 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416539524 0 0) 0x7fdf90003530 con 0x7fdfa8144080 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 >> v2:172.21.15.155:3300/0 conn(0x7fdfa8073540 msgr2=0x7fdfa8168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 --2- 172.21.15.83:0/2168297303 >> v2:172.21.15.155:3300/0 conn(0x7fdfa8073540 0x7fdfa8168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 >> v1:172.21.15.155:6789/0 conn(0x7fdfa80d1040 legacy=0x7fdfa815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa815f000 con 0x7fdfa8144080 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fdf9c001700 con 0x7fdfa8144080 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 >> v1:172.21.15.83:6789/0 conn(0x7fdfa8144080 legacy=0x7fdfa8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90003a60 0x7fdf90003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdf90004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90005260 0x7fdf9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa815ed20 con 0x7fdf90004810 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9000c1f0 con 0x7fdf90005260 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9000c3f0 con 0x7fdf90003a60 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.736+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdf90004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa659c640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90003a60 0x7fdf90003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 -- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90003a60 msgr2=0x7fdf90003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90003a60 0x7fdf90003ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa6d9d640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90005260 0x7fdf9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 -- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90005260 msgr2=0x7fdf9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90005260 0x7fdf9000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa6d9d640 1 -- 172.21.15.83:0/2168297303 reap_dead start 2024-05-17T21:59:57.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 -- 172.21.15.83:0/2168297303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf90003530 con 0x7fdf90004810 2024-05-17T21:59:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdf90004c20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8073ed0 tx=0x7fdf9c01f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e494ca7236af42c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c007cf0 con 0x7fdf90004810 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9c005ae0 con 0x7fdf90004810 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfa559a640 1 -- 172.21.15.83:0/2168297303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c007070 con 0x7fdf90004810 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 msgr2=0x7fdf90004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdf90004c20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8073ed0 tx=0x7fdf9c01f2d0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2168297303 shutdown_connections 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2168297303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdf90004c20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2168297303 >> 172.21.15.83:0/2168297303 conn(0x7fdfa806cd90 msgr2=0x7fdfa80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2168297303 shutdown_connections 2024-05-17T21:59:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.740+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2168297303 wait complete. 2024-05-17T21:59:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 Processor -- start 2024-05-17T21:59:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- start start 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa8161130 0x7fdfa8161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa8140e90 0x7fdfa81412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8144490 con 0x7fdf90004810 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa806f210 con 0x7fdfa8161130 2024-05-17T21:59:57.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa815e080 con 0x7fdfa8140e90 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47824/0 (socket says 172.21.15.83:47824) 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 -- 172.21.15.83:0/2058791138 learned_addr learned my addr 172.21.15.83:0/2058791138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa8161130 0x7fdfa8161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa6d9d640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa8140e90 0x7fdfa81412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 -- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa8140e90 msgr2=0x7fdfa81412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa8140e90 0x7fdfa81412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 -- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa8161130 msgr2=0x7fdfa8161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa8161130 0x7fdfa8161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 -- 172.21.15.83:0/2058791138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa81417e0 con 0x7fdf90004810 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfa659c640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d910 tx=0x7fdf9800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98018070 con 0x7fdf90004810 2024-05-17T21:59:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf98003db0 con 0x7fdf90004810 2024-05-17T21:59:57.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa813d1d0 con 0x7fdf90004810 2024-05-17T21:59:57.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800b970 con 0x7fdf90004810 2024-05-17T21:59:57.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa813d790 con 0x7fdf90004810 2024-05-17T21:59:57.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.744+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa815e3b0 con 0x7fdf90004810 2024-05-17T21:59:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.748+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdf9801b070 con 0x7fdf90004810 2024-05-17T21:59:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.748+0000 7fdf96ffd640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 0x7fdf88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.748+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 0x7fdf88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.748+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fdf9808ed20 con 0x7fdf90004810 2024-05-17T21:59:57.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.748+0000 7fdfa5d9b640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 0x7fdf88071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ae000 tx=0x7fdf9c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:57.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.752+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdfa81417e0 con 0x7fdf90004810 2024-05-17T21:59:57.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.900+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfa8070830 con 0x7fdf90004810 2024-05-17T21:59:57.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdf96ffd640 1 -- 172.21.15.83:0/2058791138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16654 (secure 0 0 0) 0x7fdf9805bca0 con 0x7fdf90004810 2024-05-17T21:59:57.897 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:56.927251+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:59:54.592647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-17T21:59:55.893117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:57.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 msgr2=0x7fdf88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 0x7fdf88071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ae000 tx=0x7fdf9c007040 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 msgr2=0x7fdfa81f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d910 tx=0x7fdf9800ddd0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 shutdown_connections 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfa8140e90 0x7fdfa81412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf8806f400 0x7fdf88071910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa8161130 0x7fdfa8161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 --2- 172.21.15.83:0/2058791138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90004810 0x7fdfa81f2e00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 >> 172.21.15.83:0/2058791138 conn(0x7fdfa806cd90 msgr2=0x7fdfa8143450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 shutdown_connections 2024-05-17T21:59:57.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:57.904+0000 7fdfac9b5640 1 -- 172.21.15.83:0/2058791138 wait complete. 2024-05-17T21:59:57.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T21:59:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 Processor -- start 2024-05-17T21:59:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 -- start start 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14100d1040 0x7f1410144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f141015e080 0x7f1410072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141015ea40 con 0x7f141015e490 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f141015ed20 con 0x7f141015e080 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1416ebc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f141015f000 con 0x7f14100d1040 2024-05-17T21:59:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f141015e080 0x7f1410072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1415459640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f141015e490 0x7f14101616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50236/0 (socket says 172.21.15.83:50236) 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f141015e080 0x7f1410072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47828/0 (socket says 172.21.15.83:47828) 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1415459640 1 -- 172.21.15.83:0/3722087848 learned_addr learned my addr 172.21.15.83:0/3722087848 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f1414c58640 1 --2- 172.21.15.83:0/3722087848 >> v2:172.21.15.155:3300/0 conn(0x7f14100d1040 0x7f1410144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 -- 172.21.15.83:0/3722087848 >> v2:172.21.15.155:3300/0 conn(0x7f14100d1040 msgr2=0x7f1410144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 --2- 172.21.15.83:0/3722087848 >> v2:172.21.15.155:3300/0 conn(0x7f14100d1040 0x7f1410144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 -- 172.21.15.83:0/3722087848 >> v1:172.21.15.83:6789/0 conn(0x7f141015e490 legacy=0x7f14101616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140f7fe640 1 -- 172.21.15.83:0/3722087848 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785155163 0 0) 0x7f141015ea40 con 0x7f141015e490 2024-05-17T21:59:58.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.060+0000 7f140ffff640 1 -- 172.21.15.83:0/3722087848 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141006ef30 con 0x7f141015e080 2024-05-17T21:59:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f140ffff640 1 --2- 172.21.15.83:0/3722087848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e080 0x7f1410072e30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1410071d60 tx=0x7f140001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883d1b5ba6ac123d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f140f7fe640 1 -- 172.21.15.83:0/3722087848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000b1070 con 0x7f141015e080 2024-05-17T21:59:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f140f7fe640 1 -- 172.21.15.83:0/3722087848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14000ac5f0 con 0x7f141015e080 2024-05-17T21:59:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f140f7fe640 1 -- 172.21.15.83:0/3722087848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000ac9c0 con 0x7f141015e080 2024-05-17T21:59:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 -- 172.21.15.83:0/3722087848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e080 msgr2=0x7f1410072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 --2- 172.21.15.83:0/3722087848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e080 0x7f1410072e30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1410071d60 tx=0x7f140001c480 comp rx=0 tx=0).stop 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 -- 172.21.15.83:0/3722087848 shutdown_connections 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 --2- 172.21.15.83:0/3722087848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e080 0x7f1410072e30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 --2- 172.21.15.83:0/3722087848 >> v2:172.21.15.155:3300/0 conn(0x7f14100d1040 0x7f1410144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 -- 172.21.15.83:0/3722087848 >> 172.21.15.83:0/3722087848 conn(0x7f141006cd90 msgr2=0x7f14100b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 -- 172.21.15.83:0/3722087848 shutdown_connections 2024-05-17T21:59:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.064+0000 7f1416ebc640 1 -- 172.21.15.83:0/3722087848 wait complete. 2024-05-17T21:59:58.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 Processor -- start 2024-05-17T21:59:58.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- start start 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14100d1040 0x7f141015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141015c4c0 0x7f141015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f141016abe0 con 0x7f141015e490 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f141016aa60 con 0x7f141015c4c0 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f141016ad60 con 0x7f14100d1040 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47834/0 (socket says 172.21.15.83:47834) 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 -- 172.21.15.83:0/2919379253 learned_addr learned my addr 172.21.15.83:0/2919379253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1414c58640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14100d1040 0x7f141015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 -- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14100d1040 msgr2=0x7f141015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14100d1040 0x7f141015a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 -- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141015c4c0 msgr2=0x7f141015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1415459640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141015c4c0 0x7f141015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141015c4c0 0x7f141015c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 -- 172.21.15.83:0/2919379253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141015ce10 con 0x7f141015e490 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140ffff640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f140001c960 tx=0x7f1400019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000b1040 con 0x7f141015e490 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14000ba070 con 0x7f141015e490 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141015a6a0 con 0x7f141015e490 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400006e20 con 0x7f141015e490 2024-05-17T21:59:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101468b0 con 0x7f141015e490 2024-05-17T21:59:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.068+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1410072e30 con 0x7f141015e490 2024-05-17T21:59:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.072+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14000b5670 con 0x7f141015e490 2024-05-17T21:59:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.072+0000 7f140d7fa640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 0x7f13f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.072+0000 7f1414c58640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 0x7f13f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.072+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f1400129080 con 0x7f141015e490 2024-05-17T21:59:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.072+0000 7f1414c58640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 0x7f13f0071940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13f80096f0 tx=0x7f13f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.076+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f141015ce10 con 0x7f141015e490 2024-05-17T21:59:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.224+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f141015de60 con 0x7f141015e490 2024-05-17T21:59:58.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f140d7fa640 1 -- 172.21.15.83:0/2919379253 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16654 (secure 0 0 0) 0x7f14000a9070 con 0x7f141015e490 2024-05-17T21:59:58.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T21:59:58.220 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:56.927251+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:59:54.592647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-17T21:59:55.893117+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 msgr2=0x7f13f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 0x7f13f0071940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f13f80096f0 tx=0x7f13f8008040 comp rx=0 tx=0).stop 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 msgr2=0x7f14101446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f140001c960 tx=0x7f1400019220 comp rx=0 tx=0).stop 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 shutdown_connections 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f13f006f430 0x7f13f0071940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f141015c4c0 0x7f141015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141015e490 0x7f14101446d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 --2- 172.21.15.83:0/2919379253 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14100d1040 0x7f141015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 >> 172.21.15.83:0/2919379253 conn(0x7f141006cd90 msgr2=0x7f141013d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:58.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 shutdown_connections 2024-05-17T21:59:58.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.228+0000 7f1416ebc640 1 -- 172.21.15.83:0/2919379253 wait complete. 2024-05-17T21:59:58.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T21:59:58.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 Processor -- start 2024-05-17T21:59:58.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 -- start start 2024-05-17T21:59:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 0x7f122c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f122c073590 0x7f122c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c0d10d0 con 0x7f122c143f20 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f122c0b4110 con 0x7f122c073590 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f12330f7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f122c070770 con 0x7f122c0b3d00 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f122c073590 0x7f122c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f122c073590 0x7f122c1447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47844/0 (socket says 172.21.15.83:47844) 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 -- 172.21.15.83:0/1949706459 learned_addr learned my addr 172.21.15.83:0/1949706459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1230e93640 1 --2- 172.21.15.83:0/1949706459 >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 0x7f122c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 -- 172.21.15.83:0/1949706459 >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 msgr2=0x7f122c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 --2- 172.21.15.83:0/1949706459 >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 0x7f122c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 -- 172.21.15.83:0/1949706459 >> v1:172.21.15.155:6789/0 conn(0x7f122c143f20 legacy=0x7f122c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.396+0000 7f1231694640 1 -- 172.21.15.83:0/1949706459 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c070120 con 0x7f122c073590 2024-05-17T21:59:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f1230e93640 1 --2- 172.21.15.83:0/1949706459 >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 0x7f122c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f1231694640 1 --2- 172.21.15.83:0/1949706459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c073590 0x7f122c1447f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1224009e50 tx=0x7f122401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e60c5e48d3f2d13 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f122b7fe640 1 -- 172.21.15.83:0/1949706459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240a5700 con 0x7f122c073590 2024-05-17T21:59:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f122b7fe640 1 -- 172.21.15.83:0/1949706459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12240a58a0 con 0x7f122c073590 2024-05-17T21:59:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f122b7fe640 1 -- 172.21.15.83:0/1949706459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240a5c70 con 0x7f122c073590 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 -- 172.21.15.83:0/1949706459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c073590 msgr2=0x7f122c1447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 --2- 172.21.15.83:0/1949706459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c073590 0x7f122c1447f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1224009e50 tx=0x7f122401c9f0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 -- 172.21.15.83:0/1949706459 shutdown_connections 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 --2- 172.21.15.83:0/1949706459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c073590 0x7f122c1447f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 --2- 172.21.15.83:0/1949706459 >> v2:172.21.15.155:3300/0 conn(0x7f122c0b3d00 0x7f122c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 -- 172.21.15.83:0/1949706459 >> 172.21.15.83:0/1949706459 conn(0x7f122c06cd90 msgr2=0x7f122c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 -- 172.21.15.83:0/1949706459 shutdown_connections 2024-05-17T21:59:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.400+0000 7f12330f7640 1 -- 172.21.15.83:0/1949706459 wait complete. 2024-05-17T21:59:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 Processor -- start 2024-05-17T21:59:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- start start 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f122c073590 0x7f122c163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f122c0b3d00 0x7f122c1f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f122c070c00 con 0x7f122c143f20 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f122c145540 con 0x7f122c0b3d00 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f122c13ee20 con 0x7f122c073590 2024-05-17T21:59:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47854/0 (socket says 172.21.15.83:47854) 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 -- 172.21.15.83:0/1775483214 learned_addr learned my addr 172.21.15.83:0/1775483214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1230e93640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f122c073590 0x7f122c163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f122bfff640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f122c0b3d00 0x7f122c1f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 -- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f122c073590 msgr2=0x7f122c163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f122c073590 0x7f122c163dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 -- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f122c0b3d00 msgr2=0x7f122c1f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f122c0b3d00 0x7f122c1f9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 -- 172.21.15.83:0/1775483214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c161670 con 0x7f122c143f20 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f1231694640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f122401b0a0 tx=0x7f1224019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240b8670 con 0x7f122c143f20 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12240be070 con 0x7f122c143f20 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c161950 con 0x7f122c143f20 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240b8a30 con 0x7f122c143f20 2024-05-17T21:59:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.404+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c161f10 con 0x7f122c143f20 2024-05-17T21:59:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c1447f0 con 0x7f122c143f20 2024-05-17T21:59:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f12240b8bd0 con 0x7f122c143f20 2024-05-17T21:59:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f12297fa640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 0x7f1210071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f122412a3b0 con 0x7f122c143f20 2024-05-17T21:59:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f1230e93640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 0x7f1210071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.408+0000 7f1230e93640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 0x7f1210071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1220006fd0 tx=0x7f1220008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.412+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f12240165e0 con 0x7f122c143f20 2024-05-17T21:59:58.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:58.600+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f122c146c10 con 0x7f122c143f20 2024-05-17T21:59:59.499 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-05-17T21:59:59.504+0000 7f7d2ae0a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T21:59:59.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.588+0000 7f12297fa640 1 -- 172.21.15.83:0/1775483214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f12240f7330 con 0x7f122c143f20 2024-05-17T21:59:59.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 msgr2=0x7f1210071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 0x7f1210071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1220006fd0 tx=0x7f1220008040 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 msgr2=0x7f122c1fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f122401b0a0 tx=0x7f1224019790 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 shutdown_connections 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f122c143f20 0x7f122c1fa520 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f121006f400 0x7f1210071910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f122c0b3d00 0x7f122c1f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 --2- 172.21.15.83:0/1775483214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f122c073590 0x7f122c163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 >> 172.21.15.83:0/1775483214 conn(0x7f122c06cd90 msgr2=0x7f122c145900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 shutdown_connections 2024-05-17T21:59:59.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.592+0000 7f12330f7640 1 -- 172.21.15.83:0/1775483214 wait complete. 2024-05-17T21:59:59.610 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T21:59:59.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T21:59:59.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 Processor -- start 2024-05-17T21:59:59.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 -- start start 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7100cf830 0x7fe7100b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7100b5640 con 0x7fe7100cdf40 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7100b58e0 con 0x7fe71015d700 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe717bec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe710070870 con 0x7fe7100cf830 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe715187640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe7100cdf40 0x7fe71015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50252/0 (socket says 172.21.15.83:50252) 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe715187640 1 -- 172.21.15.83:0/3419570370 learned_addr learned my addr 172.21.15.83:0/3419570370 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe715988640 1 --2- 172.21.15.83:0/3419570370 >> v2:172.21.15.155:3300/0 conn(0x7fe7100cf830 0x7fe7100b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1334554602 0 0) 0x7fe7100b5640 con 0x7fe7100cdf40 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6fc003530 con 0x7fe7100cdf40 2024-05-17T21:59:59.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2926366321 0 0) 0x7fe6fc003530 con 0x7fe7100cdf40 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 >> v2:172.21.15.155:3300/0 conn(0x7fe7100cf830 msgr2=0x7fe7100b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 --2- 172.21.15.83:0/3419570370 >> v2:172.21.15.155:3300/0 conn(0x7fe7100cf830 0x7fe7100b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 >> v1:172.21.15.155:6789/0 conn(0x7fe71015d700 legacy=0x7fe7101663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71006f060 con 0x7fe7100cdf40 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe700001720 con 0x7fe7100cdf40 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 >> v1:172.21.15.83:6789/0 conn(0x7fe7100cdf40 legacy=0x7fe71015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6fc003a60 0x7fe6fc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc004810 0x7fe6fc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 0x7fe6fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c280 con 0x7fe6fc005310 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c430 con 0x7fe6fc004810 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c630 con 0x7fe6fc003a60 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe715988640 1 --2- 172.21.15.83:0/3419570370 >> v2:172.21.15.155:3300/0 conn(0x7fe7100cf830 0x7fe7100b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T21:59:59.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe716189640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 0x7fe6fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.764+0000 7fe715187640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc004810 0x7fe6fc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe715988640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6fc003a60 0x7fe6fc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 -- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6fc003a60 msgr2=0x7fe6fc003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6fc003a60 0x7fe6fc003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 -- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc004810 msgr2=0x7fe6fc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc004810 0x7fe6fc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 -- 172.21.15.83:0/3419570370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc003530 con 0x7fe6fc005310 2024-05-17T21:59:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe715988640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6fc003a60 0x7fe6fc003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T21:59:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 -- 172.21.15.83:0/3419570370 reap_dead start 2024-05-17T21:59:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe716189640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 0x7fe6fc00b9d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe708001e90 tx=0x7fe70801c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e56b75fc4e792ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708001ae0 con 0x7fe6fc005310 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe708003b80 con 0x7fe6fc005310 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe714986640 1 -- 172.21.15.83:0/3419570370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080050d0 con 0x7fe6fc005310 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 -- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 msgr2=0x7fe6fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 0x7fe6fc00b9d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe708001e90 tx=0x7fe70801c770 comp rx=0 tx=0).stop 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 -- 172.21.15.83:0/3419570370 shutdown_connections 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 --2- 172.21.15.83:0/3419570370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6fc005310 0x7fe6fc00b9d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 -- 172.21.15.83:0/3419570370 >> 172.21.15.83:0/3419570370 conn(0x7fe71006cd90 msgr2=0x7fe7100b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 -- 172.21.15.83:0/3419570370 shutdown_connections 2024-05-17T21:59:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.768+0000 7fe717bec640 1 -- 172.21.15.83:0/3419570370 wait complete. 2024-05-17T21:59:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 Processor -- start 2024-05-17T21:59:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- start start 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc005310 0x7fe7101f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7101e5450 0x7fe710157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe71015db10 con 0x7fe7101e5040 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b54a0 con 0x7fe6fc005310 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe71015dc90 con 0x7fe7101e5450 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47866/0 (socket says 172.21.15.83:47866) 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 -- 172.21.15.83:0/2212639158 learned_addr learned my addr 172.21.15.83:0/2212639158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T21:59:59.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 -- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7101e5450 msgr2=0x7fe710157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7101e5450 0x7fe710157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 -- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc005310 msgr2=0x7fe7101f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc005310 0x7fe7101f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 -- 172.21.15.83:0/2212639158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7101e5c20 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe715187640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7000017d0 tx=0x7fe700007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000f680 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700018070 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7101e6470 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000fa20 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.772+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7101e6a30 con 0x7fe7101e5040 2024-05-17T21:59:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7101e5990 con 0x7fe7101e5040 2024-05-17T21:59:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe70000fbc0 con 0x7fe7101e5040 2024-05-17T21:59:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe7067fc640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 0x7fe6f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T21:59:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe715988640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 0x7fe6f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T21:59:59.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe70008f130 con 0x7fe7101e5040 2024-05-17T21:59:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.776+0000 7fe715988640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 0x7fe6f4071940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0096c0 tx=0x7fe70c006e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T21:59:59.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.780+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe70005c020 con 0x7fe7101e5040 2024-05-17T21:59:59.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T21:59:59.964+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe71015cff0 con 0x7fe7101e5040 2024-05-17T22:00:00.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:00:00.041 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:00:00.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.600+0000 7fe7067fc640 1 -- 172.21.15.83:0/2212639158 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fe70005b920 con 0x7fe7101e5040 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 msgr2=0x7fe6f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 0x7fe6f4071940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0096c0 tx=0x7fe70c006e00 comp rx=0 tx=0).stop 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 msgr2=0x7fe710156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7000017d0 tx=0x7fe700007430 comp rx=0 tx=0).stop 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 shutdown_connections 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7101e5450 0x7fe710157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7101e5040 0x7fe710156fe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6f406f430 0x7fe6f4071940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 --2- 172.21.15.83:0/2212639158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6fc005310 0x7fe7101f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 >> 172.21.15.83:0/2212639158 conn(0x7fe71006cd90 msgr2=0x7fe710075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:00.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 shutdown_connections 2024-05-17T22:00:00.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.604+0000 7fe717bec640 1 -- 172.21.15.83:0/2212639158 wait complete. 2024-05-17T22:00:00.616 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T22:00:00.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:00.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.764+0000 7f9fdefd6640 1 Processor -- start 2024-05-17T22:00:00.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.764+0000 7f9fdefd6640 1 -- start start 2024-05-17T22:00:00.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdefd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd8073540 0x7f9fd8168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdefd6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd815e790 con 0x7f9fd8144080 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdefd6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd815ea40 con 0x7f9fd80d1040 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdefd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd815ed20 con 0x7f9fd8073540 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9fd8144080 0x7f9fd8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50254/0 (socket says 172.21.15.83:50254) 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd7fff640 1 -- 172.21.15.83:0/2153270056 learned_addr learned my addr 172.21.15.83:0/2153270056 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdd573640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd8073540 0x7f9fd8168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8369255 0 0) 0x7f9fd815e790 con 0x7f9fd8144080 2024-05-17T22:00:00.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc0003530 con 0x7f9fd8144080 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3027312218 0 0) 0x7f9fc0003530 con 0x7f9fd8144080 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd8073540 msgr2=0x7f9fd8168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd8073540 0x7f9fd8168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 >> v1:172.21.15.155:6789/0 conn(0x7f9fd80d1040 legacy=0x7f9fd815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd815f000 con 0x7f9fd8144080 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9fcc001700 con 0x7f9fd8144080 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 >> v1:172.21.15.83:6789/0 conn(0x7f9fd8144080 legacy=0x7f9fd8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc0003a40 0x7f9fc0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0004810 0x7f9fc0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fc0005280 0x7f9fc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd815ed20 con 0x7f9fc0003a40 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc000bfc0 con 0x7f9fc0004810 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc000c1c0 con 0x7f9fc0005280 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc0003a40 0x7f9fc0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdd573640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fc0005280 0x7f9fc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 -- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fc0005280 msgr2=0x7f9fc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fc0005280 0x7f9fc000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fd7fff640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0004810 0x7f9fc0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 -- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0004810 msgr2=0x7f9fc0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0004810 0x7f9fc0004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdcd72640 1 -- 172.21.15.83:0/2153270056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc0003530 con 0x7f9fc0003a40 2024-05-17T22:00:00.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.768+0000 7f9fdd573640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fc0005280 0x7f9fc000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:00.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc0003a40 0x7f9fc0003eb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8001790 tx=0x7f9fc801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11c21a5975c2d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80013e0 con 0x7f9fc0003a40 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc80052d0 con 0x7f9fc0003a40 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fd77fe640 1 -- 172.21.15.83:0/2153270056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8003840 con 0x7f9fc0003a40 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 -- 172.21.15.83:0/2153270056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc0003a40 msgr2=0x7f9fc0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/2153270056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc0003a40 0x7f9fc0003eb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8001790 tx=0x7f9fc801c3c0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdd573640 1 -- 172.21.15.83:0/2153270056 reap_dead start 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 -- 172.21.15.83:0/2153270056 shutdown_connections 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 -- 172.21.15.83:0/2153270056 >> 172.21.15.83:0/2153270056 conn(0x7f9fd806cd90 msgr2=0x7f9fd80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 -- 172.21.15.83:0/2153270056 shutdown_connections 2024-05-17T22:00:00.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 -- 172.21.15.83:0/2153270056 wait complete. 2024-05-17T22:00:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.772+0000 7f9fdefd6640 1 Processor -- start 2024-05-17T22:00:00.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- start start 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0003a40 0x7f9fd813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd81f9610 0x7f9fd80b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd80736c0 con 0x7f9fd81f9200 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd8073540 con 0x7f9fc0003a40 2024-05-17T22:00:00.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd8070740 con 0x7f9fd81f9610 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47898/0 (socket says 172.21.15.83:47898) 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 -- 172.21.15.83:0/4135165733 learned_addr learned my addr 172.21.15.83:0/4135165733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdd573640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd81f9610 0x7f9fd80b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0003a40 0x7f9fd813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 -- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd81f9610 msgr2=0x7f9fd80b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd81f9610 0x7f9fd80b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 -- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0003a40 msgr2=0x7f9fd813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0003a40 0x7f9fd813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 -- 172.21.15.83:0/4135165733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd813e840 con 0x7f9fd81f9200 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd7fff640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d7a0 tx=0x7f9fcc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc019070 con 0x7f9fd81f9200 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0072d0 con 0x7f9fd81f9200 2024-05-17T22:00:00.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd813ed50 con 0x7f9fd81f9200 2024-05-17T22:00:00.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0047f0 con 0x7f9fd81f9200 2024-05-17T22:00:00.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd8140730 con 0x7f9fd81f9200 2024-05-17T22:00:00.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd8072e30 con 0x7f9fd81f9200 2024-05-17T22:00:00.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.776+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fcc01c070 con 0x7f9fd81f9200 2024-05-17T22:00:00.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.780+0000 7f9fd57fa640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 0x7f9fc4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:00.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.780+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc08eb60 con 0x7f9fd81f9200 2024-05-17T22:00:00.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.780+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 0x7f9fc4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:00.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.780+0000 7f9fdcd72640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 0x7f9fc4071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8007fd0 tx=0x7f9fc80011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:00.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.784+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9fcc05ba50 con 0x7f9fd81f9200 2024-05-17T22:00:00.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd80d0440 con 0x7f9fd81f9200 2024-05-17T22:00:00.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fd57fa640 1 -- 172.21.15.83:0/4135165733 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5184 (secure 0 0 0) 0x7f9fcc05b350 con 0x7f9fd81f9200 2024-05-17T22:00:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:00.957 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T21:59:59.970206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:00.957 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 msgr2=0x7f9fc4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 0x7f9fc4071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8007fd0 tx=0x7f9fc80011a0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 msgr2=0x7f9fd80b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d7a0 tx=0x7f9fcc00dc60 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 shutdown_connections 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fd81f9610 0x7f9fd80b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fd81f9200 0x7f9fd80b21b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9fc406f400 0x7f9fc4071910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 --2- 172.21.15.83:0/4135165733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9fc0003a40 0x7f9fd813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:00.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 >> 172.21.15.83:0/4135165733 conn(0x7f9fd806cd90 msgr2=0x7f9fd8147120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 shutdown_connections 2024-05-17T22:00:00.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:00.964+0000 7f9fdefd6640 1 -- 172.21.15.83:0/4135165733 wait complete. 2024-05-17T22:00:00.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:00:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.124+0000 7ff80d12c640 1 Processor -- start 2024-05-17T22:00:01.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.124+0000 7ff80d12c640 1 -- start start 2024-05-17T22:00:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8080d1040 0x7ff808144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff80815e080 0x7ff808072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 0x7ff808073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff80815ea40 con 0x7ff8080d1040 2024-05-17T22:00:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff80815ed20 con 0x7ff80815e080 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80d12c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff80815f000 con 0x7ff80815e490 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8080d1040 0x7ff808144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8080d1040 0x7ff808144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47908/0 (socket says 172.21.15.83:47908) 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 -- 172.21.15.83:0/431972178 learned_addr learned my addr 172.21.15.83:0/431972178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80659c640 1 --2- 172.21.15.83:0/431972178 >> v2:172.21.15.155:3300/0 conn(0x7ff80815e080 0x7ff808072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80759e640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 0x7ff808073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 -- 172.21.15.83:0/431972178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 msgr2=0x7ff808073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 0x7ff808073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 -- 172.21.15.83:0/431972178 >> v2:172.21.15.155:3300/0 conn(0x7ff80815e080 msgr2=0x7ff808072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 --2- 172.21.15.83:0/431972178 >> v2:172.21.15.155:3300/0 conn(0x7ff80815e080 0x7ff808072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff806d9d640 1 -- 172.21.15.83:0/431972178 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80806ef30 con 0x7ff8080d1040 2024-05-17T22:00:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.128+0000 7ff80759e640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 0x7ff808073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff806d9d640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8080d1040 0x7ff808144080 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff80815ef30 tx=0x7ff7f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19066277003bb6b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff805d9b640 1 -- 172.21.15.83:0/431972178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80a5780 con 0x7ff8080d1040 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff805d9b640 1 -- 172.21.15.83:0/431972178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f80a5920 con 0x7ff8080d1040 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff805d9b640 1 -- 172.21.15.83:0/431972178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80a5cf0 con 0x7ff8080d1040 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- 172.21.15.83:0/431972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8080d1040 msgr2=0x7ff808144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8080d1040 0x7ff808144080 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff80815ef30 tx=0x7ff7f801c9f0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- 172.21.15.83:0/431972178 shutdown_connections 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff80815e490 0x7ff808073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 --2- 172.21.15.83:0/431972178 >> v2:172.21.15.155:3300/0 conn(0x7ff80815e080 0x7ff808072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 --2- 172.21.15.83:0/431972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8080d1040 0x7ff808144080 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- 172.21.15.83:0/431972178 >> 172.21.15.83:0/431972178 conn(0x7ff80806cd90 msgr2=0x7ff8080b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- 172.21.15.83:0/431972178 shutdown_connections 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- 172.21.15.83:0/431972178 wait complete. 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 Processor -- start 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.132+0000 7ff80d12c640 1 -- start start 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8080d1040 0x7ff808140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff80815e080 0x7ff8081592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff808162260 con 0x7ff80815e490 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8081620e0 con 0x7ff80815e080 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff8081623e0 con 0x7ff8080d1040 2024-05-17T22:00:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47920/0 (socket says 172.21.15.83:47920) 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 -- 172.21.15.83:0/4110522198 learned_addr learned my addr 172.21.15.83:0/4110522198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80659c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff80815e080 0x7ff8081592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff806d9d640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8080d1040 0x7ff808140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 -- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8080d1040 msgr2=0x7ff808140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8080d1040 0x7ff808140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 -- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff80815e080 msgr2=0x7ff8081592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff80815e080 0x7ff8081592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 -- 172.21.15.83:0/4110522198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80813ca50 con 0x7ff80815e490 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80759e640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c5c0 tx=0x7ff80000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800015070 con 0x7ff80815e490 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff800011820 con 0x7ff80815e490 2024-05-17T22:00:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff808159830 con 0x7ff80815e490 2024-05-17T22:00:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800011c10 con 0x7ff80815e490 2024-05-17T22:00:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80813d240 con 0x7ff80815e490 2024-05-17T22:00:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff808144080 con 0x7ff80815e490 2024-05-17T22:00:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff800018070 con 0x7ff80815e490 2024-05-17T22:00:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 0x7ff7f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7ff80008f670 con 0x7ff80815e490 2024-05-17T22:00:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff806d9d640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 0x7ff7f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.136+0000 7ff806d9d640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 0x7ff7f4071910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80ae000 tx=0x7ff7f80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.140+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff80005c5d0 con 0x7ff80815e490 2024-05-17T22:00:01.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.288+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8080b3000 con 0x7ff80815e490 2024-05-17T22:00:01.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.292+0000 7ff7ef7fe640 1 -- 172.21.15.83:0/4110522198 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16641 (secure 0 0 0) 0x7ff80005bed0 con 0x7ff80815e490 2024-05-17T22:00:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:01.286 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:58.567137+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:59:54.592647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-17T21:59:55.893117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:00:01.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 msgr2=0x7ff7f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 0x7ff7f4071910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80ae000 tx=0x7ff7f80a7070 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 msgr2=0x7ff80813b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c5c0 tx=0x7ff80000ca80 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 shutdown_connections 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff80815e490 0x7ff80813b7b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff7f406f400 0x7ff7f4071910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff80815e080 0x7ff8081592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 --2- 172.21.15.83:0/4110522198 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8080d1040 0x7ff808140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 >> 172.21.15.83:0/4110522198 conn(0x7ff80806cd90 msgr2=0x7ff808075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 shutdown_connections 2024-05-17T22:00:01.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.296+0000 7ff80d12c640 1 -- 172.21.15.83:0/4110522198 wait complete. 2024-05-17T22:00:01.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:00:01.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:00:01.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 Processor -- start 2024-05-17T22:00:01.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 -- start start 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f060c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c1465d0 con 0x7f060c1605a0 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f060c070920 con 0x7f060c1461c0 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f0611c27640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f060c06ef30 con 0x7f060c06ffd0 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f060c1605a0 0x7f060c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50260/0 (socket says 172.21.15.83:50260) 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060bfff640 1 -- 172.21.15.83:0/1847403273 learned_addr learned my addr 172.21.15.83:0/1847403273 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060affd640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f060c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827729922 0 0) 0x7f060c1465d0 con 0x7f060c1605a0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05f4003530 con 0x7f060c1605a0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2117735142 0 0) 0x7f05f4003530 con 0x7f060c1605a0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 msgr2=0x7f060c0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f060c0703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 >> v1:172.21.15.155:6789/0 conn(0x7f060c1461c0 legacy=0x7f060c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c06f210 con 0x7f060c1605a0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f05fc001a80 con 0x7f060c1605a0 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 >> v1:172.21.15.83:6789/0 conn(0x7f060c1605a0 legacy=0x7f060c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f05f4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4004030 0x7f05f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f4004d50 0x7f05f4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f060c06ef30 con 0x7f05f4004d50 2024-05-17T22:00:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05f400bc10 con 0x7f05f4004030 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f05f400be10 con 0x7f060c06ffd0 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060bfff640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f4004d50 0x7f05f4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060affd640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4004030 0x7f05f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.460+0000 7f060b7fe640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f05f4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 -- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 msgr2=0x7f05f4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c06ffd0 0x7f05f4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 -- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4004030 msgr2=0x7f05f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4004030 0x7f05f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 -- 172.21.15.83:0/1847403273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f4003530 con 0x7f05f4004d50 2024-05-17T22:00:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060affd640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4004030 0x7f05f4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f4004d50 0x7f05f4005220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f060c071250 tx=0x7f05fc01d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba346944a5124e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00a4a0 con 0x7f05f4004d50 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc0ad650 con 0x7f05f4004d50 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060a7fc640 1 -- 172.21.15.83:0/1847403273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0ada40 con 0x7f05f4004d50 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 -- 172.21.15.83:0/1847403273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f4004d50 msgr2=0x7f05f4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 --2- 172.21.15.83:0/1847403273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f4004d50 0x7f05f4005220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f060c071250 tx=0x7f05fc01d280 comp rx=0 tx=0).stop 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f060bfff640 1 -- 172.21.15.83:0/1847403273 reap_dead start 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 -- 172.21.15.83:0/1847403273 shutdown_connections 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 -- 172.21.15.83:0/1847403273 >> 172.21.15.83:0/1847403273 conn(0x7f060c06cd90 msgr2=0x7f060c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 -- 172.21.15.83:0/1847403273 shutdown_connections 2024-05-17T22:00:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.464+0000 7f0611c27640 1 -- 172.21.15.83:0/1847403273 wait complete. 2024-05-17T22:00:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 Processor -- start 2024-05-17T22:00:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- start start 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c13efc0 0x7f060c0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c0b2270 0x7f060c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f060c06fab0 con 0x7f060c15b910 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f060c06f7d0 con 0x7f060c0b2270 2024-05-17T22:00:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f060c15e880 con 0x7f060c13efc0 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47940/0 (socket says 172.21.15.83:47940) 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 -- 172.21.15.83:0/1735410073 learned_addr learned my addr 172.21.15.83:0/1735410073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060bfff640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c0b2270 0x7f060c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 -- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c13efc0 msgr2=0x7f060c0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c13efc0 0x7f060c0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 -- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c0b2270 msgr2=0x7f060c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c0b2270 0x7f060c0b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 -- 172.21.15.83:0/1735410073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c0b2bc0 con 0x7f060c15b910 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f060affd640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0600009fd0 tx=0x7f0600011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600004220 con 0x7f060c15b910 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06000043c0 con 0x7f060c15b910 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c0aee60 con 0x7f060c15b910 2024-05-17T22:00:01.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060000f9e0 con 0x7f060c15b910 2024-05-17T22:00:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.468+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c0af3a0 con 0x7f060c15b910 2024-05-17T22:00:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f060c0703e0 con 0x7f060c15b910 2024-05-17T22:00:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f060000fb80 con 0x7f060c15b910 2024-05-17T22:00:01.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f05f3fff640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 0x7f05f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f060008f930 con 0x7f060c15b910 2024-05-17T22:00:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f060b7fe640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 0x7f05f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.472+0000 7f060b7fe640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 0x7f05f8071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0604007af0 tx=0x7f0604009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.476+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f060c0b2bc0 con 0x7f060c15b910 2024-05-17T22:00:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.632+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f060c0b41a0 con 0x7f05f806f400 2024-05-17T22:00:01.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.644+0000 7f05f3fff640 1 -- 172.21.15.83:0/1735410073 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f060c0b41a0 con 0x7f05f806f400 2024-05-17T22:00:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:01.643 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:00:01.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 msgr2=0x7f05f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 0x7f05f8071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0604007af0 tx=0x7f0604009040 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 msgr2=0x7f060c15bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0600009fd0 tx=0x7f0600011b30 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 shutdown_connections 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f05f806f400 0x7f05f8071910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c0b2270 0x7f060c0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060c15b910 0x7f060c15bd20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 --2- 172.21.15.83:0/1735410073 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c13efc0 0x7f060c0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 >> 172.21.15.83:0/1735410073 conn(0x7f060c06cd90 msgr2=0x7f060c0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 shutdown_connections 2024-05-17T22:00:01.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.648+0000 7f0611c27640 1 -- 172.21.15.83:0/1735410073 wait complete. 2024-05-17T22:00:01.661 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-05-17T22:00:01.018843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239232,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735488,"statfs":{"total":773094113280,"available":772849139712,"internally_reserved":0,"allocated":6529024,"data_stored":3359020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.503732"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582773+0000","last_change":"2024-05-17T21:59:57.593439+0000","last_active":"2024-05-17T21:59:58.582773+0000","last_peered":"2024-05-17T21:59:58.582773+0000","last_clean":"2024-05-17T21:59:58.582773+0000","last_became_active":"2024-05-17T21:59:57.592898+0000","last_became_peered":"2024-05-17T21:59:57.592898+0000","last_unstale":"2024-05-17T21:59:58.582773+0000","last_undegraded":"2024-05-17T21:59:58.582773+0000","last_fullsized":"2024-05-17T21:59:58.582773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:05:38.810696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949783+0000","last_change":"2024-05-17T21:59:57.577584+0000","last_active":"2024-05-17T21:59:58.949783+0000","last_peered":"2024-05-17T21:59:58.949783+0000","last_clean":"2024-05-17T21:59:58.949783+0000","last_became_active":"2024-05-17T21:59:57.577096+0000","last_became_peered":"2024-05-17T21:59:57.577096+0000","last_unstale":"2024-05-17T21:59:58.949783+0000","last_undegraded":"2024-05-17T21:59:58.949783+0000","last_fullsized":"2024-05-17T21:59:58.949783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:46:39.087363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946387+0000","last_change":"2024-05-17T21:59:57.593222+0000","last_active":"2024-05-17T21:59:58.946387+0000","last_peered":"2024-05-17T21:59:58.946387+0000","last_clean":"2024-05-17T21:59:58.946387+0000","last_became_active":"2024-05-17T21:59:57.592399+0000","last_became_peered":"2024-05-17T21:59:57.592399+0000","last_unstale":"2024-05-17T21:59:58.946387+0000","last_undegraded":"2024-05-17T21:59:58.946387+0000","last_fullsized":"2024-05-17T21:59:58.946387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588433+0000","last_change":"2024-05-17T21:59:57.606023+0000","last_active":"2024-05-17T21:59:58.588433+0000","last_peered":"2024-05-17T21:59:58.588433+0000","last_clean":"2024-05-17T21:59:58.588433+0000","last_became_active":"2024-05-17T21:59:57.604799+0000","last_became_peered":"2024-05-17T21:59:57.604799+0000","last_unstale":"2024-05-17T21:59:58.588433+0000","last_undegraded":"2024-05-17T21:59:58.588433+0000","last_fullsized":"2024-05-17T21:59:58.588433+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582692+0000","last_change":"2024-05-17T21:59:57.606260+0000","last_active":"2024-05-17T21:59:58.582692+0000","last_peered":"2024-05-17T21:59:58.582692+0000","last_clean":"2024-05-17T21:59:58.582692+0000","last_became_active":"2024-05-17T21:59:57.605949+0000","last_became_peered":"2024-05-17T21:59:57.605949+0000","last_unstale":"2024-05-17T21:59:58.582692+0000","last_undegraded":"2024-05-17T21:59:58.582692+0000","last_fullsized":"2024-05-17T21:59:58.582692+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:25.921921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946309+0000","last_change":"2024-05-17T21:59:57.576495+0000","last_active":"2024-05-17T21:59:58.946309+0000","last_peered":"2024-05-17T21:59:58.946309+0000","last_clean":"2024-05-17T21:59:58.946309+0000","last_became_active":"2024-05-17T21:59:57.575977+0000","last_became_peered":"2024-05-17T21:59:57.575977+0000","last_unstale":"2024-05-17T21:59:58.946309+0000","last_undegraded":"2024-05-17T21:59:58.946309+0000","last_fullsized":"2024-05-17T21:59:58.946309+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585006+0000","last_change":"2024-05-17T21:59:57.608621+0000","last_active":"2024-05-17T21:59:58.585006+0000","last_peered":"2024-05-17T21:59:58.585006+0000","last_clean":"2024-05-17T21:59:58.585006+0000","last_became_active":"2024-05-17T21:59:57.607615+0000","last_became_peered":"2024-05-17T21:59:57.607615+0000","last_unstale":"2024-05-17T21:59:58.585006+0000","last_undegraded":"2024-05-17T21:59:58.585006+0000","last_fullsized":"2024-05-17T21:59:58.585006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:26.066389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949008+0000","last_change":"2024-05-17T21:59:57.562290+0000","last_active":"2024-05-17T21:59:58.949008+0000","last_peered":"2024-05-17T21:59:58.949008+0000","last_clean":"2024-05-17T21:59:58.949008+0000","last_became_active":"2024-05-17T21:59:57.561977+0000","last_became_peered":"2024-05-17T21:59:57.561977+0000","last_unstale":"2024-05-17T21:59:58.949008+0000","last_undegraded":"2024-05-17T21:59:58.949008+0000","last_fullsized":"2024-05-17T21:59:58.949008+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:17:48.998895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585910+0000","last_change":"2024-05-17T21:59:57.589058+0000","last_active":"2024-05-17T21:59:58.585910+0000","last_peered":"2024-05-17T21:59:58.585910+0000","last_clean":"2024-05-17T21:59:58.585910+0000","last_became_active":"2024-05-17T21:59:57.588268+0000","last_became_peered":"2024-05-17T21:59:57.588268+0000","last_unstale":"2024-05-17T21:59:58.585910+0000","last_undegraded":"2024-05-17T21:59:58.585910+0000","last_fullsized":"2024-05-17T21:59:58.585910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:17:23.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947092+0000","last_change":"2024-05-17T21:59:57.589623+0000","last_active":"2024-05-17T21:59:58.947092+0000","last_peered":"2024-05-17T21:59:58.947092+0000","last_clean":"2024-05-17T21:59:58.947092+0000","last_became_active":"2024-05-17T21:59:57.589340+0000","last_became_peered":"2024-05-17T21:59:57.589340+0000","last_unstale":"2024-05-17T21:59:58.947092+0000","last_undegraded":"2024-05-17T21:59:58.947092+0000","last_fullsized":"2024-05-17T21:59:58.947092+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:55.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946385+0000","last_change":"2024-05-17T21:59:57.593337+0000","last_active":"2024-05-17T21:59:58.946385+0000","last_peered":"2024-05-17T21:59:58.946385+0000","last_clean":"2024-05-17T21:59:58.946385+0000","last_became_active":"2024-05-17T21:59:57.593001+0000","last_became_peered":"2024-05-17T21:59:57.593001+0000","last_unstale":"2024-05-17T21:59:58.946385+0000","last_undegraded":"2024-05-17T21:59:58.946385+0000","last_fullsized":"2024-05-17T21:59:58.946385+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:13.550814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588987+0000","last_change":"2024-05-17T21:59:57.619088+0000","last_active":"2024-05-17T21:59:58.588987+0000","last_peered":"2024-05-17T21:59:58.588987+0000","last_clean":"2024-05-17T21:59:58.588987+0000","last_became_active":"2024-05-17T21:59:57.618364+0000","last_became_peered":"2024-05-17T21:59:57.618364+0000","last_unstale":"2024-05-17T21:59:58.588987+0000","last_undegraded":"2024-05-17T21:59:58.588987+0000","last_fullsized":"2024-05-17T21:59:58.588987+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946700+0000","last_change":"2024-05-17T21:59:57.588866+0000","last_active":"2024-05-17T21:59:58.946700+0000","last_peered":"2024-05-17T21:59:58.946700+0000","last_clean":"2024-05-17T21:59:58.946700+0000","last_became_active":"2024-05-17T21:59:57.587823+0000","last_became_peered":"2024-05-17T21:59:57.587823+0000","last_unstale":"2024-05-17T21:59:58.946700+0000","last_undegraded":"2024-05-17T21:59:58.946700+0000","last_fullsized":"2024-05-17T21:59:58.946700+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:17:26.370984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585902+0000","last_change":"2024-05-17T21:59:57.579238+0000","last_active":"2024-05-17T21:59:58.585902+0000","last_peered":"2024-05-17T21:59:58.585902+0000","last_clean":"2024-05-17T21:59:58.585902+0000","last_became_active":"2024-05-17T21:59:57.578428+0000","last_became_peered":"2024-05-17T21:59:57.578428+0000","last_unstale":"2024-05-17T21:59:58.585902+0000","last_undegraded":"2024-05-17T21:59:58.585902+0000","last_fullsized":"2024-05-17T21:59:58.585902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:09.989753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585472+0000","last_change":"2024-05-17T21:59:57.568112+0000","last_active":"2024-05-17T21:59:58.585472+0000","last_peered":"2024-05-17T21:59:58.585472+0000","last_clean":"2024-05-17T21:59:58.585472+0000","last_became_active":"2024-05-17T21:59:57.567185+0000","last_became_peered":"2024-05-17T21:59:57.567185+0000","last_unstale":"2024-05-17T21:59:58.585472+0000","last_undegraded":"2024-05-17T21:59:58.585472+0000","last_fullsized":"2024-05-17T21:59:58.585472+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:53.830481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948066+0000","last_change":"2024-05-17T21:59:57.590738+0000","last_active":"2024-05-17T21:59:58.948066+0000","last_peered":"2024-05-17T21:59:58.948066+0000","last_clean":"2024-05-17T21:59:58.948066+0000","last_became_active":"2024-05-17T21:59:57.590063+0000","last_became_peered":"2024-05-17T21:59:57.590063+0000","last_unstale":"2024-05-17T21:59:58.948066+0000","last_undegraded":"2024-05-17T21:59:58.948066+0000","last_fullsized":"2024-05-17T21:59:58.948066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:51.977412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949711+0000","last_change":"2024-05-17T21:59:57.591365+0000","last_active":"2024-05-17T21:59:58.949711+0000","last_peered":"2024-05-17T21:59:58.949711+0000","last_clean":"2024-05-17T21:59:58.949711+0000","last_became_active":"2024-05-17T21:59:57.590811+0000","last_became_peered":"2024-05-17T21:59:57.590811+0000","last_unstale":"2024-05-17T21:59:58.949711+0000","last_undegraded":"2024-05-17T21:59:58.949711+0000","last_fullsized":"2024-05-17T21:59:58.949711+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:43:58.576535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946929+0000","last_change":"2024-05-17T21:59:57.591663+0000","last_active":"2024-05-17T21:59:58.946929+0000","last_peered":"2024-05-17T21:59:58.946929+0000","last_clean":"2024-05-17T21:59:58.946929+0000","last_became_active":"2024-05-17T21:59:57.588824+0000","last_became_peered":"2024-05-17T21:59:57.588824+0000","last_unstale":"2024-05-17T21:59:58.946929+0000","last_undegraded":"2024-05-17T21:59:58.946929+0000","last_fullsized":"2024-05-17T21:59:58.946929+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:05.324699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582848+0000","last_change":"2024-05-17T21:59:57.593258+0000","last_active":"2024-05-17T21:59:58.582848+0000","last_peered":"2024-05-17T21:59:58.582848+0000","last_clean":"2024-05-17T21:59:58.582848+0000","last_became_active":"2024-05-17T21:59:57.592984+0000","last_became_peered":"2024-05-17T21:59:57.592984+0000","last_unstale":"2024-05-17T21:59:58.582848+0000","last_undegraded":"2024-05-17T21:59:58.582848+0000","last_fullsized":"2024-05-17T21:59:58.582848+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:00:44.260853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949079+0000","last_change":"2024-05-17T21:59:57.577350+0000","last_active":"2024-05-17T21:59:58.949079+0000","last_peered":"2024-05-17T21:59:58.949079+0000","last_clean":"2024-05-17T21:59:58.949079+0000","last_became_active":"2024-05-17T21:59:57.577013+0000","last_became_peered":"2024-05-17T21:59:57.577013+0000","last_unstale":"2024-05-17T21:59:58.949079+0000","last_undegraded":"2024-05-17T21:59:58.949079+0000","last_fullsized":"2024-05-17T21:59:58.949079+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:10.214313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946795+0000","last_change":"2024-05-17T21:59:57.613709+0000","last_active":"2024-05-17T21:59:58.946795+0000","last_peered":"2024-05-17T21:59:58.946795+0000","last_clean":"2024-05-17T21:59:58.946795+0000","last_became_active":"2024-05-17T21:59:57.613085+0000","last_became_peered":"2024-05-17T21:59:57.613085+0000","last_unstale":"2024-05-17T21:59:58.946795+0000","last_undegraded":"2024-05-17T21:59:58.946795+0000","last_fullsized":"2024-05-17T21:59:58.946795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948731+0000","last_change":"2024-05-17T21:59:57.582259+0000","last_active":"2024-05-17T21:59:58.948731+0000","last_peered":"2024-05-17T21:59:58.948731+0000","last_clean":"2024-05-17T21:59:58.948731+0000","last_became_active":"2024-05-17T21:59:57.581940+0000","last_became_peered":"2024-05-17T21:59:57.581940+0000","last_unstale":"2024-05-17T21:59:58.948731+0000","last_undegraded":"2024-05-17T21:59:58.948731+0000","last_fullsized":"2024-05-17T21:59:58.948731+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:41:46.064289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945940+0000","last_change":"2024-05-17T21:59:57.594999+0000","last_active":"2024-05-17T21:59:58.945940+0000","last_peered":"2024-05-17T21:59:58.945940+0000","last_clean":"2024-05-17T21:59:58.945940+0000","last_became_active":"2024-05-17T21:59:57.594157+0000","last_became_peered":"2024-05-17T21:59:57.594157+0000","last_unstale":"2024-05-17T21:59:58.945940+0000","last_undegraded":"2024-05-17T21:59:58.945940+0000","last_fullsized":"2024-05-17T21:59:58.945940+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:59:56.429573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947589+0000","last_change":"2024-05-17T21:59:57.591965+0000","last_active":"2024-05-17T21:59:58.947589+0000","last_peered":"2024-05-17T21:59:58.947589+0000","last_clean":"2024-05-17T21:59:58.947589+0000","last_became_active":"2024-05-17T21:59:57.591040+0000","last_became_peered":"2024-05-17T21:59:57.591040+0000","last_unstale":"2024-05-17T21:59:58.947589+0000","last_undegraded":"2024-05-17T21:59:58.947589+0000","last_fullsized":"2024-05-17T21:59:58.947589+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:37.805311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582910+0000","last_change":"2024-05-17T21:59:57.576986+0000","last_active":"2024-05-17T21:59:58.582910+0000","last_peered":"2024-05-17T21:59:58.582910+0000","last_clean":"2024-05-17T21:59:58.582910+0000","last_became_active":"2024-05-17T21:59:57.576482+0000","last_became_peered":"2024-05-17T21:59:57.576482+0000","last_unstale":"2024-05-17T21:59:58.582910+0000","last_undegraded":"2024-05-17T21:59:58.582910+0000","last_fullsized":"2024-05-17T21:59:58.582910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:07:07.164818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.589004+0000","last_change":"2024-05-17T21:59:57.608774+0000","last_active":"2024-05-17T21:59:58.589004+0000","last_peered":"2024-05-17T21:59:58.589004+0000","last_clean":"2024-05-17T21:59:58.589004+0000","last_became_active":"2024-05-17T21:59:57.605813+0000","last_became_peered":"2024-05-17T21:59:57.605813+0000","last_unstale":"2024-05-17T21:59:58.589004+0000","last_undegraded":"2024-05-17T21:59:58.589004+0000","last_fullsized":"2024-05-17T21:59:58.589004+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945860+0000","last_change":"2024-05-17T21:59:57.587766+0000","last_active":"2024-05-17T21:59:58.945860+0000","last_peered":"2024-05-17T21:59:58.945860+0000","last_clean":"2024-05-17T21:59:58.945860+0000","last_became_active":"2024-05-17T21:59:57.587418+0000","last_became_peered":"2024-05-17T21:59:57.587418+0000","last_unstale":"2024-05-17T21:59:58.945860+0000","last_undegraded":"2024-05-17T21:59:58.945860+0000","last_fullsized":"2024-05-17T21:59:58.945860+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582686+0000","last_change":"2024-05-17T21:59:57.609281+0000","last_active":"2024-05-17T21:59:58.582686+0000","last_peered":"2024-05-17T21:59:58.582686+0000","last_clean":"2024-05-17T21:59:58.582686+0000","last_became_active":"2024-05-17T21:59:57.608943+0000","last_became_peered":"2024-05-17T21:59:57.608943+0000","last_unstale":"2024-05-17T21:59:58.582686+0000","last_undegraded":"2024-05-17T21:59:58.582686+0000","last_fullsized":"2024-05-17T21:59:58.582686+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:05.104866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949776+0000","last_change":"2024-05-17T21:59:57.606178+0000","last_active":"2024-05-17T21:59:58.949776+0000","last_peered":"2024-05-17T21:59:58.949776+0000","last_clean":"2024-05-17T21:59:58.949776+0000","last_became_active":"2024-05-17T21:59:57.605656+0000","last_became_peered":"2024-05-17T21:59:57.605656+0000","last_unstale":"2024-05-17T21:59:58.949776+0000","last_undegraded":"2024-05-17T21:59:58.949776+0000","last_fullsized":"2024-05-17T21:59:58.949776+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:40:47.421281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946893+0000","last_change":"2024-05-17T21:59:57.587667+0000","last_active":"2024-05-17T21:59:58.946893+0000","last_peered":"2024-05-17T21:59:58.946893+0000","last_clean":"2024-05-17T21:59:58.946893+0000","last_became_active":"2024-05-17T21:59:57.586806+0000","last_became_peered":"2024-05-17T21:59:57.586806+0000","last_unstale":"2024-05-17T21:59:58.946893+0000","last_undegraded":"2024-05-17T21:59:58.946893+0000","last_fullsized":"2024-05-17T21:59:58.946893+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582523+0000","last_change":"2024-05-17T21:59:57.574009+0000","last_active":"2024-05-17T21:59:58.582523+0000","last_peered":"2024-05-17T21:59:58.582523+0000","last_clean":"2024-05-17T21:59:58.582523+0000","last_became_active":"2024-05-17T21:59:57.573707+0000","last_became_peered":"2024-05-17T21:59:57.573707+0000","last_unstale":"2024-05-17T21:59:58.582523+0000","last_undegraded":"2024-05-17T21:59:58.582523+0000","last_fullsized":"2024-05-17T21:59:58.582523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:38:34.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.944533+0000","last_change":"2024-05-17T21:59:57.593736+0000","last_active":"2024-05-17T21:59:58.944533+0000","last_peered":"2024-05-17T21:59:58.944533+0000","last_clean":"2024-05-17T21:59:58.944533+0000","last_became_active":"2024-05-17T21:59:57.593444+0000","last_became_peered":"2024-05-17T21:59:57.593444+0000","last_unstale":"2024-05-17T21:59:58.944533+0000","last_undegraded":"2024-05-17T21:59:58.944533+0000","last_fullsized":"2024-05-17T21:59:58.944533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946903+0000","last_change":"2024-05-17T21:59:57.615270+0000","last_active":"2024-05-17T21:59:58.946903+0000","last_peered":"2024-05-17T21:59:58.946903+0000","last_clean":"2024-05-17T21:59:58.946903+0000","last_became_active":"2024-05-17T21:59:57.614820+0000","last_became_peered":"2024-05-17T21:59:57.614820+0000","last_unstale":"2024-05-17T21:59:58.946903+0000","last_undegraded":"2024-05-17T21:59:58.946903+0000","last_fullsized":"2024-05-17T21:59:58.946903+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:05.324699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946037+0000","last_change":"2024-05-17T21:59:56.552443+0000","last_active":"2024-05-17T21:59:58.946037+0000","last_peered":"2024-05-17T21:59:58.946037+0000","last_clean":"2024-05-17T21:59:58.946037+0000","last_became_active":"2024-05-17T21:59:56.552010+0000","last_became_peered":"2024-05-17T21:59:56.552010+0000","last_unstale":"2024-05-17T21:59:58.946037+0000","last_undegraded":"2024-05-17T21:59:58.946037+0000","last_fullsized":"2024-05-17T21:59:58.946037+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947864+0000","last_change":"2024-05-17T21:59:57.572902+0000","last_active":"2024-05-17T21:59:58.947864+0000","last_peered":"2024-05-17T21:59:58.947864+0000","last_clean":"2024-05-17T21:59:58.947864+0000","last_became_active":"2024-05-17T21:59:57.572552+0000","last_became_peered":"2024-05-17T21:59:57.572552+0000","last_unstale":"2024-05-17T21:59:58.947864+0000","last_undegraded":"2024-05-17T21:59:58.947864+0000","last_fullsized":"2024-05-17T21:59:58.947864+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:15.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949957+0000","last_change":"2024-05-17T21:59:56.549351+0000","last_active":"2024-05-17T21:59:58.949957+0000","last_peered":"2024-05-17T21:59:58.949957+0000","last_clean":"2024-05-17T21:59:58.949957+0000","last_became_active":"2024-05-17T21:59:56.548950+0000","last_became_peered":"2024-05-17T21:59:56.548950+0000","last_unstale":"2024-05-17T21:59:58.949957+0000","last_undegraded":"2024-05-17T21:59:58.949957+0000","last_fullsized":"2024-05-17T21:59:58.949957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:29.210821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945125+0000","last_change":"2024-05-17T21:59:57.572350+0000","last_active":"2024-05-17T21:59:58.945125+0000","last_peered":"2024-05-17T21:59:58.945125+0000","last_clean":"2024-05-17T21:59:58.945125+0000","last_became_active":"2024-05-17T21:59:57.572022+0000","last_became_peered":"2024-05-17T21:59:57.572022+0000","last_unstale":"2024-05-17T21:59:58.945125+0000","last_undegraded":"2024-05-17T21:59:58.945125+0000","last_fullsized":"2024-05-17T21:59:58.945125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946425+0000","last_change":"2024-05-17T21:59:56.547759+0000","last_active":"2024-05-17T21:59:58.946425+0000","last_peered":"2024-05-17T21:59:58.946425+0000","last_clean":"2024-05-17T21:59:58.946425+0000","last_became_active":"2024-05-17T21:59:56.547230+0000","last_became_peered":"2024-05-17T21:59:56.547230+0000","last_unstale":"2024-05-17T21:59:58.946425+0000","last_undegraded":"2024-05-17T21:59:58.946425+0000","last_fullsized":"2024-05-17T21:59:58.946425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:54.834567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946032+0000","last_change":"2024-05-17T21:59:57.607085+0000","last_active":"2024-05-17T21:59:58.946032+0000","last_peered":"2024-05-17T21:59:58.946032+0000","last_clean":"2024-05-17T21:59:58.946032+0000","last_became_active":"2024-05-17T21:59:57.606756+0000","last_became_peered":"2024-05-17T21:59:57.606756+0000","last_unstale":"2024-05-17T21:59:58.946032+0000","last_undegraded":"2024-05-17T21:59:58.946032+0000","last_fullsized":"2024-05-17T21:59:58.946032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588814+0000","last_change":"2024-05-17T21:59:56.551338+0000","last_active":"2024-05-17T21:59:58.588814+0000","last_peered":"2024-05-17T21:59:58.588814+0000","last_clean":"2024-05-17T21:59:58.588814+0000","last_became_active":"2024-05-17T21:59:56.550399+0000","last_became_peered":"2024-05-17T21:59:56.550399+0000","last_unstale":"2024-05-17T21:59:58.588814+0000","last_undegraded":"2024-05-17T21:59:58.588814+0000","last_fullsized":"2024-05-17T21:59:58.588814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:54.834567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948794+0000","last_change":"2024-05-17T21:59:57.581922+0000","last_active":"2024-05-17T21:59:58.948794+0000","last_peered":"2024-05-17T21:59:58.948794+0000","last_clean":"2024-05-17T21:59:58.948794+0000","last_became_active":"2024-05-17T21:59:57.581584+0000","last_became_peered":"2024-05-17T21:59:57.581584+0000","last_unstale":"2024-05-17T21:59:58.948794+0000","last_undegraded":"2024-05-17T21:59:58.948794+0000","last_fullsized":"2024-05-17T21:59:58.948794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:34:30.351774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585090+0000","last_change":"2024-05-17T21:59:56.561217+0000","last_active":"2024-05-17T21:59:58.585090+0000","last_peered":"2024-05-17T21:59:58.585090+0000","last_clean":"2024-05-17T21:59:58.585090+0000","last_became_active":"2024-05-17T21:59:56.560418+0000","last_became_peered":"2024-05-17T21:59:56.560418+0000","last_unstale":"2024-05-17T21:59:58.585090+0000","last_undegraded":"2024-05-17T21:59:58.585090+0000","last_fullsized":"2024-05-17T21:59:58.585090+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949628+0000","last_change":"2024-05-17T21:59:57.581796+0000","last_active":"2024-05-17T21:59:58.949628+0000","last_peered":"2024-05-17T21:59:58.949628+0000","last_clean":"2024-05-17T21:59:58.949628+0000","last_became_active":"2024-05-17T21:59:57.577817+0000","last_became_peered":"2024-05-17T21:59:57.577817+0000","last_unstale":"2024-05-17T21:59:58.949628+0000","last_undegraded":"2024-05-17T21:59:58.949628+0000","last_fullsized":"2024-05-17T21:59:58.949628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:31.320505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947349+0000","last_change":"2024-05-17T21:59:56.570765+0000","last_active":"2024-05-17T21:59:58.947349+0000","last_peered":"2024-05-17T21:59:58.947349+0000","last_clean":"2024-05-17T21:59:58.947349+0000","last_became_active":"2024-05-17T21:59:56.570307+0000","last_became_peered":"2024-05-17T21:59:56.570307+0000","last_unstale":"2024-05-17T21:59:58.947349+0000","last_undegraded":"2024-05-17T21:59:58.947349+0000","last_fullsized":"2024-05-17T21:59:58.947349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947212+0000","last_change":"2024-05-17T21:59:57.617436+0000","last_active":"2024-05-17T21:59:58.947212+0000","last_peered":"2024-05-17T21:59:58.947212+0000","last_clean":"2024-05-17T21:59:58.947212+0000","last_became_active":"2024-05-17T21:59:57.616638+0000","last_became_peered":"2024-05-17T21:59:57.616638+0000","last_unstale":"2024-05-17T21:59:58.947212+0000","last_undegraded":"2024-05-17T21:59:58.947212+0000","last_fullsized":"2024-05-17T21:59:58.947212+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:12:53.321187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587551+0000","last_change":"2024-05-17T21:59:56.562496+0000","last_active":"2024-05-17T21:59:58.587551+0000","last_peered":"2024-05-17T21:59:58.587551+0000","last_clean":"2024-05-17T21:59:58.587551+0000","last_became_active":"2024-05-17T21:59:56.561630+0000","last_became_peered":"2024-05-17T21:59:56.561630+0000","last_unstale":"2024-05-17T21:59:58.587551+0000","last_undegraded":"2024-05-17T21:59:58.587551+0000","last_fullsized":"2024-05-17T21:59:58.587551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948922+0000","last_change":"2024-05-17T21:59:57.591717+0000","last_active":"2024-05-17T21:59:58.948922+0000","last_peered":"2024-05-17T21:59:58.948922+0000","last_clean":"2024-05-17T21:59:58.948922+0000","last_became_active":"2024-05-17T21:59:57.591390+0000","last_became_peered":"2024-05-17T21:59:57.591390+0000","last_unstale":"2024-05-17T21:59:58.948922+0000","last_undegraded":"2024-05-17T21:59:58.948922+0000","last_fullsized":"2024-05-17T21:59:58.948922+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:43.381519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.584964+0000","last_change":"2024-05-17T21:59:56.557893+0000","last_active":"2024-05-17T21:59:58.584964+0000","last_peered":"2024-05-17T21:59:58.584964+0000","last_clean":"2024-05-17T21:59:58.584964+0000","last_became_active":"2024-05-17T21:59:56.557057+0000","last_became_peered":"2024-05-17T21:59:56.557057+0000","last_unstale":"2024-05-17T21:59:58.584964+0000","last_undegraded":"2024-05-17T21:59:58.584964+0000","last_fullsized":"2024-05-17T21:59:58.584964+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:39:34.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585813+0000","last_change":"2024-05-17T21:59:57.584253+0000","last_active":"2024-05-17T21:59:58.585813+0000","last_peered":"2024-05-17T21:59:58.585813+0000","last_clean":"2024-05-17T21:59:58.585813+0000","last_became_active":"2024-05-17T21:59:57.583170+0000","last_became_peered":"2024-05-17T21:59:57.583170+0000","last_unstale":"2024-05-17T21:59:58.585813+0000","last_undegraded":"2024-05-17T21:59:58.585813+0000","last_fullsized":"2024-05-17T21:59:58.585813+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:43:54.750624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947608+0000","last_change":"2024-05-17T21:59:56.564800+0000","last_active":"2024-05-17T21:59:58.947608+0000","last_peered":"2024-05-17T21:59:58.947608+0000","last_clean":"2024-05-17T21:59:58.947608+0000","last_became_active":"2024-05-17T21:59:56.563934+0000","last_became_peered":"2024-05-17T21:59:56.563934+0000","last_unstale":"2024-05-17T21:59:58.947608+0000","last_undegraded":"2024-05-17T21:59:58.947608+0000","last_fullsized":"2024-05-17T21:59:58.947608+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947783+0000","last_change":"2024-05-17T21:59:57.612674+0000","last_active":"2024-05-17T21:59:58.947783+0000","last_peered":"2024-05-17T21:59:58.947783+0000","last_clean":"2024-05-17T21:59:58.947783+0000","last_became_active":"2024-05-17T21:59:57.612004+0000","last_became_peered":"2024-05-17T21:59:57.612004+0000","last_unstale":"2024-05-17T21:59:58.947783+0000","last_undegraded":"2024-05-17T21:59:58.947783+0000","last_fullsized":"2024-05-17T21:59:58.947783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:39:05.022711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.950115+0000","last_change":"2024-05-17T21:59:56.544055+0000","last_active":"2024-05-17T21:59:58.950115+0000","last_peered":"2024-05-17T21:59:58.950115+0000","last_clean":"2024-05-17T21:59:58.950115+0000","last_became_active":"2024-05-17T21:59:56.543730+0000","last_became_peered":"2024-05-17T21:59:56.543730+0000","last_unstale":"2024-05-17T21:59:58.950115+0000","last_undegraded":"2024-05-17T21:59:58.950115+0000","last_fullsized":"2024-05-17T21:59:58.950115+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:54.155845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947074+0000","last_change":"2024-05-17T21:59:57.585508+0000","last_active":"2024-05-17T21:59:58.947074+0000","last_peered":"2024-05-17T21:59:58.947074+0000","last_clean":"2024-05-17T21:59:58.947074+0000","last_became_active":"2024-05-17T21:59:57.585158+0000","last_became_peered":"2024-05-17T21:59:57.585158+0000","last_unstale":"2024-05-17T21:59:58.947074+0000","last_undegraded":"2024-05-17T21:59:58.947074+0000","last_fullsized":"2024-05-17T21:59:58.947074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:53.659004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947773+0000","last_change":"2024-05-17T21:59:56.554990+0000","last_active":"2024-05-17T21:59:58.947773+0000","last_peered":"2024-05-17T21:59:58.947773+0000","last_clean":"2024-05-17T21:59:58.947773+0000","last_became_active":"2024-05-17T21:59:56.554213+0000","last_became_peered":"2024-05-17T21:59:56.554213+0000","last_unstale":"2024-05-17T21:59:58.947773+0000","last_undegraded":"2024-05-17T21:59:58.947773+0000","last_fullsized":"2024-05-17T21:59:58.947773+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946456+0000","last_change":"2024-05-17T21:59:57.593127+0000","last_active":"2024-05-17T21:59:58.946456+0000","last_peered":"2024-05-17T21:59:58.946456+0000","last_clean":"2024-05-17T21:59:58.946456+0000","last_became_active":"2024-05-17T21:59:57.586477+0000","last_became_peered":"2024-05-17T21:59:57.586477+0000","last_unstale":"2024-05-17T21:59:58.946456+0000","last_undegraded":"2024-05-17T21:59:58.946456+0000","last_fullsized":"2024-05-17T21:59:58.946456+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'6","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585784+0000","last_change":"2024-05-17T21:59:56.566611+0000","last_active":"2024-05-17T21:59:58.585784+0000","last_peered":"2024-05-17T21:59:58.585784+0000","last_clean":"2024-05-17T21:59:58.585784+0000","last_became_active":"2024-05-17T21:59:56.565817+0000","last_became_peered":"2024-05-17T21:59:56.565817+0000","last_unstale":"2024-05-17T21:59:58.585784+0000","last_undegraded":"2024-05-17T21:59:58.585784+0000","last_fullsized":"2024-05-17T21:59:58.585784+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:23.236663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587739+0000","last_change":"2024-05-17T21:59:57.610368+0000","last_active":"2024-05-17T21:59:58.587739+0000","last_peered":"2024-05-17T21:59:58.587739+0000","last_clean":"2024-05-17T21:59:58.587739+0000","last_became_active":"2024-05-17T21:59:57.609571+0000","last_became_peered":"2024-05-17T21:59:57.609571+0000","last_unstale":"2024-05-17T21:59:58.587739+0000","last_undegraded":"2024-05-17T21:59:58.587739+0000","last_fullsized":"2024-05-17T21:59:58.587739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:59:56.429573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949289+0000","last_change":"2024-05-17T21:59:56.551263+0000","last_active":"2024-05-17T21:59:58.949289+0000","last_peered":"2024-05-17T21:59:58.949289+0000","last_clean":"2024-05-17T21:59:58.949289+0000","last_became_active":"2024-05-17T21:59:56.550941+0000","last_became_peered":"2024-05-17T21:59:56.550941+0000","last_unstale":"2024-05-17T21:59:58.949289+0000","last_undegraded":"2024-05-17T21:59:58.949289+0000","last_fullsized":"2024-05-17T21:59:58.949289+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:09.847811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946186+0000","last_change":"2024-05-17T21:59:57.592589+0000","last_active":"2024-05-17T21:59:58.946186+0000","last_peered":"2024-05-17T21:59:58.946186+0000","last_clean":"2024-05-17T21:59:58.946186+0000","last_became_active":"2024-05-17T21:59:57.591014+0000","last_became_peered":"2024-05-17T21:59:57.591014+0000","last_unstale":"2024-05-17T21:59:58.946186+0000","last_undegraded":"2024-05-17T21:59:58.946186+0000","last_fullsized":"2024-05-17T21:59:58.946186+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945753+0000","last_change":"2024-05-17T21:59:56.559255+0000","last_active":"2024-05-17T21:59:58.945753+0000","last_peered":"2024-05-17T21:59:58.945753+0000","last_clean":"2024-05-17T21:59:58.945753+0000","last_became_active":"2024-05-17T21:59:56.557360+0000","last_became_peered":"2024-05-17T21:59:56.557360+0000","last_unstale":"2024-05-17T21:59:58.945753+0000","last_undegraded":"2024-05-17T21:59:58.945753+0000","last_fullsized":"2024-05-17T21:59:58.945753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:49:24.540087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585342+0000","last_change":"2024-05-17T21:59:57.571310+0000","last_active":"2024-05-17T21:59:58.585342+0000","last_peered":"2024-05-17T21:59:58.585342+0000","last_clean":"2024-05-17T21:59:58.585342+0000","last_became_active":"2024-05-17T21:59:57.570449+0000","last_became_peered":"2024-05-17T21:59:57.570449+0000","last_unstale":"2024-05-17T21:59:58.585342+0000","last_undegraded":"2024-05-17T21:59:58.585342+0000","last_fullsized":"2024-05-17T21:59:58.585342+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:09:10.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587728+0000","last_change":"2024-05-17T21:59:56.553311+0000","last_active":"2024-05-17T21:59:58.587728+0000","last_peered":"2024-05-17T21:59:58.587728+0000","last_clean":"2024-05-17T21:59:58.587728+0000","last_became_active":"2024-05-17T21:59:56.552507+0000","last_became_peered":"2024-05-17T21:59:56.552507+0000","last_unstale":"2024-05-17T21:59:58.587728+0000","last_undegraded":"2024-05-17T21:59:58.587728+0000","last_fullsized":"2024-05-17T21:59:58.587728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:54.275995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946436+0000","last_change":"2024-05-17T21:59:57.594685+0000","last_active":"2024-05-17T21:59:58.946436+0000","last_peered":"2024-05-17T21:59:58.946436+0000","last_clean":"2024-05-17T21:59:58.946436+0000","last_became_active":"2024-05-17T21:59:57.593847+0000","last_became_peered":"2024-05-17T21:59:57.593847+0000","last_unstale":"2024-05-17T21:59:58.946436+0000","last_undegraded":"2024-05-17T21:59:58.946436+0000","last_fullsized":"2024-05-17T21:59:58.946436+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945633+0000","last_change":"2024-05-17T21:59:56.572177+0000","last_active":"2024-05-17T21:59:58.945633+0000","last_peered":"2024-05-17T21:59:58.945633+0000","last_clean":"2024-05-17T21:59:58.945633+0000","last_became_active":"2024-05-17T21:59:56.571143+0000","last_became_peered":"2024-05-17T21:59:56.571143+0000","last_unstale":"2024-05-17T21:59:58.945633+0000","last_undegraded":"2024-05-17T21:59:58.945633+0000","last_fullsized":"2024-05-17T21:59:58.945633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946910+0000","last_change":"2024-05-17T21:59:57.606299+0000","last_active":"2024-05-17T21:59:58.946910+0000","last_peered":"2024-05-17T21:59:58.946910+0000","last_clean":"2024-05-17T21:59:58.946910+0000","last_became_active":"2024-05-17T21:59:57.605269+0000","last_became_peered":"2024-05-17T21:59:57.605269+0000","last_unstale":"2024-05-17T21:59:58.946910+0000","last_undegraded":"2024-05-17T21:59:58.946910+0000","last_fullsized":"2024-05-17T21:59:58.946910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:23.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.950389+0000","last_change":"2024-05-17T21:59:56.543327+0000","last_active":"2024-05-17T21:59:58.950389+0000","last_peered":"2024-05-17T21:59:58.950389+0000","last_clean":"2024-05-17T21:59:58.950389+0000","last_became_active":"2024-05-17T21:59:56.543007+0000","last_became_peered":"2024-05-17T21:59:56.543007+0000","last_unstale":"2024-05-17T21:59:58.950389+0000","last_undegraded":"2024-05-17T21:59:58.950389+0000","last_fullsized":"2024-05-17T21:59:58.950389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:34.453336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588837+0000","last_change":"2024-05-17T21:59:57.609683+0000","last_active":"2024-05-17T21:59:58.588837+0000","last_peered":"2024-05-17T21:59:58.588837+0000","last_clean":"2024-05-17T21:59:58.588837+0000","last_became_active":"2024-05-17T21:59:57.606723+0000","last_became_peered":"2024-05-17T21:59:57.606723+0000","last_unstale":"2024-05-17T21:59:58.588837+0000","last_undegraded":"2024-05-17T21:59:58.588837+0000","last_fullsized":"2024-05-17T21:59:58.588837+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947214+0000","last_change":"2024-05-17T21:59:56.553813+0000","last_active":"2024-05-17T21:59:58.947214+0000","last_peered":"2024-05-17T21:59:58.947214+0000","last_clean":"2024-05-17T21:59:58.947214+0000","last_became_active":"2024-05-17T21:59:56.553343+0000","last_became_peered":"2024-05-17T21:59:56.553343+0000","last_unstale":"2024-05-17T21:59:58.947214+0000","last_undegraded":"2024-05-17T21:59:58.947214+0000","last_fullsized":"2024-05-17T21:59:58.947214+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:14.021056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949658+0000","last_change":"2024-05-17T21:59:57.560239+0000","last_active":"2024-05-17T21:59:58.949658+0000","last_peered":"2024-05-17T21:59:58.949658+0000","last_clean":"2024-05-17T21:59:58.949658+0000","last_became_active":"2024-05-17T21:59:57.559789+0000","last_became_peered":"2024-05-17T21:59:57.559789+0000","last_unstale":"2024-05-17T21:59:58.949658+0000","last_undegraded":"2024-05-17T21:59:58.949658+0000","last_fullsized":"2024-05-17T21:59:58.949658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:57:10.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588775+0000","last_change":"2024-05-17T21:59:56.555291+0000","last_active":"2024-05-17T21:59:58.588775+0000","last_peered":"2024-05-17T21:59:58.588775+0000","last_clean":"2024-05-17T21:59:58.588775+0000","last_became_active":"2024-05-17T21:59:56.554565+0000","last_became_peered":"2024-05-17T21:59:56.554565+0000","last_unstale":"2024-05-17T21:59:58.588775+0000","last_undegraded":"2024-05-17T21:59:58.588775+0000","last_fullsized":"2024-05-17T21:59:58.588775+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:38:03.296880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947124+0000","last_change":"2024-05-17T21:59:57.571363+0000","last_active":"2024-05-17T21:59:58.947124+0000","last_peered":"2024-05-17T21:59:58.947124+0000","last_clean":"2024-05-17T21:59:58.947124+0000","last_became_active":"2024-05-17T21:59:57.570532+0000","last_became_peered":"2024-05-17T21:59:57.570532+0000","last_unstale":"2024-05-17T21:59:58.947124+0000","last_undegraded":"2024-05-17T21:59:58.947124+0000","last_fullsized":"2024-05-17T21:59:58.947124+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:51.977412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945379+0000","last_change":"2024-05-17T21:59:56.545943+0000","last_active":"2024-05-17T21:59:58.945379+0000","last_peered":"2024-05-17T21:59:58.945379+0000","last_clean":"2024-05-17T21:59:58.945379+0000","last_became_active":"2024-05-17T21:59:56.545484+0000","last_became_peered":"2024-05-17T21:59:56.545484+0000","last_unstale":"2024-05-17T21:59:58.945379+0000","last_undegraded":"2024-05-17T21:59:58.945379+0000","last_fullsized":"2024-05-17T21:59:58.945379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587766+0000","last_change":"2024-05-17T21:59:57.605763+0000","last_active":"2024-05-17T21:59:58.587766+0000","last_peered":"2024-05-17T21:59:58.587766+0000","last_clean":"2024-05-17T21:59:58.587766+0000","last_became_active":"2024-05-17T21:59:57.605008+0000","last_became_peered":"2024-05-17T21:59:57.605008+0000","last_unstale":"2024-05-17T21:59:58.587766+0000","last_undegraded":"2024-05-17T21:59:58.587766+0000","last_fullsized":"2024-05-17T21:59:58.587766+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949947+0000","last_change":"2024-05-17T21:59:56.565665+0000","last_active":"2024-05-17T21:59:58.949947+0000","last_peered":"2024-05-17T21:59:58.949947+0000","last_clean":"2024-05-17T21:59:58.949947+0000","last_became_active":"2024-05-17T21:59:56.565311+0000","last_became_peered":"2024-05-17T21:59:56.565311+0000","last_unstale":"2024-05-17T21:59:58.949947+0000","last_undegraded":"2024-05-17T21:59:58.949947+0000","last_fullsized":"2024-05-17T21:59:58.949947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:22.717000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946733+0000","last_change":"2024-05-17T21:59:57.591073+0000","last_active":"2024-05-17T21:59:58.946733+0000","last_peered":"2024-05-17T21:59:58.946733+0000","last_clean":"2024-05-17T21:59:58.946733+0000","last_became_active":"2024-05-17T21:59:57.590088+0000","last_became_peered":"2024-05-17T21:59:57.590088+0000","last_unstale":"2024-05-17T21:59:58.946733+0000","last_undegraded":"2024-05-17T21:59:58.946733+0000","last_fullsized":"2024-05-17T21:59:58.946733+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:23.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945478+0000","last_change":"2024-05-17T21:59:56.531280+0000","last_active":"2024-05-17T21:59:58.945478+0000","last_peered":"2024-05-17T21:59:58.945478+0000","last_clean":"2024-05-17T21:59:58.945478+0000","last_became_active":"2024-05-17T21:59:56.530987+0000","last_became_peered":"2024-05-17T21:59:56.530987+0000","last_unstale":"2024-05-17T21:59:58.945478+0000","last_undegraded":"2024-05-17T21:59:58.945478+0000","last_fullsized":"2024-05-17T21:59:58.945478+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:17.374303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.944455+0000","last_change":"2024-05-17T21:59:57.574579+0000","last_active":"2024-05-17T21:59:58.944455+0000","last_peered":"2024-05-17T21:59:58.944455+0000","last_clean":"2024-05-17T21:59:58.944455+0000","last_became_active":"2024-05-17T21:59:57.572362+0000","last_became_peered":"2024-05-17T21:59:57.572362+0000","last_unstale":"2024-05-17T21:59:58.944455+0000","last_undegraded":"2024-05-17T21:59:58.944455+0000","last_fullsized":"2024-05-17T21:59:58.944455+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587613+0000","last_change":"2024-05-17T21:59:56.546993+0000","last_active":"2024-05-17T21:59:58.587613+0000","last_peered":"2024-05-17T21:59:58.587613+0000","last_clean":"2024-05-17T21:59:58.587613+0000","last_became_active":"2024-05-17T21:59:56.545808+0000","last_became_peered":"2024-05-17T21:59:56.545808+0000","last_unstale":"2024-05-17T21:59:58.587613+0000","last_undegraded":"2024-05-17T21:59:58.587613+0000","last_fullsized":"2024-05-17T21:59:58.587613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588814+0000","last_change":"2024-05-17T21:59:57.609714+0000","last_active":"2024-05-17T21:59:58.588814+0000","last_peered":"2024-05-17T21:59:58.588814+0000","last_clean":"2024-05-17T21:59:58.588814+0000","last_became_active":"2024-05-17T21:59:57.606514+0000","last_became_peered":"2024-05-17T21:59:57.606514+0000","last_unstale":"2024-05-17T21:59:58.588814+0000","last_undegraded":"2024-05-17T21:59:58.588814+0000","last_fullsized":"2024-05-17T21:59:58.588814+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947883+0000","last_change":"2024-05-17T21:59:56.543524+0000","last_active":"2024-05-17T21:59:58.947883+0000","last_peered":"2024-05-17T21:59:58.947883+0000","last_clean":"2024-05-17T21:59:58.947883+0000","last_became_active":"2024-05-17T21:59:56.543182+0000","last_became_peered":"2024-05-17T21:59:56.543182+0000","last_unstale":"2024-05-17T21:59:58.947883+0000","last_undegraded":"2024-05-17T21:59:58.947883+0000","last_fullsized":"2024-05-17T21:59:58.947883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:22:49.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588978+0000","last_change":"2024-05-17T21:59:57.619081+0000","last_active":"2024-05-17T21:59:58.588978+0000","last_peered":"2024-05-17T21:59:58.588978+0000","last_clean":"2024-05-17T21:59:58.588978+0000","last_became_active":"2024-05-17T21:59:57.618439+0000","last_became_peered":"2024-05-17T21:59:57.618439+0000","last_unstale":"2024-05-17T21:59:58.588978+0000","last_undegraded":"2024-05-17T21:59:58.588978+0000","last_fullsized":"2024-05-17T21:59:58.588978+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947291+0000","last_change":"2024-05-17T21:59:56.568281+0000","last_active":"2024-05-17T21:59:58.947291+0000","last_peered":"2024-05-17T21:59:58.947291+0000","last_clean":"2024-05-17T21:59:58.947291+0000","last_became_active":"2024-05-17T21:59:56.567962+0000","last_became_peered":"2024-05-17T21:59:56.567962+0000","last_unstale":"2024-05-17T21:59:58.947291+0000","last_undegraded":"2024-05-17T21:59:58.947291+0000","last_fullsized":"2024-05-17T21:59:58.947291+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:09:09.351072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946926+0000","last_change":"2024-05-17T21:59:57.585662+0000","last_active":"2024-05-17T21:59:58.946926+0000","last_peered":"2024-05-17T21:59:58.946926+0000","last_clean":"2024-05-17T21:59:58.946926+0000","last_became_active":"2024-05-17T21:59:57.585231+0000","last_became_peered":"2024-05-17T21:59:57.585231+0000","last_unstale":"2024-05-17T21:59:58.946926+0000","last_undegraded":"2024-05-17T21:59:58.946926+0000","last_fullsized":"2024-05-17T21:59:58.946926+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:27.287126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947025+0000","last_change":"2024-05-17T21:59:56.567011+0000","last_active":"2024-05-17T21:59:58.947025+0000","last_peered":"2024-05-17T21:59:58.947025+0000","last_clean":"2024-05-17T21:59:58.947025+0000","last_became_active":"2024-05-17T21:59:56.561654+0000","last_became_peered":"2024-05-17T21:59:56.561654+0000","last_unstale":"2024-05-17T21:59:58.947025+0000","last_undegraded":"2024-05-17T21:59:58.947025+0000","last_fullsized":"2024-05-17T21:59:58.947025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:29.278746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946344+0000","last_change":"2024-05-17T21:59:57.606005+0000","last_active":"2024-05-17T21:59:58.946344+0000","last_peered":"2024-05-17T21:59:58.946344+0000","last_clean":"2024-05-17T21:59:58.946344+0000","last_became_active":"2024-05-17T21:59:57.605649+0000","last_became_peered":"2024-05-17T21:59:57.605649+0000","last_unstale":"2024-05-17T21:59:58.946344+0000","last_undegraded":"2024-05-17T21:59:58.946344+0000","last_fullsized":"2024-05-17T21:59:58.946344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.589128+0000","last_change":"2024-05-17T21:59:56.563094+0000","last_active":"2024-05-17T21:59:58.589128+0000","last_peered":"2024-05-17T21:59:58.589128+0000","last_clean":"2024-05-17T21:59:58.589128+0000","last_became_active":"2024-05-17T21:59:56.561722+0000","last_became_peered":"2024-05-17T21:59:56.561722+0000","last_unstale":"2024-05-17T21:59:58.589128+0000","last_undegraded":"2024-05-17T21:59:58.589128+0000","last_fullsized":"2024-05-17T21:59:58.589128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:14.021056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949214+0000","last_change":"2024-05-17T21:59:57.581568+0000","last_active":"2024-05-17T21:59:58.949214+0000","last_peered":"2024-05-17T21:59:58.949214+0000","last_clean":"2024-05-17T21:59:58.949214+0000","last_became_active":"2024-05-17T21:59:57.568279+0000","last_became_peered":"2024-05-17T21:59:57.568279+0000","last_unstale":"2024-05-17T21:59:58.949214+0000","last_undegraded":"2024-05-17T21:59:58.949214+0000","last_fullsized":"2024-05-17T21:59:58.949214+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:06.103008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945026+0000","last_change":"2024-05-17T21:59:56.545923+0000","last_active":"2024-05-17T21:59:58.945026+0000","last_peered":"2024-05-17T21:59:58.945026+0000","last_clean":"2024-05-17T21:59:58.945026+0000","last_became_active":"2024-05-17T21:59:56.545578+0000","last_became_peered":"2024-05-17T21:59:56.545578+0000","last_unstale":"2024-05-17T21:59:58.945026+0000","last_undegraded":"2024-05-17T21:59:58.945026+0000","last_fullsized":"2024-05-17T21:59:58.945026+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:54:13.110045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946653+0000","last_change":"2024-05-17T21:59:57.595435+0000","last_active":"2024-05-17T21:59:58.946653+0000","last_peered":"2024-05-17T21:59:58.946653+0000","last_clean":"2024-05-17T21:59:58.946653+0000","last_became_active":"2024-05-17T21:59:57.594503+0000","last_became_peered":"2024-05-17T21:59:57.594503+0000","last_unstale":"2024-05-17T21:59:58.946653+0000","last_undegraded":"2024-05-17T21:59:58.946653+0000","last_fullsized":"2024-05-17T21:59:58.946653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585875+0000","last_change":"2024-05-17T21:59:56.560679+0000","last_active":"2024-05-17T21:59:58.585875+0000","last_peered":"2024-05-17T21:59:58.585875+0000","last_clean":"2024-05-17T21:59:58.585875+0000","last_became_active":"2024-05-17T21:59:56.559426+0000","last_became_peered":"2024-05-17T21:59:56.559426+0000","last_unstale":"2024-05-17T21:59:58.585875+0000","last_undegraded":"2024-05-17T21:59:58.585875+0000","last_fullsized":"2024-05-17T21:59:58.585875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:17.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945650+0000","last_change":"2024-05-17T21:59:57.589684+0000","last_active":"2024-05-17T21:59:58.945650+0000","last_peered":"2024-05-17T21:59:58.945650+0000","last_clean":"2024-05-17T21:59:58.945650+0000","last_became_active":"2024-05-17T21:59:57.589373+0000","last_became_peered":"2024-05-17T21:59:57.589373+0000","last_unstale":"2024-05-17T21:59:58.945650+0000","last_undegraded":"2024-05-17T21:59:58.945650+0000","last_fullsized":"2024-05-17T21:59:58.945650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:24.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945237+0000","last_change":"2024-05-17T21:59:56.548697+0000","last_active":"2024-05-17T21:59:58.945237+0000","last_peered":"2024-05-17T21:59:58.945237+0000","last_clean":"2024-05-17T21:59:58.945237+0000","last_became_active":"2024-05-17T21:59:56.548414+0000","last_became_peered":"2024-05-17T21:59:56.548414+0000","last_unstale":"2024-05-17T21:59:58.945237+0000","last_undegraded":"2024-05-17T21:59:58.945237+0000","last_fullsized":"2024-05-17T21:59:58.945237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:50:05.496567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948975+0000","last_change":"2024-05-17T21:59:57.591063+0000","last_active":"2024-05-17T21:59:58.948975+0000","last_peered":"2024-05-17T21:59:58.948975+0000","last_clean":"2024-05-17T21:59:58.948975+0000","last_became_active":"2024-05-17T21:59:57.590681+0000","last_became_peered":"2024-05-17T21:59:57.590681+0000","last_unstale":"2024-05-17T21:59:58.948975+0000","last_undegraded":"2024-05-17T21:59:58.948975+0000","last_fullsized":"2024-05-17T21:59:58.948975+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:18:28.589127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949318+0000","last_change":"2024-05-17T21:59:56.545007+0000","last_active":"2024-05-17T21:59:58.949318+0000","last_peered":"2024-05-17T21:59:58.949318+0000","last_clean":"2024-05-17T21:59:58.949318+0000","last_became_active":"2024-05-17T21:59:56.543719+0000","last_became_peered":"2024-05-17T21:59:56.543719+0000","last_unstale":"2024-05-17T21:59:58.949318+0000","last_undegraded":"2024-05-17T21:59:58.949318+0000","last_fullsized":"2024-05-17T21:59:58.949318+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:24.456369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948804+0000","last_change":"2024-05-17T21:59:57.584583+0000","last_active":"2024-05-17T21:59:58.948804+0000","last_peered":"2024-05-17T21:59:58.948804+0000","last_clean":"2024-05-17T21:59:58.948804+0000","last_became_active":"2024-05-17T21:59:57.577984+0000","last_became_peered":"2024-05-17T21:59:57.577984+0000","last_unstale":"2024-05-17T21:59:58.948804+0000","last_undegraded":"2024-05-17T21:59:58.948804+0000","last_fullsized":"2024-05-17T21:59:58.948804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:20:35.522434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945990+0000","last_change":"2024-05-17T21:59:56.557845+0000","last_active":"2024-05-17T21:59:58.945990+0000","last_peered":"2024-05-17T21:59:58.945990+0000","last_clean":"2024-05-17T21:59:58.945990+0000","last_became_active":"2024-05-17T21:59:56.556982+0000","last_became_peered":"2024-05-17T21:59:56.556982+0000","last_unstale":"2024-05-17T21:59:58.945990+0000","last_undegraded":"2024-05-17T21:59:58.945990+0000","last_fullsized":"2024-05-17T21:59:58.945990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:17.374303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":175,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945725+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:59:58.945725+0000","last_peered":"2024-05-17T21:59:58.945725+0000","last_clean":"2024-05-17T21:59:58.945725+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:59:58.945725+0000","last_undegraded":"2024-05-17T21:59:58.945725+0000","last_fullsized":"2024-05-17T21:59:58.945725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":862211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":271683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":272335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":273295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":271739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":667648,"data_stored":271739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":271773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":864245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:00:01.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:00:01.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 Processor -- start 2024-05-17T22:00:01.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 -- start start 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ba815fef0 0x7f1ba806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ba8073540 0x7f1ba80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba80b49b0 con 0x7f1ba80b5a50 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba80b4c90 con 0x7f1ba8073540 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1badcb0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba80b4f70 con 0x7f1ba815fef0 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1ba80b5a50 0x7f1ba8072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50266/0 (socket says 172.21.15.83:50266) 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba6ffd640 1 -- 172.21.15.83:0/678236302 learned_addr learned my addr 172.21.15.83:0/678236302 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/678236302 >> v2:172.21.15.83:3300/0 conn(0x7f1ba8073540 0x7f1ba80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba77fe640 1 --2- 172.21.15.83:0/678236302 >> v2:172.21.15.155:3300/0 conn(0x7f1ba815fef0 0x7f1ba806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba7fff640 1 -- 172.21.15.83:0/678236302 >> v2:172.21.15.155:3300/0 conn(0x7f1ba815fef0 msgr2=0x7f1ba806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/678236302 >> v2:172.21.15.155:3300/0 conn(0x7f1ba815fef0 0x7f1ba806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba7fff640 1 -- 172.21.15.83:0/678236302 >> v1:172.21.15.83:6789/0 conn(0x7f1ba80b5a50 legacy=0x7f1ba8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.860+0000 7f1ba7fff640 1 -- 172.21.15.83:0/678236302 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba80b5250 con 0x7f1ba8073540 2024-05-17T22:00:01.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/678236302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba8073540 0x7f1ba80b5e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1ba806d730 tx=0x7f1ba001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca31fc6b86a15977 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1ba67fc640 1 -- 172.21.15.83:0/678236302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00b2070 con 0x7f1ba8073540 2024-05-17T22:00:01.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1ba67fc640 1 -- 172.21.15.83:0/678236302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba00a5790 con 0x7f1ba8073540 2024-05-17T22:00:01.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1ba67fc640 1 -- 172.21.15.83:0/678236302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00a5b60 con 0x7f1ba8073540 2024-05-17T22:00:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 -- 172.21.15.83:0/678236302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba8073540 msgr2=0x7f1ba80b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 --2- 172.21.15.83:0/678236302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba8073540 0x7f1ba80b5e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1ba806d730 tx=0x7f1ba001c730 comp rx=0 tx=0).stop 2024-05-17T22:00:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 -- 172.21.15.83:0/678236302 shutdown_connections 2024-05-17T22:00:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 --2- 172.21.15.83:0/678236302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba8073540 0x7f1ba80b5e60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 --2- 172.21.15.83:0/678236302 >> v2:172.21.15.155:3300/0 conn(0x7f1ba815fef0 0x7f1ba806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 -- 172.21.15.83:0/678236302 >> 172.21.15.83:0/678236302 conn(0x7f1ba806cd90 msgr2=0x7f1ba80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 -- 172.21.15.83:0/678236302 shutdown_connections 2024-05-17T22:00:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.864+0000 7f1badcb0640 1 -- 172.21.15.83:0/678236302 wait complete. 2024-05-17T22:00:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 Processor -- start 2024-05-17T22:00:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- start start 2024-05-17T22:00:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba80b5a50 0x7f1ba80cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 0x7f1ba814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80b4750 con 0x7f1ba80b1240 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba80b45d0 con 0x7f1ba80b5a50 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba806fd90 con 0x7f1ba815fef0 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47946/0 (socket says 172.21.15.83:47946) 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 -- 172.21.15.83:0/4168318681 learned_addr learned my addr 172.21.15.83:0/4168318681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba77fe640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba80b5a50 0x7f1ba80cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba6ffd640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 0x7f1ba814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 -- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 msgr2=0x7f1ba814bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 0x7f1ba814bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 -- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba80b5a50 msgr2=0x7f1ba80cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba80b5a50 0x7f1ba80cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 -- 172.21.15.83:0/4168318681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba80b1b90 con 0x7f1ba80b1240 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba6ffd640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 0x7f1ba814bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1ba7fff640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00b0040 tx=0x7f1ba00194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00b2050 con 0x7f1ba80b1240 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba00be070 con 0x7f1ba80b1240 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00b9450 con 0x7f1ba80b1240 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba80cfa80 con 0x7f1ba80b1240 2024-05-17T22:00:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba815a180 con 0x7f1ba80b1240 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.868+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba80b5e60 con 0x7f1ba80b1240 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.872+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ba00b95f0 con 0x7f1ba80b1240 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.872+0000 7f1b9ffff640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 0x7f1b88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.872+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f1ba012a880 con 0x7f1ba80b1240 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.872+0000 7f1ba77fe640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 0x7f1b88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.872+0000 7f1ba77fe640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 0x7f1b88071940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800a8d0 tx=0x7f1b98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:01.876+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1ba80b1b90 con 0x7f1ba80b1240 2024-05-17T22:00:02.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.020+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ba80cffa0 con 0x7f1b8806f430 2024-05-17T22:00:02.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.032+0000 7f1b9ffff640 1 -- 172.21.15.83:0/4168318681 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f1b9800a5a0 con 0x7f1b8806f430 2024-05-17T22:00:02.025 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:02.028 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 msgr2=0x7f1b88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 0x7f1b88071940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800a8d0 tx=0x7f1b98008040 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 msgr2=0x7f1ba80b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00b0040 tx=0x7f1ba00194d0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 shutdown_connections 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ba80b1240 0x7f1ba80b1650 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ba815fef0 0x7f1ba814bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b8806f430 0x7f1b88071940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 --2- 172.21.15.83:0/4168318681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ba80b5a50 0x7f1ba80cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 >> 172.21.15.83:0/4168318681 conn(0x7f1ba806cd90 msgr2=0x7f1ba80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 shutdown_connections 2024-05-17T22:00:02.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.036+0000 7f1badcb0640 1 -- 172.21.15.83:0/4168318681 wait complete. 2024-05-17T22:00:02.046 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-05-17T22:00:01.018843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239232,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735488,"statfs":{"total":773094113280,"available":772849139712,"internally_reserved":0,"allocated":6529024,"data_stored":3359020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.503732"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582773+0000","last_change":"2024-05-17T21:59:57.593439+0000","last_active":"2024-05-17T21:59:58.582773+0000","last_peered":"2024-05-17T21:59:58.582773+0000","last_clean":"2024-05-17T21:59:58.582773+0000","last_became_active":"2024-05-17T21:59:57.592898+0000","last_became_peered":"2024-05-17T21:59:57.592898+0000","last_unstale":"2024-05-17T21:59:58.582773+0000","last_undegraded":"2024-05-17T21:59:58.582773+0000","last_fullsized":"2024-05-17T21:59:58.582773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:05:38.810696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949783+0000","last_change":"2024-05-17T21:59:57.577584+0000","last_active":"2024-05-17T21:59:58.949783+0000","last_peered":"2024-05-17T21:59:58.949783+0000","last_clean":"2024-05-17T21:59:58.949783+0000","last_became_active":"2024-05-17T21:59:57.577096+0000","last_became_peered":"2024-05-17T21:59:57.577096+0000","last_unstale":"2024-05-17T21:59:58.949783+0000","last_undegraded":"2024-05-17T21:59:58.949783+0000","last_fullsized":"2024-05-17T21:59:58.949783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:46:39.087363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946387+0000","last_change":"2024-05-17T21:59:57.593222+0000","last_active":"2024-05-17T21:59:58.946387+0000","last_peered":"2024-05-17T21:59:58.946387+0000","last_clean":"2024-05-17T21:59:58.946387+0000","last_became_active":"2024-05-17T21:59:57.592399+0000","last_became_peered":"2024-05-17T21:59:57.592399+0000","last_unstale":"2024-05-17T21:59:58.946387+0000","last_undegraded":"2024-05-17T21:59:58.946387+0000","last_fullsized":"2024-05-17T21:59:58.946387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588433+0000","last_change":"2024-05-17T21:59:57.606023+0000","last_active":"2024-05-17T21:59:58.588433+0000","last_peered":"2024-05-17T21:59:58.588433+0000","last_clean":"2024-05-17T21:59:58.588433+0000","last_became_active":"2024-05-17T21:59:57.604799+0000","last_became_peered":"2024-05-17T21:59:57.604799+0000","last_unstale":"2024-05-17T21:59:58.588433+0000","last_undegraded":"2024-05-17T21:59:58.588433+0000","last_fullsized":"2024-05-17T21:59:58.588433+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582692+0000","last_change":"2024-05-17T21:59:57.606260+0000","last_active":"2024-05-17T21:59:58.582692+0000","last_peered":"2024-05-17T21:59:58.582692+0000","last_clean":"2024-05-17T21:59:58.582692+0000","last_became_active":"2024-05-17T21:59:57.605949+0000","last_became_peered":"2024-05-17T21:59:57.605949+0000","last_unstale":"2024-05-17T21:59:58.582692+0000","last_undegraded":"2024-05-17T21:59:58.582692+0000","last_fullsized":"2024-05-17T21:59:58.582692+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:25.921921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946309+0000","last_change":"2024-05-17T21:59:57.576495+0000","last_active":"2024-05-17T21:59:58.946309+0000","last_peered":"2024-05-17T21:59:58.946309+0000","last_clean":"2024-05-17T21:59:58.946309+0000","last_became_active":"2024-05-17T21:59:57.575977+0000","last_became_peered":"2024-05-17T21:59:57.575977+0000","last_unstale":"2024-05-17T21:59:58.946309+0000","last_undegraded":"2024-05-17T21:59:58.946309+0000","last_fullsized":"2024-05-17T21:59:58.946309+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585006+0000","last_change":"2024-05-17T21:59:57.608621+0000","last_active":"2024-05-17T21:59:58.585006+0000","last_peered":"2024-05-17T21:59:58.585006+0000","last_clean":"2024-05-17T21:59:58.585006+0000","last_became_active":"2024-05-17T21:59:57.607615+0000","last_became_peered":"2024-05-17T21:59:57.607615+0000","last_unstale":"2024-05-17T21:59:58.585006+0000","last_undegraded":"2024-05-17T21:59:58.585006+0000","last_fullsized":"2024-05-17T21:59:58.585006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:39:26.066389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949008+0000","last_change":"2024-05-17T21:59:57.562290+0000","last_active":"2024-05-17T21:59:58.949008+0000","last_peered":"2024-05-17T21:59:58.949008+0000","last_clean":"2024-05-17T21:59:58.949008+0000","last_became_active":"2024-05-17T21:59:57.561977+0000","last_became_peered":"2024-05-17T21:59:57.561977+0000","last_unstale":"2024-05-17T21:59:58.949008+0000","last_undegraded":"2024-05-17T21:59:58.949008+0000","last_fullsized":"2024-05-17T21:59:58.949008+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:17:48.998895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585910+0000","last_change":"2024-05-17T21:59:57.589058+0000","last_active":"2024-05-17T21:59:58.585910+0000","last_peered":"2024-05-17T21:59:58.585910+0000","last_clean":"2024-05-17T21:59:58.585910+0000","last_became_active":"2024-05-17T21:59:57.588268+0000","last_became_peered":"2024-05-17T21:59:57.588268+0000","last_unstale":"2024-05-17T21:59:58.585910+0000","last_undegraded":"2024-05-17T21:59:58.585910+0000","last_fullsized":"2024-05-17T21:59:58.585910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:17:23.346065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947092+0000","last_change":"2024-05-17T21:59:57.589623+0000","last_active":"2024-05-17T21:59:58.947092+0000","last_peered":"2024-05-17T21:59:58.947092+0000","last_clean":"2024-05-17T21:59:58.947092+0000","last_became_active":"2024-05-17T21:59:57.589340+0000","last_became_peered":"2024-05-17T21:59:57.589340+0000","last_unstale":"2024-05-17T21:59:58.947092+0000","last_undegraded":"2024-05-17T21:59:58.947092+0000","last_fullsized":"2024-05-17T21:59:58.947092+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:55.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946385+0000","last_change":"2024-05-17T21:59:57.593337+0000","last_active":"2024-05-17T21:59:58.946385+0000","last_peered":"2024-05-17T21:59:58.946385+0000","last_clean":"2024-05-17T21:59:58.946385+0000","last_became_active":"2024-05-17T21:59:57.593001+0000","last_became_peered":"2024-05-17T21:59:57.593001+0000","last_unstale":"2024-05-17T21:59:58.946385+0000","last_undegraded":"2024-05-17T21:59:58.946385+0000","last_fullsized":"2024-05-17T21:59:58.946385+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:13.550814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588987+0000","last_change":"2024-05-17T21:59:57.619088+0000","last_active":"2024-05-17T21:59:58.588987+0000","last_peered":"2024-05-17T21:59:58.588987+0000","last_clean":"2024-05-17T21:59:58.588987+0000","last_became_active":"2024-05-17T21:59:57.618364+0000","last_became_peered":"2024-05-17T21:59:57.618364+0000","last_unstale":"2024-05-17T21:59:58.588987+0000","last_undegraded":"2024-05-17T21:59:58.588987+0000","last_fullsized":"2024-05-17T21:59:58.588987+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946700+0000","last_change":"2024-05-17T21:59:57.588866+0000","last_active":"2024-05-17T21:59:58.946700+0000","last_peered":"2024-05-17T21:59:58.946700+0000","last_clean":"2024-05-17T21:59:58.946700+0000","last_became_active":"2024-05-17T21:59:57.587823+0000","last_became_peered":"2024-05-17T21:59:57.587823+0000","last_unstale":"2024-05-17T21:59:58.946700+0000","last_undegraded":"2024-05-17T21:59:58.946700+0000","last_fullsized":"2024-05-17T21:59:58.946700+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:17:26.370984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585902+0000","last_change":"2024-05-17T21:59:57.579238+0000","last_active":"2024-05-17T21:59:58.585902+0000","last_peered":"2024-05-17T21:59:58.585902+0000","last_clean":"2024-05-17T21:59:58.585902+0000","last_became_active":"2024-05-17T21:59:57.578428+0000","last_became_peered":"2024-05-17T21:59:57.578428+0000","last_unstale":"2024-05-17T21:59:58.585902+0000","last_undegraded":"2024-05-17T21:59:58.585902+0000","last_fullsized":"2024-05-17T21:59:58.585902+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:09.989753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585472+0000","last_change":"2024-05-17T21:59:57.568112+0000","last_active":"2024-05-17T21:59:58.585472+0000","last_peered":"2024-05-17T21:59:58.585472+0000","last_clean":"2024-05-17T21:59:58.585472+0000","last_became_active":"2024-05-17T21:59:57.567185+0000","last_became_peered":"2024-05-17T21:59:57.567185+0000","last_unstale":"2024-05-17T21:59:58.585472+0000","last_undegraded":"2024-05-17T21:59:58.585472+0000","last_fullsized":"2024-05-17T21:59:58.585472+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:57:53.830481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948066+0000","last_change":"2024-05-17T21:59:57.590738+0000","last_active":"2024-05-17T21:59:58.948066+0000","last_peered":"2024-05-17T21:59:58.948066+0000","last_clean":"2024-05-17T21:59:58.948066+0000","last_became_active":"2024-05-17T21:59:57.590063+0000","last_became_peered":"2024-05-17T21:59:57.590063+0000","last_unstale":"2024-05-17T21:59:58.948066+0000","last_undegraded":"2024-05-17T21:59:58.948066+0000","last_fullsized":"2024-05-17T21:59:58.948066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:51.977412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949711+0000","last_change":"2024-05-17T21:59:57.591365+0000","last_active":"2024-05-17T21:59:58.949711+0000","last_peered":"2024-05-17T21:59:58.949711+0000","last_clean":"2024-05-17T21:59:58.949711+0000","last_became_active":"2024-05-17T21:59:57.590811+0000","last_became_peered":"2024-05-17T21:59:57.590811+0000","last_unstale":"2024-05-17T21:59:58.949711+0000","last_undegraded":"2024-05-17T21:59:58.949711+0000","last_fullsized":"2024-05-17T21:59:58.949711+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:43:58.576535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946929+0000","last_change":"2024-05-17T21:59:57.591663+0000","last_active":"2024-05-17T21:59:58.946929+0000","last_peered":"2024-05-17T21:59:58.946929+0000","last_clean":"2024-05-17T21:59:58.946929+0000","last_became_active":"2024-05-17T21:59:57.588824+0000","last_became_peered":"2024-05-17T21:59:57.588824+0000","last_unstale":"2024-05-17T21:59:58.946929+0000","last_undegraded":"2024-05-17T21:59:58.946929+0000","last_fullsized":"2024-05-17T21:59:58.946929+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:05.324699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582848+0000","last_change":"2024-05-17T21:59:57.593258+0000","last_active":"2024-05-17T21:59:58.582848+0000","last_peered":"2024-05-17T21:59:58.582848+0000","last_clean":"2024-05-17T21:59:58.582848+0000","last_became_active":"2024-05-17T21:59:57.592984+0000","last_became_peered":"2024-05-17T21:59:57.592984+0000","last_unstale":"2024-05-17T21:59:58.582848+0000","last_undegraded":"2024-05-17T21:59:58.582848+0000","last_fullsized":"2024-05-17T21:59:58.582848+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:00:44.260853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949079+0000","last_change":"2024-05-17T21:59:57.577350+0000","last_active":"2024-05-17T21:59:58.949079+0000","last_peered":"2024-05-17T21:59:58.949079+0000","last_clean":"2024-05-17T21:59:58.949079+0000","last_became_active":"2024-05-17T21:59:57.577013+0000","last_became_peered":"2024-05-17T21:59:57.577013+0000","last_unstale":"2024-05-17T21:59:58.949079+0000","last_undegraded":"2024-05-17T21:59:58.949079+0000","last_fullsized":"2024-05-17T21:59:58.949079+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:10.214313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946795+0000","last_change":"2024-05-17T21:59:57.613709+0000","last_active":"2024-05-17T21:59:58.946795+0000","last_peered":"2024-05-17T21:59:58.946795+0000","last_clean":"2024-05-17T21:59:58.946795+0000","last_became_active":"2024-05-17T21:59:57.613085+0000","last_became_peered":"2024-05-17T21:59:57.613085+0000","last_unstale":"2024-05-17T21:59:58.946795+0000","last_undegraded":"2024-05-17T21:59:58.946795+0000","last_fullsized":"2024-05-17T21:59:58.946795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948731+0000","last_change":"2024-05-17T21:59:57.582259+0000","last_active":"2024-05-17T21:59:58.948731+0000","last_peered":"2024-05-17T21:59:58.948731+0000","last_clean":"2024-05-17T21:59:58.948731+0000","last_became_active":"2024-05-17T21:59:57.581940+0000","last_became_peered":"2024-05-17T21:59:57.581940+0000","last_unstale":"2024-05-17T21:59:58.948731+0000","last_undegraded":"2024-05-17T21:59:58.948731+0000","last_fullsized":"2024-05-17T21:59:58.948731+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:41:46.064289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945940+0000","last_change":"2024-05-17T21:59:57.594999+0000","last_active":"2024-05-17T21:59:58.945940+0000","last_peered":"2024-05-17T21:59:58.945940+0000","last_clean":"2024-05-17T21:59:58.945940+0000","last_became_active":"2024-05-17T21:59:57.594157+0000","last_became_peered":"2024-05-17T21:59:57.594157+0000","last_unstale":"2024-05-17T21:59:58.945940+0000","last_undegraded":"2024-05-17T21:59:58.945940+0000","last_fullsized":"2024-05-17T21:59:58.945940+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:59:56.429573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947589+0000","last_change":"2024-05-17T21:59:57.591965+0000","last_active":"2024-05-17T21:59:58.947589+0000","last_peered":"2024-05-17T21:59:58.947589+0000","last_clean":"2024-05-17T21:59:58.947589+0000","last_became_active":"2024-05-17T21:59:57.591040+0000","last_became_peered":"2024-05-17T21:59:57.591040+0000","last_unstale":"2024-05-17T21:59:58.947589+0000","last_undegraded":"2024-05-17T21:59:58.947589+0000","last_fullsized":"2024-05-17T21:59:58.947589+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:37.805311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582910+0000","last_change":"2024-05-17T21:59:57.576986+0000","last_active":"2024-05-17T21:59:58.582910+0000","last_peered":"2024-05-17T21:59:58.582910+0000","last_clean":"2024-05-17T21:59:58.582910+0000","last_became_active":"2024-05-17T21:59:57.576482+0000","last_became_peered":"2024-05-17T21:59:57.576482+0000","last_unstale":"2024-05-17T21:59:58.582910+0000","last_undegraded":"2024-05-17T21:59:58.582910+0000","last_fullsized":"2024-05-17T21:59:58.582910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:07:07.164818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.589004+0000","last_change":"2024-05-17T21:59:57.608774+0000","last_active":"2024-05-17T21:59:58.589004+0000","last_peered":"2024-05-17T21:59:58.589004+0000","last_clean":"2024-05-17T21:59:58.589004+0000","last_became_active":"2024-05-17T21:59:57.605813+0000","last_became_peered":"2024-05-17T21:59:57.605813+0000","last_unstale":"2024-05-17T21:59:58.589004+0000","last_undegraded":"2024-05-17T21:59:58.589004+0000","last_fullsized":"2024-05-17T21:59:58.589004+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945860+0000","last_change":"2024-05-17T21:59:57.587766+0000","last_active":"2024-05-17T21:59:58.945860+0000","last_peered":"2024-05-17T21:59:58.945860+0000","last_clean":"2024-05-17T21:59:58.945860+0000","last_became_active":"2024-05-17T21:59:57.587418+0000","last_became_peered":"2024-05-17T21:59:57.587418+0000","last_unstale":"2024-05-17T21:59:58.945860+0000","last_undegraded":"2024-05-17T21:59:58.945860+0000","last_fullsized":"2024-05-17T21:59:58.945860+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582686+0000","last_change":"2024-05-17T21:59:57.609281+0000","last_active":"2024-05-17T21:59:58.582686+0000","last_peered":"2024-05-17T21:59:58.582686+0000","last_clean":"2024-05-17T21:59:58.582686+0000","last_became_active":"2024-05-17T21:59:57.608943+0000","last_became_peered":"2024-05-17T21:59:57.608943+0000","last_unstale":"2024-05-17T21:59:58.582686+0000","last_undegraded":"2024-05-17T21:59:58.582686+0000","last_fullsized":"2024-05-17T21:59:58.582686+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:05.104866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949776+0000","last_change":"2024-05-17T21:59:57.606178+0000","last_active":"2024-05-17T21:59:58.949776+0000","last_peered":"2024-05-17T21:59:58.949776+0000","last_clean":"2024-05-17T21:59:58.949776+0000","last_became_active":"2024-05-17T21:59:57.605656+0000","last_became_peered":"2024-05-17T21:59:57.605656+0000","last_unstale":"2024-05-17T21:59:58.949776+0000","last_undegraded":"2024-05-17T21:59:58.949776+0000","last_fullsized":"2024-05-17T21:59:58.949776+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:40:47.421281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946893+0000","last_change":"2024-05-17T21:59:57.587667+0000","last_active":"2024-05-17T21:59:58.946893+0000","last_peered":"2024-05-17T21:59:58.946893+0000","last_clean":"2024-05-17T21:59:58.946893+0000","last_became_active":"2024-05-17T21:59:57.586806+0000","last_became_peered":"2024-05-17T21:59:57.586806+0000","last_unstale":"2024-05-17T21:59:58.946893+0000","last_undegraded":"2024-05-17T21:59:58.946893+0000","last_fullsized":"2024-05-17T21:59:58.946893+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.582523+0000","last_change":"2024-05-17T21:59:57.574009+0000","last_active":"2024-05-17T21:59:58.582523+0000","last_peered":"2024-05-17T21:59:58.582523+0000","last_clean":"2024-05-17T21:59:58.582523+0000","last_became_active":"2024-05-17T21:59:57.573707+0000","last_became_peered":"2024-05-17T21:59:57.573707+0000","last_unstale":"2024-05-17T21:59:58.582523+0000","last_undegraded":"2024-05-17T21:59:58.582523+0000","last_fullsized":"2024-05-17T21:59:58.582523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:38:34.612881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.944533+0000","last_change":"2024-05-17T21:59:57.593736+0000","last_active":"2024-05-17T21:59:58.944533+0000","last_peered":"2024-05-17T21:59:58.944533+0000","last_clean":"2024-05-17T21:59:58.944533+0000","last_became_active":"2024-05-17T21:59:57.593444+0000","last_became_peered":"2024-05-17T21:59:57.593444+0000","last_unstale":"2024-05-17T21:59:58.944533+0000","last_undegraded":"2024-05-17T21:59:58.944533+0000","last_fullsized":"2024-05-17T21:59:58.944533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:12.614047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946903+0000","last_change":"2024-05-17T21:59:57.615270+0000","last_active":"2024-05-17T21:59:58.946903+0000","last_peered":"2024-05-17T21:59:58.946903+0000","last_clean":"2024-05-17T21:59:58.946903+0000","last_became_active":"2024-05-17T21:59:57.614820+0000","last_became_peered":"2024-05-17T21:59:57.614820+0000","last_unstale":"2024-05-17T21:59:58.946903+0000","last_undegraded":"2024-05-17T21:59:58.946903+0000","last_fullsized":"2024-05-17T21:59:58.946903+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:05.324699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946037+0000","last_change":"2024-05-17T21:59:56.552443+0000","last_active":"2024-05-17T21:59:58.946037+0000","last_peered":"2024-05-17T21:59:58.946037+0000","last_clean":"2024-05-17T21:59:58.946037+0000","last_became_active":"2024-05-17T21:59:56.552010+0000","last_became_peered":"2024-05-17T21:59:56.552010+0000","last_unstale":"2024-05-17T21:59:58.946037+0000","last_undegraded":"2024-05-17T21:59:58.946037+0000","last_fullsized":"2024-05-17T21:59:58.946037+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947864+0000","last_change":"2024-05-17T21:59:57.572902+0000","last_active":"2024-05-17T21:59:58.947864+0000","last_peered":"2024-05-17T21:59:58.947864+0000","last_clean":"2024-05-17T21:59:58.947864+0000","last_became_active":"2024-05-17T21:59:57.572552+0000","last_became_peered":"2024-05-17T21:59:57.572552+0000","last_unstale":"2024-05-17T21:59:58.947864+0000","last_undegraded":"2024-05-17T21:59:58.947864+0000","last_fullsized":"2024-05-17T21:59:58.947864+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:15.034728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949957+0000","last_change":"2024-05-17T21:59:56.549351+0000","last_active":"2024-05-17T21:59:58.949957+0000","last_peered":"2024-05-17T21:59:58.949957+0000","last_clean":"2024-05-17T21:59:58.949957+0000","last_became_active":"2024-05-17T21:59:56.548950+0000","last_became_peered":"2024-05-17T21:59:56.548950+0000","last_unstale":"2024-05-17T21:59:58.949957+0000","last_undegraded":"2024-05-17T21:59:58.949957+0000","last_fullsized":"2024-05-17T21:59:58.949957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:29.210821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945125+0000","last_change":"2024-05-17T21:59:57.572350+0000","last_active":"2024-05-17T21:59:58.945125+0000","last_peered":"2024-05-17T21:59:58.945125+0000","last_clean":"2024-05-17T21:59:58.945125+0000","last_became_active":"2024-05-17T21:59:57.572022+0000","last_became_peered":"2024-05-17T21:59:57.572022+0000","last_unstale":"2024-05-17T21:59:58.945125+0000","last_undegraded":"2024-05-17T21:59:58.945125+0000","last_fullsized":"2024-05-17T21:59:58.945125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946425+0000","last_change":"2024-05-17T21:59:56.547759+0000","last_active":"2024-05-17T21:59:58.946425+0000","last_peered":"2024-05-17T21:59:58.946425+0000","last_clean":"2024-05-17T21:59:58.946425+0000","last_became_active":"2024-05-17T21:59:56.547230+0000","last_became_peered":"2024-05-17T21:59:56.547230+0000","last_unstale":"2024-05-17T21:59:58.946425+0000","last_undegraded":"2024-05-17T21:59:58.946425+0000","last_fullsized":"2024-05-17T21:59:58.946425+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:54.834567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946032+0000","last_change":"2024-05-17T21:59:57.607085+0000","last_active":"2024-05-17T21:59:58.946032+0000","last_peered":"2024-05-17T21:59:58.946032+0000","last_clean":"2024-05-17T21:59:58.946032+0000","last_became_active":"2024-05-17T21:59:57.606756+0000","last_became_peered":"2024-05-17T21:59:57.606756+0000","last_unstale":"2024-05-17T21:59:58.946032+0000","last_undegraded":"2024-05-17T21:59:58.946032+0000","last_fullsized":"2024-05-17T21:59:58.946032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588814+0000","last_change":"2024-05-17T21:59:56.551338+0000","last_active":"2024-05-17T21:59:58.588814+0000","last_peered":"2024-05-17T21:59:58.588814+0000","last_clean":"2024-05-17T21:59:58.588814+0000","last_became_active":"2024-05-17T21:59:56.550399+0000","last_became_peered":"2024-05-17T21:59:56.550399+0000","last_unstale":"2024-05-17T21:59:58.588814+0000","last_undegraded":"2024-05-17T21:59:58.588814+0000","last_fullsized":"2024-05-17T21:59:58.588814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:54.834567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948794+0000","last_change":"2024-05-17T21:59:57.581922+0000","last_active":"2024-05-17T21:59:58.948794+0000","last_peered":"2024-05-17T21:59:58.948794+0000","last_clean":"2024-05-17T21:59:58.948794+0000","last_became_active":"2024-05-17T21:59:57.581584+0000","last_became_peered":"2024-05-17T21:59:57.581584+0000","last_unstale":"2024-05-17T21:59:58.948794+0000","last_undegraded":"2024-05-17T21:59:58.948794+0000","last_fullsized":"2024-05-17T21:59:58.948794+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:34:30.351774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585090+0000","last_change":"2024-05-17T21:59:56.561217+0000","last_active":"2024-05-17T21:59:58.585090+0000","last_peered":"2024-05-17T21:59:58.585090+0000","last_clean":"2024-05-17T21:59:58.585090+0000","last_became_active":"2024-05-17T21:59:56.560418+0000","last_became_peered":"2024-05-17T21:59:56.560418+0000","last_unstale":"2024-05-17T21:59:58.585090+0000","last_undegraded":"2024-05-17T21:59:58.585090+0000","last_fullsized":"2024-05-17T21:59:58.585090+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949628+0000","last_change":"2024-05-17T21:59:57.581796+0000","last_active":"2024-05-17T21:59:58.949628+0000","last_peered":"2024-05-17T21:59:58.949628+0000","last_clean":"2024-05-17T21:59:58.949628+0000","last_became_active":"2024-05-17T21:59:57.577817+0000","last_became_peered":"2024-05-17T21:59:57.577817+0000","last_unstale":"2024-05-17T21:59:58.949628+0000","last_undegraded":"2024-05-17T21:59:58.949628+0000","last_fullsized":"2024-05-17T21:59:58.949628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:30:31.320505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947349+0000","last_change":"2024-05-17T21:59:56.570765+0000","last_active":"2024-05-17T21:59:58.947349+0000","last_peered":"2024-05-17T21:59:58.947349+0000","last_clean":"2024-05-17T21:59:58.947349+0000","last_became_active":"2024-05-17T21:59:56.570307+0000","last_became_peered":"2024-05-17T21:59:56.570307+0000","last_unstale":"2024-05-17T21:59:58.947349+0000","last_undegraded":"2024-05-17T21:59:58.947349+0000","last_fullsized":"2024-05-17T21:59:58.947349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947212+0000","last_change":"2024-05-17T21:59:57.617436+0000","last_active":"2024-05-17T21:59:58.947212+0000","last_peered":"2024-05-17T21:59:58.947212+0000","last_clean":"2024-05-17T21:59:58.947212+0000","last_became_active":"2024-05-17T21:59:57.616638+0000","last_became_peered":"2024-05-17T21:59:57.616638+0000","last_unstale":"2024-05-17T21:59:58.947212+0000","last_undegraded":"2024-05-17T21:59:58.947212+0000","last_fullsized":"2024-05-17T21:59:58.947212+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:12:53.321187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587551+0000","last_change":"2024-05-17T21:59:56.562496+0000","last_active":"2024-05-17T21:59:58.587551+0000","last_peered":"2024-05-17T21:59:58.587551+0000","last_clean":"2024-05-17T21:59:58.587551+0000","last_became_active":"2024-05-17T21:59:56.561630+0000","last_became_peered":"2024-05-17T21:59:56.561630+0000","last_unstale":"2024-05-17T21:59:58.587551+0000","last_undegraded":"2024-05-17T21:59:58.587551+0000","last_fullsized":"2024-05-17T21:59:58.587551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948922+0000","last_change":"2024-05-17T21:59:57.591717+0000","last_active":"2024-05-17T21:59:58.948922+0000","last_peered":"2024-05-17T21:59:58.948922+0000","last_clean":"2024-05-17T21:59:58.948922+0000","last_became_active":"2024-05-17T21:59:57.591390+0000","last_became_peered":"2024-05-17T21:59:57.591390+0000","last_unstale":"2024-05-17T21:59:58.948922+0000","last_undegraded":"2024-05-17T21:59:58.948922+0000","last_fullsized":"2024-05-17T21:59:58.948922+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:39:43.381519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.584964+0000","last_change":"2024-05-17T21:59:56.557893+0000","last_active":"2024-05-17T21:59:58.584964+0000","last_peered":"2024-05-17T21:59:58.584964+0000","last_clean":"2024-05-17T21:59:58.584964+0000","last_became_active":"2024-05-17T21:59:56.557057+0000","last_became_peered":"2024-05-17T21:59:56.557057+0000","last_unstale":"2024-05-17T21:59:58.584964+0000","last_undegraded":"2024-05-17T21:59:58.584964+0000","last_fullsized":"2024-05-17T21:59:58.584964+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:39:34.437552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585813+0000","last_change":"2024-05-17T21:59:57.584253+0000","last_active":"2024-05-17T21:59:58.585813+0000","last_peered":"2024-05-17T21:59:58.585813+0000","last_clean":"2024-05-17T21:59:58.585813+0000","last_became_active":"2024-05-17T21:59:57.583170+0000","last_became_peered":"2024-05-17T21:59:57.583170+0000","last_unstale":"2024-05-17T21:59:58.585813+0000","last_undegraded":"2024-05-17T21:59:58.585813+0000","last_fullsized":"2024-05-17T21:59:58.585813+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:43:54.750624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947608+0000","last_change":"2024-05-17T21:59:56.564800+0000","last_active":"2024-05-17T21:59:58.947608+0000","last_peered":"2024-05-17T21:59:58.947608+0000","last_clean":"2024-05-17T21:59:58.947608+0000","last_became_active":"2024-05-17T21:59:56.563934+0000","last_became_peered":"2024-05-17T21:59:56.563934+0000","last_unstale":"2024-05-17T21:59:58.947608+0000","last_undegraded":"2024-05-17T21:59:58.947608+0000","last_fullsized":"2024-05-17T21:59:58.947608+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:52.645333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947783+0000","last_change":"2024-05-17T21:59:57.612674+0000","last_active":"2024-05-17T21:59:58.947783+0000","last_peered":"2024-05-17T21:59:58.947783+0000","last_clean":"2024-05-17T21:59:58.947783+0000","last_became_active":"2024-05-17T21:59:57.612004+0000","last_became_peered":"2024-05-17T21:59:57.612004+0000","last_unstale":"2024-05-17T21:59:58.947783+0000","last_undegraded":"2024-05-17T21:59:58.947783+0000","last_fullsized":"2024-05-17T21:59:58.947783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:39:05.022711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.950115+0000","last_change":"2024-05-17T21:59:56.544055+0000","last_active":"2024-05-17T21:59:58.950115+0000","last_peered":"2024-05-17T21:59:58.950115+0000","last_clean":"2024-05-17T21:59:58.950115+0000","last_became_active":"2024-05-17T21:59:56.543730+0000","last_became_peered":"2024-05-17T21:59:56.543730+0000","last_unstale":"2024-05-17T21:59:58.950115+0000","last_undegraded":"2024-05-17T21:59:58.950115+0000","last_fullsized":"2024-05-17T21:59:58.950115+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:54.155845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947074+0000","last_change":"2024-05-17T21:59:57.585508+0000","last_active":"2024-05-17T21:59:58.947074+0000","last_peered":"2024-05-17T21:59:58.947074+0000","last_clean":"2024-05-17T21:59:58.947074+0000","last_became_active":"2024-05-17T21:59:57.585158+0000","last_became_peered":"2024-05-17T21:59:57.585158+0000","last_unstale":"2024-05-17T21:59:58.947074+0000","last_undegraded":"2024-05-17T21:59:58.947074+0000","last_fullsized":"2024-05-17T21:59:58.947074+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:30:53.659004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947773+0000","last_change":"2024-05-17T21:59:56.554990+0000","last_active":"2024-05-17T21:59:58.947773+0000","last_peered":"2024-05-17T21:59:58.947773+0000","last_clean":"2024-05-17T21:59:58.947773+0000","last_became_active":"2024-05-17T21:59:56.554213+0000","last_became_peered":"2024-05-17T21:59:56.554213+0000","last_unstale":"2024-05-17T21:59:58.947773+0000","last_undegraded":"2024-05-17T21:59:58.947773+0000","last_fullsized":"2024-05-17T21:59:58.947773+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946456+0000","last_change":"2024-05-17T21:59:57.593127+0000","last_active":"2024-05-17T21:59:58.946456+0000","last_peered":"2024-05-17T21:59:58.946456+0000","last_clean":"2024-05-17T21:59:58.946456+0000","last_became_active":"2024-05-17T21:59:57.586477+0000","last_became_peered":"2024-05-17T21:59:57.586477+0000","last_unstale":"2024-05-17T21:59:58.946456+0000","last_undegraded":"2024-05-17T21:59:58.946456+0000","last_fullsized":"2024-05-17T21:59:58.946456+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'6","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585784+0000","last_change":"2024-05-17T21:59:56.566611+0000","last_active":"2024-05-17T21:59:58.585784+0000","last_peered":"2024-05-17T21:59:58.585784+0000","last_clean":"2024-05-17T21:59:58.585784+0000","last_became_active":"2024-05-17T21:59:56.565817+0000","last_became_peered":"2024-05-17T21:59:56.565817+0000","last_unstale":"2024-05-17T21:59:58.585784+0000","last_undegraded":"2024-05-17T21:59:58.585784+0000","last_fullsized":"2024-05-17T21:59:58.585784+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:23.236663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587739+0000","last_change":"2024-05-17T21:59:57.610368+0000","last_active":"2024-05-17T21:59:58.587739+0000","last_peered":"2024-05-17T21:59:58.587739+0000","last_clean":"2024-05-17T21:59:58.587739+0000","last_became_active":"2024-05-17T21:59:57.609571+0000","last_became_peered":"2024-05-17T21:59:57.609571+0000","last_unstale":"2024-05-17T21:59:58.587739+0000","last_undegraded":"2024-05-17T21:59:58.587739+0000","last_fullsized":"2024-05-17T21:59:58.587739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:59:56.429573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949289+0000","last_change":"2024-05-17T21:59:56.551263+0000","last_active":"2024-05-17T21:59:58.949289+0000","last_peered":"2024-05-17T21:59:58.949289+0000","last_clean":"2024-05-17T21:59:58.949289+0000","last_became_active":"2024-05-17T21:59:56.550941+0000","last_became_peered":"2024-05-17T21:59:56.550941+0000","last_unstale":"2024-05-17T21:59:58.949289+0000","last_undegraded":"2024-05-17T21:59:58.949289+0000","last_fullsized":"2024-05-17T21:59:58.949289+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:09.847811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946186+0000","last_change":"2024-05-17T21:59:57.592589+0000","last_active":"2024-05-17T21:59:58.946186+0000","last_peered":"2024-05-17T21:59:58.946186+0000","last_clean":"2024-05-17T21:59:58.946186+0000","last_became_active":"2024-05-17T21:59:57.591014+0000","last_became_peered":"2024-05-17T21:59:57.591014+0000","last_unstale":"2024-05-17T21:59:58.946186+0000","last_undegraded":"2024-05-17T21:59:58.946186+0000","last_fullsized":"2024-05-17T21:59:58.946186+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945753+0000","last_change":"2024-05-17T21:59:56.559255+0000","last_active":"2024-05-17T21:59:58.945753+0000","last_peered":"2024-05-17T21:59:58.945753+0000","last_clean":"2024-05-17T21:59:58.945753+0000","last_became_active":"2024-05-17T21:59:56.557360+0000","last_became_peered":"2024-05-17T21:59:56.557360+0000","last_unstale":"2024-05-17T21:59:58.945753+0000","last_undegraded":"2024-05-17T21:59:58.945753+0000","last_fullsized":"2024-05-17T21:59:58.945753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:49:24.540087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585342+0000","last_change":"2024-05-17T21:59:57.571310+0000","last_active":"2024-05-17T21:59:58.585342+0000","last_peered":"2024-05-17T21:59:58.585342+0000","last_clean":"2024-05-17T21:59:58.585342+0000","last_became_active":"2024-05-17T21:59:57.570449+0000","last_became_peered":"2024-05-17T21:59:57.570449+0000","last_unstale":"2024-05-17T21:59:58.585342+0000","last_undegraded":"2024-05-17T21:59:58.585342+0000","last_fullsized":"2024-05-17T21:59:58.585342+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:09:10.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587728+0000","last_change":"2024-05-17T21:59:56.553311+0000","last_active":"2024-05-17T21:59:58.587728+0000","last_peered":"2024-05-17T21:59:58.587728+0000","last_clean":"2024-05-17T21:59:58.587728+0000","last_became_active":"2024-05-17T21:59:56.552507+0000","last_became_peered":"2024-05-17T21:59:56.552507+0000","last_unstale":"2024-05-17T21:59:58.587728+0000","last_undegraded":"2024-05-17T21:59:58.587728+0000","last_fullsized":"2024-05-17T21:59:58.587728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:54.275995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946436+0000","last_change":"2024-05-17T21:59:57.594685+0000","last_active":"2024-05-17T21:59:58.946436+0000","last_peered":"2024-05-17T21:59:58.946436+0000","last_clean":"2024-05-17T21:59:58.946436+0000","last_became_active":"2024-05-17T21:59:57.593847+0000","last_became_peered":"2024-05-17T21:59:57.593847+0000","last_unstale":"2024-05-17T21:59:58.946436+0000","last_undegraded":"2024-05-17T21:59:58.946436+0000","last_fullsized":"2024-05-17T21:59:58.946436+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:25.470040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945633+0000","last_change":"2024-05-17T21:59:56.572177+0000","last_active":"2024-05-17T21:59:58.945633+0000","last_peered":"2024-05-17T21:59:58.945633+0000","last_clean":"2024-05-17T21:59:58.945633+0000","last_became_active":"2024-05-17T21:59:56.571143+0000","last_became_peered":"2024-05-17T21:59:56.571143+0000","last_unstale":"2024-05-17T21:59:58.945633+0000","last_undegraded":"2024-05-17T21:59:58.945633+0000","last_fullsized":"2024-05-17T21:59:58.945633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:18:12.537142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946910+0000","last_change":"2024-05-17T21:59:57.606299+0000","last_active":"2024-05-17T21:59:58.946910+0000","last_peered":"2024-05-17T21:59:58.946910+0000","last_clean":"2024-05-17T21:59:58.946910+0000","last_became_active":"2024-05-17T21:59:57.605269+0000","last_became_peered":"2024-05-17T21:59:57.605269+0000","last_unstale":"2024-05-17T21:59:58.946910+0000","last_undegraded":"2024-05-17T21:59:58.946910+0000","last_fullsized":"2024-05-17T21:59:58.946910+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:23.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.950389+0000","last_change":"2024-05-17T21:59:56.543327+0000","last_active":"2024-05-17T21:59:58.950389+0000","last_peered":"2024-05-17T21:59:58.950389+0000","last_clean":"2024-05-17T21:59:58.950389+0000","last_became_active":"2024-05-17T21:59:56.543007+0000","last_became_peered":"2024-05-17T21:59:56.543007+0000","last_unstale":"2024-05-17T21:59:58.950389+0000","last_undegraded":"2024-05-17T21:59:58.950389+0000","last_fullsized":"2024-05-17T21:59:58.950389+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:34.453336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588837+0000","last_change":"2024-05-17T21:59:57.609683+0000","last_active":"2024-05-17T21:59:58.588837+0000","last_peered":"2024-05-17T21:59:58.588837+0000","last_clean":"2024-05-17T21:59:58.588837+0000","last_became_active":"2024-05-17T21:59:57.606723+0000","last_became_peered":"2024-05-17T21:59:57.606723+0000","last_unstale":"2024-05-17T21:59:58.588837+0000","last_undegraded":"2024-05-17T21:59:58.588837+0000","last_fullsized":"2024-05-17T21:59:58.588837+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947214+0000","last_change":"2024-05-17T21:59:56.553813+0000","last_active":"2024-05-17T21:59:58.947214+0000","last_peered":"2024-05-17T21:59:58.947214+0000","last_clean":"2024-05-17T21:59:58.947214+0000","last_became_active":"2024-05-17T21:59:56.553343+0000","last_became_peered":"2024-05-17T21:59:56.553343+0000","last_unstale":"2024-05-17T21:59:58.947214+0000","last_undegraded":"2024-05-17T21:59:58.947214+0000","last_fullsized":"2024-05-17T21:59:58.947214+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:14.021056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949658+0000","last_change":"2024-05-17T21:59:57.560239+0000","last_active":"2024-05-17T21:59:58.949658+0000","last_peered":"2024-05-17T21:59:58.949658+0000","last_clean":"2024-05-17T21:59:58.949658+0000","last_became_active":"2024-05-17T21:59:57.559789+0000","last_became_peered":"2024-05-17T21:59:57.559789+0000","last_unstale":"2024-05-17T21:59:58.949658+0000","last_undegraded":"2024-05-17T21:59:58.949658+0000","last_fullsized":"2024-05-17T21:59:58.949658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:57:10.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588775+0000","last_change":"2024-05-17T21:59:56.555291+0000","last_active":"2024-05-17T21:59:58.588775+0000","last_peered":"2024-05-17T21:59:58.588775+0000","last_clean":"2024-05-17T21:59:58.588775+0000","last_became_active":"2024-05-17T21:59:56.554565+0000","last_became_peered":"2024-05-17T21:59:56.554565+0000","last_unstale":"2024-05-17T21:59:58.588775+0000","last_undegraded":"2024-05-17T21:59:58.588775+0000","last_fullsized":"2024-05-17T21:59:58.588775+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:38:03.296880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947124+0000","last_change":"2024-05-17T21:59:57.571363+0000","last_active":"2024-05-17T21:59:58.947124+0000","last_peered":"2024-05-17T21:59:58.947124+0000","last_clean":"2024-05-17T21:59:58.947124+0000","last_became_active":"2024-05-17T21:59:57.570532+0000","last_became_peered":"2024-05-17T21:59:57.570532+0000","last_unstale":"2024-05-17T21:59:58.947124+0000","last_undegraded":"2024-05-17T21:59:58.947124+0000","last_fullsized":"2024-05-17T21:59:58.947124+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:51.977412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945379+0000","last_change":"2024-05-17T21:59:56.545943+0000","last_active":"2024-05-17T21:59:58.945379+0000","last_peered":"2024-05-17T21:59:58.945379+0000","last_clean":"2024-05-17T21:59:58.945379+0000","last_became_active":"2024-05-17T21:59:56.545484+0000","last_became_peered":"2024-05-17T21:59:56.545484+0000","last_unstale":"2024-05-17T21:59:58.945379+0000","last_undegraded":"2024-05-17T21:59:58.945379+0000","last_fullsized":"2024-05-17T21:59:58.945379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587766+0000","last_change":"2024-05-17T21:59:57.605763+0000","last_active":"2024-05-17T21:59:58.587766+0000","last_peered":"2024-05-17T21:59:58.587766+0000","last_clean":"2024-05-17T21:59:58.587766+0000","last_became_active":"2024-05-17T21:59:57.605008+0000","last_became_peered":"2024-05-17T21:59:57.605008+0000","last_unstale":"2024-05-17T21:59:58.587766+0000","last_undegraded":"2024-05-17T21:59:58.587766+0000","last_fullsized":"2024-05-17T21:59:58.587766+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949947+0000","last_change":"2024-05-17T21:59:56.565665+0000","last_active":"2024-05-17T21:59:58.949947+0000","last_peered":"2024-05-17T21:59:58.949947+0000","last_clean":"2024-05-17T21:59:58.949947+0000","last_became_active":"2024-05-17T21:59:56.565311+0000","last_became_peered":"2024-05-17T21:59:56.565311+0000","last_unstale":"2024-05-17T21:59:58.949947+0000","last_undegraded":"2024-05-17T21:59:58.949947+0000","last_fullsized":"2024-05-17T21:59:58.949947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:22.717000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946733+0000","last_change":"2024-05-17T21:59:57.591073+0000","last_active":"2024-05-17T21:59:58.946733+0000","last_peered":"2024-05-17T21:59:58.946733+0000","last_clean":"2024-05-17T21:59:58.946733+0000","last_became_active":"2024-05-17T21:59:57.590088+0000","last_became_peered":"2024-05-17T21:59:57.590088+0000","last_unstale":"2024-05-17T21:59:58.946733+0000","last_undegraded":"2024-05-17T21:59:58.946733+0000","last_fullsized":"2024-05-17T21:59:58.946733+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:27:23.730672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945478+0000","last_change":"2024-05-17T21:59:56.531280+0000","last_active":"2024-05-17T21:59:58.945478+0000","last_peered":"2024-05-17T21:59:58.945478+0000","last_clean":"2024-05-17T21:59:58.945478+0000","last_became_active":"2024-05-17T21:59:56.530987+0000","last_became_peered":"2024-05-17T21:59:56.530987+0000","last_unstale":"2024-05-17T21:59:58.945478+0000","last_undegraded":"2024-05-17T21:59:58.945478+0000","last_fullsized":"2024-05-17T21:59:58.945478+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:17.374303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.944455+0000","last_change":"2024-05-17T21:59:57.574579+0000","last_active":"2024-05-17T21:59:58.944455+0000","last_peered":"2024-05-17T21:59:58.944455+0000","last_clean":"2024-05-17T21:59:58.944455+0000","last_became_active":"2024-05-17T21:59:57.572362+0000","last_became_peered":"2024-05-17T21:59:57.572362+0000","last_unstale":"2024-05-17T21:59:58.944455+0000","last_undegraded":"2024-05-17T21:59:58.944455+0000","last_fullsized":"2024-05-17T21:59:58.944455+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:13:48.986799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.587613+0000","last_change":"2024-05-17T21:59:56.546993+0000","last_active":"2024-05-17T21:59:58.587613+0000","last_peered":"2024-05-17T21:59:58.587613+0000","last_clean":"2024-05-17T21:59:58.587613+0000","last_became_active":"2024-05-17T21:59:56.545808+0000","last_became_peered":"2024-05-17T21:59:56.545808+0000","last_unstale":"2024-05-17T21:59:58.587613+0000","last_undegraded":"2024-05-17T21:59:58.587613+0000","last_fullsized":"2024-05-17T21:59:58.587613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:26.273454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588814+0000","last_change":"2024-05-17T21:59:57.609714+0000","last_active":"2024-05-17T21:59:58.588814+0000","last_peered":"2024-05-17T21:59:58.588814+0000","last_clean":"2024-05-17T21:59:58.588814+0000","last_became_active":"2024-05-17T21:59:57.606514+0000","last_became_peered":"2024-05-17T21:59:57.606514+0000","last_unstale":"2024-05-17T21:59:58.588814+0000","last_undegraded":"2024-05-17T21:59:58.588814+0000","last_fullsized":"2024-05-17T21:59:58.588814+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:58:10.861482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947883+0000","last_change":"2024-05-17T21:59:56.543524+0000","last_active":"2024-05-17T21:59:58.947883+0000","last_peered":"2024-05-17T21:59:58.947883+0000","last_clean":"2024-05-17T21:59:58.947883+0000","last_became_active":"2024-05-17T21:59:56.543182+0000","last_became_peered":"2024-05-17T21:59:56.543182+0000","last_unstale":"2024-05-17T21:59:58.947883+0000","last_undegraded":"2024-05-17T21:59:58.947883+0000","last_fullsized":"2024-05-17T21:59:58.947883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:22:49.005800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.588978+0000","last_change":"2024-05-17T21:59:57.619081+0000","last_active":"2024-05-17T21:59:58.588978+0000","last_peered":"2024-05-17T21:59:58.588978+0000","last_clean":"2024-05-17T21:59:58.588978+0000","last_became_active":"2024-05-17T21:59:57.618439+0000","last_became_peered":"2024-05-17T21:59:57.618439+0000","last_unstale":"2024-05-17T21:59:58.588978+0000","last_undegraded":"2024-05-17T21:59:58.588978+0000","last_fullsized":"2024-05-17T21:59:58.588978+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:39:35.467007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947291+0000","last_change":"2024-05-17T21:59:56.568281+0000","last_active":"2024-05-17T21:59:58.947291+0000","last_peered":"2024-05-17T21:59:58.947291+0000","last_clean":"2024-05-17T21:59:58.947291+0000","last_became_active":"2024-05-17T21:59:56.567962+0000","last_became_peered":"2024-05-17T21:59:56.567962+0000","last_unstale":"2024-05-17T21:59:58.947291+0000","last_undegraded":"2024-05-17T21:59:58.947291+0000","last_fullsized":"2024-05-17T21:59:58.947291+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:09:09.351072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946926+0000","last_change":"2024-05-17T21:59:57.585662+0000","last_active":"2024-05-17T21:59:58.946926+0000","last_peered":"2024-05-17T21:59:58.946926+0000","last_clean":"2024-05-17T21:59:58.946926+0000","last_became_active":"2024-05-17T21:59:57.585231+0000","last_became_peered":"2024-05-17T21:59:57.585231+0000","last_unstale":"2024-05-17T21:59:58.946926+0000","last_undegraded":"2024-05-17T21:59:58.946926+0000","last_fullsized":"2024-05-17T21:59:58.946926+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:27.287126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.947025+0000","last_change":"2024-05-17T21:59:56.567011+0000","last_active":"2024-05-17T21:59:58.947025+0000","last_peered":"2024-05-17T21:59:58.947025+0000","last_clean":"2024-05-17T21:59:58.947025+0000","last_became_active":"2024-05-17T21:59:56.561654+0000","last_became_peered":"2024-05-17T21:59:56.561654+0000","last_unstale":"2024-05-17T21:59:58.947025+0000","last_undegraded":"2024-05-17T21:59:58.947025+0000","last_fullsized":"2024-05-17T21:59:58.947025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:57:29.278746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946344+0000","last_change":"2024-05-17T21:59:57.606005+0000","last_active":"2024-05-17T21:59:58.946344+0000","last_peered":"2024-05-17T21:59:58.946344+0000","last_clean":"2024-05-17T21:59:58.946344+0000","last_became_active":"2024-05-17T21:59:57.605649+0000","last_became_peered":"2024-05-17T21:59:57.605649+0000","last_unstale":"2024-05-17T21:59:58.946344+0000","last_undegraded":"2024-05-17T21:59:58.946344+0000","last_fullsized":"2024-05-17T21:59:58.946344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:53:55.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.589128+0000","last_change":"2024-05-17T21:59:56.563094+0000","last_active":"2024-05-17T21:59:58.589128+0000","last_peered":"2024-05-17T21:59:58.589128+0000","last_clean":"2024-05-17T21:59:58.589128+0000","last_became_active":"2024-05-17T21:59:56.561722+0000","last_became_peered":"2024-05-17T21:59:56.561722+0000","last_unstale":"2024-05-17T21:59:58.589128+0000","last_undegraded":"2024-05-17T21:59:58.589128+0000","last_fullsized":"2024-05-17T21:59:58.589128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:14.021056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949214+0000","last_change":"2024-05-17T21:59:57.581568+0000","last_active":"2024-05-17T21:59:58.949214+0000","last_peered":"2024-05-17T21:59:58.949214+0000","last_clean":"2024-05-17T21:59:58.949214+0000","last_became_active":"2024-05-17T21:59:57.568279+0000","last_became_peered":"2024-05-17T21:59:57.568279+0000","last_unstale":"2024-05-17T21:59:58.949214+0000","last_undegraded":"2024-05-17T21:59:58.949214+0000","last_fullsized":"2024-05-17T21:59:58.949214+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:06.103008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945026+0000","last_change":"2024-05-17T21:59:56.545923+0000","last_active":"2024-05-17T21:59:58.945026+0000","last_peered":"2024-05-17T21:59:58.945026+0000","last_clean":"2024-05-17T21:59:58.945026+0000","last_became_active":"2024-05-17T21:59:56.545578+0000","last_became_peered":"2024-05-17T21:59:56.545578+0000","last_unstale":"2024-05-17T21:59:58.945026+0000","last_undegraded":"2024-05-17T21:59:58.945026+0000","last_fullsized":"2024-05-17T21:59:58.945026+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:54:13.110045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.946653+0000","last_change":"2024-05-17T21:59:57.595435+0000","last_active":"2024-05-17T21:59:58.946653+0000","last_peered":"2024-05-17T21:59:58.946653+0000","last_clean":"2024-05-17T21:59:58.946653+0000","last_became_active":"2024-05-17T21:59:57.594503+0000","last_became_peered":"2024-05-17T21:59:57.594503+0000","last_unstale":"2024-05-17T21:59:58.946653+0000","last_undegraded":"2024-05-17T21:59:58.946653+0000","last_fullsized":"2024-05-17T21:59:58.946653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:30.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.585875+0000","last_change":"2024-05-17T21:59:56.560679+0000","last_active":"2024-05-17T21:59:58.585875+0000","last_peered":"2024-05-17T21:59:58.585875+0000","last_clean":"2024-05-17T21:59:58.585875+0000","last_became_active":"2024-05-17T21:59:56.559426+0000","last_became_peered":"2024-05-17T21:59:56.559426+0000","last_unstale":"2024-05-17T21:59:58.585875+0000","last_undegraded":"2024-05-17T21:59:58.585875+0000","last_fullsized":"2024-05-17T21:59:58.585875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:46:17.681880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945650+0000","last_change":"2024-05-17T21:59:57.589684+0000","last_active":"2024-05-17T21:59:58.945650+0000","last_peered":"2024-05-17T21:59:58.945650+0000","last_clean":"2024-05-17T21:59:58.945650+0000","last_became_active":"2024-05-17T21:59:57.589373+0000","last_became_peered":"2024-05-17T21:59:57.589373+0000","last_unstale":"2024-05-17T21:59:58.945650+0000","last_undegraded":"2024-05-17T21:59:58.945650+0000","last_fullsized":"2024-05-17T21:59:58.945650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:24.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945237+0000","last_change":"2024-05-17T21:59:56.548697+0000","last_active":"2024-05-17T21:59:58.945237+0000","last_peered":"2024-05-17T21:59:58.945237+0000","last_clean":"2024-05-17T21:59:58.945237+0000","last_became_active":"2024-05-17T21:59:56.548414+0000","last_became_peered":"2024-05-17T21:59:56.548414+0000","last_unstale":"2024-05-17T21:59:58.945237+0000","last_undegraded":"2024-05-17T21:59:58.945237+0000","last_fullsized":"2024-05-17T21:59:58.945237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:50:05.496567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948975+0000","last_change":"2024-05-17T21:59:57.591063+0000","last_active":"2024-05-17T21:59:58.948975+0000","last_peered":"2024-05-17T21:59:58.948975+0000","last_clean":"2024-05-17T21:59:58.948975+0000","last_became_active":"2024-05-17T21:59:57.590681+0000","last_became_peered":"2024-05-17T21:59:57.590681+0000","last_unstale":"2024-05-17T21:59:58.948975+0000","last_undegraded":"2024-05-17T21:59:58.948975+0000","last_fullsized":"2024-05-17T21:59:58.948975+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:18:28.589127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.949318+0000","last_change":"2024-05-17T21:59:56.545007+0000","last_active":"2024-05-17T21:59:58.949318+0000","last_peered":"2024-05-17T21:59:58.949318+0000","last_clean":"2024-05-17T21:59:58.949318+0000","last_became_active":"2024-05-17T21:59:56.543719+0000","last_became_peered":"2024-05-17T21:59:56.543719+0000","last_unstale":"2024-05-17T21:59:58.949318+0000","last_undegraded":"2024-05-17T21:59:58.949318+0000","last_fullsized":"2024-05-17T21:59:58.949318+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:24.456369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.948804+0000","last_change":"2024-05-17T21:59:57.584583+0000","last_active":"2024-05-17T21:59:58.948804+0000","last_peered":"2024-05-17T21:59:58.948804+0000","last_clean":"2024-05-17T21:59:58.948804+0000","last_became_active":"2024-05-17T21:59:57.577984+0000","last_became_peered":"2024-05-17T21:59:57.577984+0000","last_unstale":"2024-05-17T21:59:58.948804+0000","last_undegraded":"2024-05-17T21:59:58.948804+0000","last_fullsized":"2024-05-17T21:59:58.948804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:56.513271+0000","last_clean_scrub_stamp":"2024-05-17T21:59:56.513271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:20:35.522434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945990+0000","last_change":"2024-05-17T21:59:56.557845+0000","last_active":"2024-05-17T21:59:58.945990+0000","last_peered":"2024-05-17T21:59:58.945990+0000","last_clean":"2024-05-17T21:59:58.945990+0000","last_became_active":"2024-05-17T21:59:56.556982+0000","last_became_peered":"2024-05-17T21:59:56.556982+0000","last_unstale":"2024-05-17T21:59:58.945990+0000","last_undegraded":"2024-05-17T21:59:58.945990+0000","last_fullsized":"2024-05-17T21:59:58.945990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:59:55.499600+0000","last_clean_scrub_stamp":"2024-05-17T21:59:55.499600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:44:17.374303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":175,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-05-17T21:59:58.945725+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T21:59:58.945725+0000","last_peered":"2024-05-17T21:59:58.945725+0000","last_clean":"2024-05-17T21:59:58.945725+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T21:59:58.945725+0000","last_undegraded":"2024-05-17T21:59:58.945725+0000","last_fullsized":"2024-05-17T21:59:58.945725+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":862211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":271683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":272335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":273295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":271739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":667648,"data_stored":271739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":663552,"data_stored":271773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":864245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:00:02.050 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:00:02.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T22:00:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 Processor -- start 2024-05-17T22:00:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 -- start start 2024-05-17T22:00:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 0x7f55e00aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55e006f5a0 0x7f55e0160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e00b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f55e0160e80 con 0x7f55e006f5a0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f55e00b6040 con 0x7f55e00b5aa0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e6add640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55e00b6a90 con 0x7f55e0160a70 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55e006f5a0 0x7f55e0160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f55e006f5a0 0x7f55e0160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47948/0 (socket says 172.21.15.83:47948) 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 -- 172.21.15.83:0/1891147964 learned_addr learned my addr 172.21.15.83:0/1891147964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e4879640 1 --2- 172.21.15.83:0/1891147964 >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 0x7f55e00aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e507a640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e00b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 -- 172.21.15.83:0/1891147964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 msgr2=0x7f55e00b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e00b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 -- 172.21.15.83:0/1891147964 >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 msgr2=0x7f55e00aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 --2- 172.21.15.83:0/1891147964 >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 0x7f55e00aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55dffff640 1 -- 172.21.15.83:0/1891147964 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e00b6d70 con 0x7f55e006f5a0 2024-05-17T22:00:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e4879640 1 --2- 172.21.15.83:0/1891147964 >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 0x7f55e00aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.252+0000 7f55e507a640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e00b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55dffff640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e0160530 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f55e0074b60 tx=0x7f55d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f791dc75376927 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55df7fe640 1 -- 172.21.15.83:0/1891147964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40b0070 con 0x7f55e006f5a0 2024-05-17T22:00:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55df7fe640 1 -- 172.21.15.83:0/1891147964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d40ab600 con 0x7f55e006f5a0 2024-05-17T22:00:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55df7fe640 1 -- 172.21.15.83:0/1891147964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40ab9d0 con 0x7f55e006f5a0 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 -- 172.21.15.83:0/1891147964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 msgr2=0x7f55e0160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e0160530 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f55e0074b60 tx=0x7f55d401c490 comp rx=0 tx=0).stop 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 -- 172.21.15.83:0/1891147964 shutdown_connections 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e00b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 --2- 172.21.15.83:0/1891147964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e0160530 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 --2- 172.21.15.83:0/1891147964 >> v2:172.21.15.155:3300/0 conn(0x7f55e00b5aa0 0x7f55e00aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 -- 172.21.15.83:0/1891147964 >> 172.21.15.83:0/1891147964 conn(0x7f55e00d20b0 msgr2=0x7f55e00d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 -- 172.21.15.83:0/1891147964 shutdown_connections 2024-05-17T22:00:02.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.256+0000 7f55e6add640 1 -- 172.21.15.83:0/1891147964 wait complete. 2024-05-17T22:00:02.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 Processor -- start 2024-05-17T22:00:02.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- start start 2024-05-17T22:00:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55e00b5aa0 0x7f55e00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01613a0 con 0x7f55e006f5a0 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0161220 con 0x7f55e00b5aa0 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f55e015cef0 con 0x7f55e0160a70 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47962/0 (socket says 172.21.15.83:47962) 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 -- 172.21.15.83:0/700981376 learned_addr learned my addr 172.21.15.83:0/700981376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e507a640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55dffff640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55e00b5aa0 0x7f55e00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 -- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 msgr2=0x7f55e014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 -- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55e00b5aa0 msgr2=0x7f55e00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55e00b5aa0 0x7f55e00cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 -- 172.21.15.83:0/700981376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e014dd90 con 0x7f55e006f5a0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e4879640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c820 tx=0x7f55d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0015070 con 0x7f55e006f5a0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d0007a30 con 0x7f55e006f5a0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0007e20 con 0x7f55e006f5a0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e014e070 con 0x7f55e006f5a0 2024-05-17T22:00:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.260+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e014e630 con 0x7f55e006f5a0 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e00aec60 con 0x7f55e006f5a0 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f55d0018070 con 0x7f55e006f5a0 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55dd7fa640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f55d008f5b0 con 0x7f55e006f5a0 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55dffff640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.264+0000 7f55dffff640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 0x7f55cc071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55d40ad000 tx=0x7f55d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.268+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f55d005c580 con 0x7f55e006f5a0 2024-05-17T22:00:02.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.472+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f55e00d1680 con 0x7f55e006f5a0 2024-05-17T22:00:02.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.476+0000 7f55dd7fa640 1 -- 172.21.15.83:0/700981376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f55d005be80 con 0x7f55e006f5a0 2024-05-17T22:00:02.470 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T22:00:02.471 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T22:00:02.471 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T22:00:02.471 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T22:00:02.471 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T22:00:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T22:00:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 msgr2=0x7f55cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 0x7f55cc071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55d40ad000 tx=0x7f55d4008040 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 msgr2=0x7f55e01499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c820 tx=0x7f55d000cce0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 shutdown_connections 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55e0160a70 0x7f55e014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f55cc06f400 0x7f55cc071910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55e00b5aa0 0x7f55e00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 --2- 172.21.15.83:0/700981376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55e006f5a0 0x7f55e01499c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 >> 172.21.15.83:0/700981376 conn(0x7f55e00d20b0 msgr2=0x7f55e006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 shutdown_connections 2024-05-17T22:00:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.480+0000 7f55e6add640 1 -- 172.21.15.83:0/700981376 wait complete. 2024-05-17T22:00:02.490 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.636+0000 7fbaac7f0640 1 Processor -- start 2024-05-17T22:00:02.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaac7f0640 1 -- start start 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaac7f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbaa415d2f0 0x7fbaa4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaac7f0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa415c830 con 0x7fbaa4073540 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaac7f0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa41608a0 con 0x7fbaa415fe20 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaac7f0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa4160b80 con 0x7fbaa415d2f0 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaad8d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbaa4073540 0x7fbaa40b0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50280/0 (socket says 172.21.15.83:50280) 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaad8d640 1 -- 172.21.15.83:0/970281065 learned_addr learned my addr 172.21.15.83:0/970281065 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaa58c640 1 --2- 172.21.15.83:0/970281065 >> v2:172.21.15.83:3300/0 conn(0x7fbaa415d2f0 0x7fbaa4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaa958a640 1 -- 172.21.15.83:0/970281065 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470930491 0 0) 0x7fbaa415c830 con 0x7fbaa4073540 2024-05-17T22:00:02.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaa958a640 1 -- 172.21.15.83:0/970281065 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba90003530 con 0x7fbaa4073540 2024-05-17T22:00:02.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaa58c640 1 -- 172.21.15.83:0/970281065 >> v1:172.21.15.155:6789/0 conn(0x7fbaa415fe20 legacy=0x7fbaa4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:02.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaa58c640 1 -- 172.21.15.83:0/970281065 >> v1:172.21.15.83:6789/0 conn(0x7fbaa4073540 legacy=0x7fbaa40b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.640+0000 7fbaaa58c640 1 -- 172.21.15.83:0/970281065 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa415caf0 con 0x7fbaa415d2f0 2024-05-17T22:00:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaaa58c640 1 --2- 172.21.15.83:0/970281065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa4159c20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097c0 tx=0x7fbaa001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21aeff66c947ecfa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaa958a640 1 -- 172.21.15.83:0/970281065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00b0070 con 0x7fbaa415d2f0 2024-05-17T22:00:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaa958a640 1 -- 172.21.15.83:0/970281065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa00ab600 con 0x7fbaa415d2f0 2024-05-17T22:00:02.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaa958a640 1 -- 172.21.15.83:0/970281065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00ab9d0 con 0x7fbaa415d2f0 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- 172.21.15.83:0/970281065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 msgr2=0x7fbaa4159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/970281065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa4159c20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097c0 tx=0x7fbaa001c4a0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- 172.21.15.83:0/970281065 shutdown_connections 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/970281065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa4159c20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- 172.21.15.83:0/970281065 >> 172.21.15.83:0/970281065 conn(0x7fbaa406cd90 msgr2=0x7fbaa40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:02.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- 172.21.15.83:0/970281065 shutdown_connections 2024-05-17T22:00:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- 172.21.15.83:0/970281065 wait complete. 2024-05-17T22:00:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 Processor -- start 2024-05-17T22:00:02.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.644+0000 7fbaac7f0640 1 -- start start 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbaa415fe20 0x7fbaa41501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaa415b7e0 0x7fbaa4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40b1010 con 0x7fbaa415d2f0 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40b0e90 con 0x7fbaa415b7e0 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa40cf1d0 con 0x7fbaa415fe20 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47974/0 (socket says 172.21.15.83:47974) 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 -- 172.21.15.83:0/393014754 learned_addr learned my addr 172.21.15.83:0/393014754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:02.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaad8d640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaa415b7e0 0x7fbaa4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 -- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbaa415fe20 msgr2=0x7fbaa41501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaa9d8b640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbaa415fe20 0x7fbaa41501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbaa415fe20 0x7fbaa41501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 -- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaa415b7e0 msgr2=0x7fbaa4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaa415b7e0 0x7fbaa4155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 -- 172.21.15.83:0/393014754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40aa850 con 0x7fbaa415d2f0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaaa58c640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00ae040 tx=0x7fbaa001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00b0050 con 0x7fbaa415d2f0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa00bc070 con 0x7fbaa415d2f0 2024-05-17T22:00:02.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00b5870 con 0x7fbaa415d2f0 2024-05-17T22:00:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa415f2f0 con 0x7fbaa415d2f0 2024-05-17T22:00:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa4155440 con 0x7fbaa415d2f0 2024-05-17T22:00:02.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4159c20 con 0x7fbaa415d2f0 2024-05-17T22:00:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbaa00b5a10 con 0x7fbaa415d2f0 2024-05-17T22:00:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.648+0000 7fba9affd640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 0x7fba88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.652+0000 7fbaa9d8b640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 0x7fba88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.652+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fbaa0129ab0 con 0x7fbaa415d2f0 2024-05-17T22:00:02.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.652+0000 7fbaa9d8b640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 0x7fba88071910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406d770 tx=0x7fba94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:02.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.656+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbaa0016090 con 0x7fbaa415d2f0 2024-05-17T22:00:02.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.832+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaa406fda0 con 0x7fbaa415d2f0 2024-05-17T22:00:02.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fba9affd640 1 -- 172.21.15.83:0/393014754 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5184 (secure 0 0 0) 0x7fbaa00f6990 con 0x7fbaa415d2f0 2024-05-17T22:00:02.828 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:02.829 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T21:59:59.970206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:02.829 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-05-17T22:00:02.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 msgr2=0x7fba88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 0x7fba88071910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbaa406d770 tx=0x7fba94008040 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 msgr2=0x7fbaa40aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00ae040 tx=0x7fbaa001ace0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 shutdown_connections 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba8806f400 0x7fba88071910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbaa415b7e0 0x7fbaa4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbaa415fe20 0x7fbaa41501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 --2- 172.21.15.83:0/393014754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa415d2f0 0x7fbaa40aa310 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 >> 172.21.15.83:0/393014754 conn(0x7fbaa406cd90 msgr2=0x7fbaa40b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 shutdown_connections 2024-05-17T22:00:02.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:02.836+0000 7fbaac7f0640 1 -- 172.21.15.83:0/393014754 wait complete. 2024-05-17T22:00:03.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T21:59:56.927208+0000', 'modified': '2024-05-17T21:59:59.970206+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/3998902165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 3998902165}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 3998902165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5580, 'qdb_cluster': [5580]} 2024-05-17T22:00:03.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:00:03.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-17T22:00:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.416+0000 7fad86b1c640 1 Processor -- start 2024-05-17T22:00:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.416+0000 7fad86b1c640 1 -- start start 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad86b1c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad800d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad86b1c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad80073590 0x7fad801447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad86b1c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800d10d0 con 0x7fad80143f20 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad86b1c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fad800b4110 con 0x7fad80073590 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad86b1c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad80070770 con 0x7fad800b3d00 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad80073590 0x7fad801447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad7ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fad80143f20 0x7fad80072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50282/0 (socket says 172.21.15.83:50282) 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad7ffff640 1 -- 172.21.15.83:0/1418449647 learned_addr learned my addr 172.21.15.83:0/1418449647 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad848b8640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad800d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 -- 172.21.15.83:0/1418449647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 msgr2=0x7fad800d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad800d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 -- 172.21.15.83:0/1418449647 >> v1:172.21.15.83:6789/0 conn(0x7fad80143f20 legacy=0x7fad80072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 -- 172.21.15.83:0/1418449647 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad80070120 con 0x7fad80073590 2024-05-17T22:00:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad7f7fe640 1 -- 172.21.15.83:0/1418449647 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787331384 0 0) 0x7fad800d10d0 con 0x7fad80143f20 2024-05-17T22:00:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad850b9640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad801447f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad78009d90 tx=0x7fad7801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4268833ab657a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad7f7fe640 1 -- 172.21.15.83:0/1418449647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780b2070 con 0x7fad80073590 2024-05-17T22:00:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.420+0000 7fad7f7fe640 1 -- 172.21.15.83:0/1418449647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad780a58c0 con 0x7fad80073590 2024-05-17T22:00:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad7f7fe640 1 -- 172.21.15.83:0/1418449647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780a5c90 con 0x7fad80073590 2024-05-17T22:00:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- 172.21.15.83:0/1418449647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 msgr2=0x7fad801447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad801447f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad78009d90 tx=0x7fad7801c970 comp rx=0 tx=0).stop 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- 172.21.15.83:0/1418449647 shutdown_connections 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad801447f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 --2- 172.21.15.83:0/1418449647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad800d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- 172.21.15.83:0/1418449647 >> 172.21.15.83:0/1418449647 conn(0x7fad8006cd90 msgr2=0x7fad800b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- 172.21.15.83:0/1418449647 shutdown_connections 2024-05-17T22:00:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- 172.21.15.83:0/1418449647 wait complete. 2024-05-17T22:00:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 Processor -- start 2024-05-17T22:00:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.424+0000 7fad86b1c640 1 -- start start 2024-05-17T22:00:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad8015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad80143f20 0x7fad801605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad80070c00 con 0x7fad80073590 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad80145540 con 0x7fad80143f20 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad8013b130 con 0x7fad800b3d00 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47990/0 (socket says 172.21.15.83:47990) 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 -- 172.21.15.83:0/2110574694 learned_addr learned my addr 172.21.15.83:0/2110574694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 -- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 msgr2=0x7fad8015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad8015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 -- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad80143f20 msgr2=0x7fad801605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad850b9640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad80143f20 0x7fad801605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad80143f20 0x7fad801605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 -- 172.21.15.83:0/2110574694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad800b00f0 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad848b8640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d620 tx=0x7fad7000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad7d7fa640 1 -- 172.21.15.83:0/2110574694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000b620 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad7d7fa640 1 -- 172.21.15.83:0/2110574694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70019070 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad7d7fa640 1 -- 172.21.15.83:0/2110574694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000b9e0 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad800b03d0 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8015b6c0 con 0x7fad80073590 2024-05-17T22:00:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.428+0000 7fad7d7fa640 1 -- 172.21.15.83:0/2110574694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad7000bb80 con 0x7fad80073590 2024-05-17T22:00:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad7d7fa640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 0x7fad6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad7d7fa640 1 -- 172.21.15.83:0/2110574694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fad7008f6d0 con 0x7fad80073590 2024-05-17T22:00:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad7ffff640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 0x7fad6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad7ffff640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 0x7fad6c071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad801fa210 tx=0x7fad74007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 Processor -- start 2024-05-17T22:00:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- start start 2024-05-17T22:00:03.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d58df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d59330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594b7bf6ca0 con 0x5594b7c12fa0 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594b7b69110 con 0x5594b7bf1d30 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5594b7b58cb0 con 0x5594b7c08600 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47998/0 (socket says 172.21.15.83:47998) 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 -- 172.21.15.83:0/367240142 learned_addr learned my addr 172.21.15.83:0/367240142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad7ffff640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d58df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad850b9640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d59330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 -- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 msgr2=0x5594b7d58df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d58df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 -- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 msgr2=0x5594b7d59330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d59330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 -- 172.21.15.83:0/367240142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b7b1d370 con 0x5594b7c12fa0 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad850b9640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad848b8640 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fad7000b0e0 tx=0x7fad7005b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbf1444949eb15b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad667fc640 1 -- 172.21.15.83:0/367240142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7005c650 con 0x5594b7c12fa0 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad667fc640 1 -- 172.21.15.83:0/367240142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7005bf50 con 0x5594b7c12fa0 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad667fc640 1 -- 172.21.15.83:0/367240142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70011dd0 con 0x5594b7c12fa0 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- 172.21.15.83:0/367240142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 msgr2=0x5594b7d588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fad7000b0e0 tx=0x7fad7005b6c0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- 172.21.15.83:0/367240142 shutdown_connections 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d59330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d58df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 --2- 172.21.15.83:0/367240142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d588b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- 172.21.15.83:0/367240142 >> 172.21.15.83:0/367240142 conn(0x5594b781a460 msgr2=0x5594b7d0de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- 172.21.15.83:0/367240142 shutdown_connections 2024-05-17T22:00:03.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.432+0000 7fad8dd96000 1 -- 172.21.15.83:0/367240142 wait complete. 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- start start 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 Processor -- start 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594b7b14d40 con 0x5594b7c12fa0 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594b7b5ccf0 con 0x5594b7bf1d30 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5594b7b01e90 con 0x5594b7c08600 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 10 client.0 _ll_register_callbacks cb 0x5594b7b36fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48006/0 (socket says 172.21.15.83:48006) 2024-05-17T22:00:03.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 -- 172.21.15.83:0/2663707038 learned_addr learned my addr 172.21.15.83:0/2663707038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad848b8640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad7ffff640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 -- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 msgr2=0x5594b7d157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 -- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 msgr2=0x5594b7d15280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d15280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 -- 172.21.15.83:0/2663707038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b7b276e0 con 0x5594b7c12fa0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad850b9640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad780b0040 tx=0x7fad7801b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce90b4e54268dd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780bd020 con 0x5594b7c12fa0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fad780a56c0 con 0x5594b7c12fa0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7005bf50 con 0x5594b7c12fa0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad780c7050 con 0x5594b7c12fa0 2024-05-17T22:00:03.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 10 client.5721 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5594b7b17690 con 0x5594b7c12fa0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7005bf50 con 0x5594b7c12fa0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fad780ca020 con 0x5594b7c12fa0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 10 client.5721 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 10 client.5721 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 20 client.5721 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ff7d0 0x5594b7d5f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 4 client.5721 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b7af4a00 con 0x5594b77ff7d0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.436+0000 7fad848b8640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ff7d0 0x5594b7d5f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad5f7fe640 10 client.5721 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad848b8640 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ff7d0 0x5594b7d5f920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a740da668bf52d server_cookie=c8a46fe624d92f in_seq=0 out_seq=0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2663707038 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5594b7af4a00 con 0x5594b77ff7d0 2024-05-17T22:00:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad5f7fe640 10 client.5721 handle_command_reply: tid=0 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 client.5721 shutdown 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 msgr2=0x5594b7d1a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fad780b0040 tx=0x7fad7801b1b0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 shutdown_connections 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7d1a460 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c08600 0x5594b7d157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ff7d0 0x5594b7d5f920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2663707038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7bf1d30 0x5594b7d15280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad5f7fe640 0 client.5721 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 >> 172.21.15.83:0/2663707038 conn(0x5594b781a460 msgr2=0x5594b7d0de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 shutdown_connections 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 1 -- 172.21.15.83:0/2663707038 wait complete. 2024-05-17T22:00:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.440+0000 7fad8dd96000 20 client.5721 trim_cache size 0 max 16384 2024-05-17T22:00:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.456+0000 7fad8dd96000 1 Processor -- start 2024-05-17T22:00:03.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.456+0000 7fad8dd96000 1 -- start start 2024-05-17T22:00:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.456+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7d703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c12fa0 0x5594b7d14230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad80143270 con 0x5594b7bf1d30 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fad801425d0 con 0x5594b7c08600 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fad80142c20 con 0x5594b7c12fa0 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48016/0 (socket says 172.21.15.83:48016) 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 -- 172.21.15.83:0/3465831921 learned_addr learned my addr 172.21.15.83:0/3465831921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 -- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c12fa0 msgr2=0x5594b7d14230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c12fa0 0x5594b7d14230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 -- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 msgr2=0x5594b7d703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7d703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 -- 172.21.15.83:0/3465831921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b7bdfcb0 con 0x5594b7bf1d30 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad848b8640 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad7005bee0 tx=0x7fad70020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cd745747eec8e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad65ffb640 1 -- 172.21.15.83:0/3465831921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7005bf50 con 0x5594b7bf1d30 2024-05-17T22:00:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad65ffb640 1 -- 172.21.15.83:0/3465831921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70060300 con 0x5594b7bf1d30 2024-05-17T22:00:03.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad65ffb640 1 -- 172.21.15.83:0/3465831921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70011e10 con 0x5594b7bf1d30 2024-05-17T22:00:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- 172.21.15.83:0/3465831921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 msgr2=0x5594b7d6feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fad7005bee0 tx=0x7fad70020000 comp rx=0 tx=0).stop 2024-05-17T22:00:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- 172.21.15.83:0/3465831921 shutdown_connections 2024-05-17T22:00:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7c12fa0 0x5594b7d14230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7d703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- 172.21.15.83:0/3465831921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7bf1d30 0x5594b7d6feb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- 172.21.15.83:0/3465831921 >> 172.21.15.83:0/3465831921 conn(0x5594b781a460 msgr2=0x5594b7d57d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- 172.21.15.83:0/3465831921 shutdown_connections 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- 172.21.15.83:0/3465831921 wait complete. 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- start start 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 Processor -- start 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7bf1d30 0x5594b7dcb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7dcb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594b7b5c000 con 0x5594b7c12fa0 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5594b7d18c20 con 0x5594b7c08600 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5594b7b67ee0 con 0x5594b7bf1d30 2024-05-17T22:00:03.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad8dd96000 10 client.0 _ll_register_callbacks cb 0x7fad780b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad850b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad850b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48030/0 (socket says 172.21.15.83:48030) 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.460+0000 7fad850b9640 1 -- 172.21.15.83:0/2202549234 learned_addr learned my addr 172.21.15.83:0/2202549234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 -- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7bf1d30 msgr2=0x5594b7dcb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7bf1d30 0x5594b7dcb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 -- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 msgr2=0x5594b7dcb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7dcb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 -- 172.21.15.83:0/2202549234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b7b184f0 con 0x5594b7c12fa0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad850b9640 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad78007130 tx=0x7fad78019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11554aed27ff42f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780bd070 con 0x5594b7c12fa0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fad780a56c0 con 0x5594b7c12fa0 2024-05-17T22:00:03.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad70060300 con 0x5594b7c12fa0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fad780b2070 con 0x5594b7c12fa0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 10 client.5727 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5594b7b224f0 con 0x5594b7c12fa0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70060300 con 0x5594b7c12fa0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fad780ca070 con 0x5594b7c12fa0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 10 client.5727 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 10 client.5727 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 20 client.5727 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ed830 0x5594b7d0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 4 client.5727 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "status"}) -- 0x5594b7b6b060 con 0x5594b77ed830 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad848b8640 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ed830 0x5594b7d0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 10 client.5727 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:03.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad848b8640 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ed830 0x5594b7d0f8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8d1504f28b07fa server_cookie=c5363e27c37f88b2 in_seq=0 out_seq=0 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 1 -- 172.21.15.83:0/2202549234 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5594b7b6b060 con 0x5594b77ed830 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 10 client.5727 handle_command_reply: tid=0 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 client.5727 shutdown 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 msgr2=0x5594b7dcbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad78007130 tx=0x7fad78019710 comp rx=0 tx=0).stop 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 shutdown_connections 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594b7c12fa0 0x5594b7dcbad0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5594b77ed830 0x5594b7d0f8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5594b7c08600 0x5594b7dcb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 --2- 172.21.15.83:0/2202549234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5594b7bf1d30 0x5594b7dcb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 >> 172.21.15.83:0/2202549234 conn(0x5594b781a460 msgr2=0x5594b7d6eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad5f7fe640 0 client.5727 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:03.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 shutdown_connections 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 1 -- 172.21.15.83:0/2202549234 wait complete. 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad8dd96000 20 client.5727 trim_cache size 0 max 16384 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5580, 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.917955139, 2024-05-17T22:00:03.463 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 32, 2024-05-17T22:00:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 56, 2024-05-17T22:00:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 56, 2024-05-17T22:00:03.464 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.860194688 2024-05-17T22:00:03.464 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:03.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 msgr2=0x7fad6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 0x7fad6c071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad801fa210 tx=0x7fad74007970 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 msgr2=0x7fad8015eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d620 tx=0x7fad7000dae0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 shutdown_connections 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fad6c06f400 0x7fad6c071910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad80143f20 0x7fad801605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad800b3d00 0x7fad8015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 --2- 172.21.15.83:0/2110574694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad80073590 0x7fad8015eb50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 >> 172.21.15.83:0/2110574694 conn(0x7fad8006cd90 msgr2=0x7fad80075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 shutdown_connections 2024-05-17T22:00:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.464+0000 7fad86b1c640 1 -- 172.21.15.83:0/2110574694 wait complete. 2024-05-17T22:00:03.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:00:03.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.612+0000 7f2278254640 1 Processor -- start 2024-05-17T22:00:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 -- start start 2024-05-17T22:00:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f227015d2f0 0x7f2270159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f2270160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227015c830 con 0x7f22700b39e0 2024-05-17T22:00:03.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22701608a0 con 0x7f227015d2f0 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2278254640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2270160b80 con 0x7f227015fe20 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f227015d2f0 0x7f2270159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f227015d2f0 0x7f2270159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48042/0 (socket says 172.21.15.83:48042) 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 -- 172.21.15.83:0/3174357940 learned_addr learned my addr 172.21.15.83:0/3174357940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f22757ef640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f2270160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 -- 172.21.15.83:0/3174357940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 msgr2=0x7f2270160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f2270160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 -- 172.21.15.83:0/3174357940 >> v1:172.21.15.155:6789/0 conn(0x7f22700b39e0 legacy=0x7f2270072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f2275ff0640 1 -- 172.21.15.83:0/3174357940 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227015caf0 con 0x7f227015d2f0 2024-05-17T22:00:03.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.616+0000 7f22757ef640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f2270160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:03.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2275ff0640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270159c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2264009870 tx=0x7f226401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2d26e2ddf637e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2274fee640 1 -- 172.21.15.83:0/3174357940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640b2070 con 0x7f227015d2f0 2024-05-17T22:00:03.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2274fee640 1 -- 172.21.15.83:0/3174357940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22640a9070 con 0x7f227015d2f0 2024-05-17T22:00:03.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2274fee640 1 -- 172.21.15.83:0/3174357940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22640ad880 con 0x7f227015d2f0 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 -- 172.21.15.83:0/3174357940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 msgr2=0x7f2270159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270159c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2264009870 tx=0x7f226401c4d0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 -- 172.21.15.83:0/3174357940 shutdown_connections 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f2270160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 --2- 172.21.15.83:0/3174357940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270159c20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 -- 172.21.15.83:0/3174357940 >> 172.21.15.83:0/3174357940 conn(0x7f227006cd90 msgr2=0x7f22700b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 -- 172.21.15.83:0/3174357940 shutdown_connections 2024-05-17T22:00:03.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.620+0000 7f2278254640 1 -- 172.21.15.83:0/3174357940 wait complete. 2024-05-17T22:00:03.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 Processor -- start 2024-05-17T22:00:03.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- start start 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 0x7f2270154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f22701553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22700736c0 con 0x7f227015d2f0 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2270073540 con 0x7f22700b39e0 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f22700b3df0 con 0x7f227015fe20 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48046/0 (socket says 172.21.15.83:48046) 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 -- 172.21.15.83:0/4185012034 learned_addr learned my addr 172.21.15.83:0/4185012034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2275ff0640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 0x7f2270154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22767f1640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f22701553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 -- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 msgr2=0x7f22701553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f22701553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 -- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 msgr2=0x7f2270154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 0x7f2270154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 -- 172.21.15.83:0/4185012034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22700aa930 con 0x7f227015d2f0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2275ff0640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 0x7f2270154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22757ef640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c820 tx=0x7f226c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c015070 con 0x7f227015d2f0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c007a30 con 0x7f227015d2f0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22700aa650 con 0x7f227015d2f0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c007e20 con 0x7f227015d2f0 2024-05-17T22:00:03.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.624+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2270158f00 con 0x7f227015d2f0 2024-05-17T22:00:03.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270159c20 con 0x7f227015d2f0 2024-05-17T22:00:03.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f226c018070 con 0x7f227015d2f0 2024-05-17T22:00:03.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f22627fc640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 0x7f2254071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:03.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f2275ff0640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 0x7f2254071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:03.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f226c08f0e0 con 0x7f227015d2f0 2024-05-17T22:00:03.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.628+0000 7f2275ff0640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 0x7f2254071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22640af000 tx=0x7f2264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:03.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.632+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f226c05c060 con 0x7f227015d2f0 2024-05-17T22:00:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.816+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2270160360 con 0x7f227015d2f0 2024-05-17T22:00:03.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.816+0000 7f22627fc640 1 -- 172.21.15.83:0/4185012034 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7f226c05b960 con 0x7f227015d2f0 2024-05-17T22:00:03.810 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 msgr2=0x7f2254071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 0x7f2254071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22640af000 tx=0x7f2264008040 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 msgr2=0x7f2270154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c820 tx=0x7f226c00cce0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 shutdown_connections 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f227015fe20 0x7f22701553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227015d2f0 0x7f2270154e90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f225406f400 0x7f2254071910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 --2- 172.21.15.83:0/4185012034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22700b39e0 0x7f2270154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 >> 172.21.15.83:0/4185012034 conn(0x7f227006cd90 msgr2=0x7f22700b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 shutdown_connections 2024-05-17T22:00:03.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:03.820+0000 7f2278254640 1 -- 172.21.15.83:0/4185012034 wait complete. 2024-05-17T22:00:03.830 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:00:03.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:00:03.830 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:00:03.830 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:00:03.830 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:03.831 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:00:03.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:00:03.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:00:03.831 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:00:03.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:03.831 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:00:03.838 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:00:03.839 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42340sec preferred_lft 42340sec 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:00:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:00:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:03.842 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:00:03.889 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:00:03.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:03.890 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:00:03.939 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:00:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:00:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:00:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:00:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:00:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:00:03.940 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:00:04.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:00:04.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:00:04.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:04.060 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:00:04.112 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:00:04.112 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:00:04.112 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:00:04.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:04.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:00:04.168 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:00:04.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:00:04.168 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:04.216 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:00:04.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:00:04.216 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:04.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:00:04.321 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:00:04.369 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:00:04.370 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:04.371 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:04.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:04.372 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:00:04.420 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:00:04.420 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:00:04.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:00:04.479 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:00:04.529 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:00:04.530 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:04.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:04.531 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:00:04.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:00:04.584+0000 7f3e27667580 -1 init, newargv = 0x55c593e5ccc0 newargc=15 2024-05-17T22:00:04.576 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[39467]: starting ceph client 2024-05-17T22:00:04.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[39467]: starting fuse 2024-05-17T22:00:05.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:00:05.692 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:00:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:00:05.742 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:00:05.743 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:05.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:05.744 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:00:05.792 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:00:05.792 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:00:05.793 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:00:05.794 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:00:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.39467.asok 2024-05-17T22:00:05.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:00:05.927 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39467.asok 2024-05-17T22:00:05.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:05.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39467.asok status 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:00:06.104 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "39467", 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5739, 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 5739 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1977786540 2024-05-17T22:00:06.105 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1977786540 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.5739 192.168.0.1:0/1977786540", 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1977786540", 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:00:06.106 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 32, 2024-05-17T22:00:06.107 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 56, 2024-05-17T22:00:06.107 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:00:06.107 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:00:06.107 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:00:06.107 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:06.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:06.122 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:00:06.122 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:06.128 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:00:06.128 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:06.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:06.128 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:06.183 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-17T22:00:06.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:06.245 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-17T22:00:06.299 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:00:06.406 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T22:00:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:00:06.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 Processor -- start 2024-05-17T22:00:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 -- start start 2024-05-17T22:00:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d4c1689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c0700e0 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d4c0aff10 con 0x7f1d4c0af7c0 2024-05-17T22:00:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d50a82640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c071590 con 0x7f1d4c1665d0 2024-05-17T22:00:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d49d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1d4c06d7a0 0x7f1d4c06dbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:50284/0 (socket says 172.21.15.83:50284) 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.620+0000 7f1d49d9b640 1 -- 172.21.15.83:0/3353756361 learned_addr learned my addr 172.21.15.83:0/3353756361 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d4c1689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257998584 0 0) 0x7f1d4c0700e0 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d38003530 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2731684343 0 0) 0x7f1d38003530 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 msgr2=0x7f1d4c1689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d4c1689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 >> v1:172.21.15.155:6789/0 conn(0x7f1d4c0af7c0 legacy=0x7f1d4c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c0703a0 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1d3c001700 con 0x7f1d4c06d7a0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d4c1689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 >> v1:172.21.15.83:6789/0 conn(0x7f1d4c06d7a0 legacy=0x7f1d4c06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d38003a40 0x7f1d38003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d38004da0 0x7f1d38005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c071590 con 0x7f1d38004da0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3800bc00 con 0x7f1d38003a40 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3800be00 con 0x7f1d4c1665d0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d38004da0 0x7f1d38005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4a59c640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d38003a40 0x7f1d38003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d49d9b640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 -- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 msgr2=0x7f1d38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1665d0 0x7f1d38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 -- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d38003a40 msgr2=0x7f1d38003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d38003a40 0x7f1d38003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.624+0000 7f1d4ad9d640 1 -- 172.21.15.83:0/3353756361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d38003530 con 0x7f1d38004da0 2024-05-17T22:00:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d38004da0 0x7f1d38005270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c068ac0 tx=0x7f1d3401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a6d041d2ca6367 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400f040 con 0x7f1d38004da0 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340b0600 con 0x7f1d38004da0 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d4959a640 1 -- 172.21.15.83:0/3353756361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340b09b0 con 0x7f1d38004da0 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- 172.21.15.83:0/3353756361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d38004da0 msgr2=0x7f1d38005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 --2- 172.21.15.83:0/3353756361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d38004da0 0x7f1d38005270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c068ac0 tx=0x7f1d3401c590 comp rx=0 tx=0).stop 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d4ad9d640 1 -- 172.21.15.83:0/3353756361 reap_dead start 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- 172.21.15.83:0/3353756361 shutdown_connections 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- 172.21.15.83:0/3353756361 >> 172.21.15.83:0/3353756361 conn(0x7f1d4c073670 msgr2=0x7f1d4c073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- 172.21.15.83:0/3353756361 shutdown_connections 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- 172.21.15.83:0/3353756361 wait complete. 2024-05-17T22:00:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 Processor -- start 2024-05-17T22:00:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.628+0000 7f1d50a82640 1 -- start start 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 0x7f1d4c1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1f42f0 0x7f1d4c1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0af500 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c0709f0 con 0x7f1d4c15f470 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c15e8e0 con 0x7f1d4c1f42f0 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48066/0 (socket says 172.21.15.83:48066) 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 -- 172.21.15.83:0/1443548369 learned_addr learned my addr 172.21.15.83:0/1443548369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d49d9b640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 0x7f1d4c1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4ad9d640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1f42f0 0x7f1d4c1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 -- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1f42f0 msgr2=0x7f1d4c1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1f42f0 0x7f1d4c1f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 -- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 msgr2=0x7f1d4c1f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 0x7f1d4c1f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 -- 172.21.15.83:0/1443548369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c09df50 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d49d9b640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 0x7f1d4c1f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d4a59c640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e870 tx=0x7f1d4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003b90 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d44003d30 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c09e230 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44006da0 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c09e7f0 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c0afc60 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1d44005070 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 0x7f1d2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:06.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f1d4408efa0 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.632+0000 7f1d49d9b640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 0x7f1d2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:06.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.636+0000 7f1d49d9b640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 0x7f1d2c071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0019d0 tx=0x7f1d3c001960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:06.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.640+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1d4405c070 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.836+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4c158020 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.836+0000 7f1d42ffd640 1 -- 172.21.15.83:0/1443548369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5184 (secure 0 0 0) 0x7f1d44064060 con 0x7f1d4c09d9b0 2024-05-17T22:00:06.831 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:06.831 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T21:59:59.970206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:06.831 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 msgr2=0x7f1d2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 0x7f1d2c071910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0019d0 tx=0x7f1d3c001960 comp rx=0 tx=0).stop 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 msgr2=0x7f1d4c1f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e870 tx=0x7f1d4400ed30 comp rx=0 tx=0).stop 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 shutdown_connections 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d4c1f42f0 0x7f1d4c1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1d2c06f400 0x7f1d2c071910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d4c15f470 0x7f1d4c1f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 --2- 172.21.15.83:0/1443548369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d4c09d9b0 0x7f1d4c1f6d30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 >> 172.21.15.83:0/1443548369 conn(0x7f1d4c073670 msgr2=0x7f1d4c165cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 shutdown_connections 2024-05-17T22:00:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:06.840+0000 7f1d50a82640 1 -- 172.21.15.83:0/1443548369 wait complete. 2024-05-17T22:00:06.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-17T22:00:07.024 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:00:07.024 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:00:07.024 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-17T22:00:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.096+0000 7fc99434f080 1 Processor -- start 2024-05-17T22:00:07.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.096+0000 7fc99434f080 1 -- start start 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed26940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f08eb9b1b0 con 0x55f08ec4c3d0 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f08ebf6ef0 con 0x55f08ed239d0 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f08ec4cb90 con 0x55f08ed23de0 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52254/0 (socket says 172.21.15.83:52254) 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 -- 172.21.15.83:0/3565613019 learned_addr learned my addr 172.21.15.83:0/3565613019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98e632640 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed26940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98d630640 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 -- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 msgr2=0x55f08ed26940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed26940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 -- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 msgr2=0x55f08ed246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 -- 172.21.15.83:0/3565613019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08eb75c10 con 0x55f08ec4c3d0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc98de31640 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b790 tx=0x7fc98000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df697b358eb21f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc987fff640 1 -- 172.21.15.83:0/3565613019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980016020 con 0x55f08ec4c3d0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc987fff640 1 -- 172.21.15.83:0/3565613019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98000d9c0 con 0x55f08ec4c3d0 2024-05-17T22:00:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc987fff640 1 -- 172.21.15.83:0/3565613019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000dd90 con 0x55f08ec4c3d0 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- 172.21.15.83:0/3565613019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 msgr2=0x55f08ed20290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b790 tx=0x7fc98000bc50 comp rx=0 tx=0).stop 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- 172.21.15.83:0/3565613019 shutdown_connections 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed26940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 --2- 172.21.15.83:0/3565613019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed20290 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- 172.21.15.83:0/3565613019 >> 172.21.15.83:0/3565613019 conn(0x55f08ec37e40 msgr2=0x55f08ed228e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- 172.21.15.83:0/3565613019 shutdown_connections 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- 172.21.15.83:0/3565613019 wait complete. 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 Processor -- start 2024-05-17T22:00:07.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.100+0000 7fc99434f080 1 -- start start 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed27ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed02e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed0e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f08ed26780 con 0x55f08ec4c3d0 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f08ec7b440 con 0x55f08ed239d0 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f08ed0f110 con 0x55f08ed23de0 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed27ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52264/0 (socket says 172.21.15.83:52264) 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 -- 172.21.15.83:0/4293802413 learned_addr learned my addr 172.21.15.83:0/4293802413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98e632640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed0e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 -- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 msgr2=0x55f08ed0e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f08ed23de0 0x55f08ed0e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 -- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 msgr2=0x55f08ed02e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98d630640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed02e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08ed239d0 0x55f08ed02e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 -- 172.21.15.83:0/4293802413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08ed0f3b0 con 0x55f08ec4c3d0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98de31640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed27ca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d5a0 tx=0x7fc9800033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 -- 172.21.15.83:0/4293802413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980016040 con 0x55f08ec4c3d0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 -- 172.21.15.83:0/4293802413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980013cf0 con 0x55f08ec4c3d0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 -- 172.21.15.83:0/4293802413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98001b620 con 0x55f08ec4c3d0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f08ed0f650 con 0x55f08ec4c3d0 2024-05-17T22:00:07.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f08ed0fb90 con 0x55f08ec4c3d0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 -- 172.21.15.83:0/4293802413 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc980019020 con 0x55f08ec4c3d0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc97006dde0 0x7fc9700702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98d630640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc97006dde0 0x7fc9700702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc986ffd640 1 -- 172.21.15.83:0/4293802413 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc98008f930 con 0x55f08ec4c3d0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55f08ed0ec60 0x55f08ed10030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56) -- 0x55f08ed1f2a0 con 0x55f08ed0ec60 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.104+0000 7fc98d630640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc97006dde0 0x7fc9700702f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9780ac000 tx=0x7fc9780a5000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc98e632640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55f08ed0ec60 0x55f08ed10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc98e632640 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55f08ed0ec60 0x55f08ed10030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc98e632640 1 -- 172.21.15.83:0/4293802413 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fc9880103b0 con 0x55f08ed0ec60 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55f08ed0ec60 msgr2=0x55f08ed10030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55f08ed0ec60 0x55f08ed10030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc97006dde0 msgr2=0x7fc9700702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc97006dde0 0x7fc9700702f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc9780ac000 tx=0x7fc9780a5000 comp rx=0 tx=0).stop 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 msgr2=0x55f08ed27ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 --2- 172.21.15.83:0/4293802413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f08ec4c3d0 0x55f08ed27ca0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d5a0 tx=0x7fc9800033b0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc98e632640 1 -- 172.21.15.83:0/4293802413 reap_dead start 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 shutdown_connections 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 >> 172.21.15.83:0/4293802413 conn(0x55f08ec37e40 msgr2=0x55f08ed07530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 shutdown_connections 2024-05-17T22:00:07.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.108+0000 7fc99434f080 1 -- 172.21.15.83:0/4293802413 wait complete. 2024-05-17T22:00:07.107 INFO:teuthology.orchestra.run.smithi083.stdout:Xfiledir_test 2024-05-17T22:00:07.107 DEBUG:teuthology.orchestra.run.smithi083:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "ancestors": [ 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1099511627776, 2024-05-17T22:00:07.159 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "file", 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 2 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "dirino": 1, 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "dir_test", 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 4 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "pool": 9, 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: "old_pools": [] 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:07.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:07.164 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-17T22:00:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.276+0000 7f96c975a080 1 Processor -- start 2024-05-17T22:00:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.276+0000 7f96c975a080 1 -- start start 2024-05-17T22:00:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e5ac0 0x56354f1e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f1e6100 0x56354f256c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56354f0eb1b0 con 0x56354f25a290 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56354f146ef0 con 0x56354f1e5ac0 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56354f271f10 con 0x56354f1e6100 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52288/0 (socket says 172.21.15.83:52288) 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3571890831 learned_addr learned my addr 172.21.15.83:0/3571890831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f1e6100 msgr2=0x56354f256c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f1e6100 0x56354f256c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e5ac0 msgr2=0x56354f1e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e5ac0 0x56354f1e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3571890831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354f0c5c10 con 0x56354f25a290 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c3a3d640 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001600 tx=0x7f96bc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62e412f34bc9c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c1a39640 1 -- 172.21.15.83:0/3571890831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc0062f0 con 0x56354f25a290 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c1a39640 1 -- 172.21.15.83:0/3571890831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc003660 con 0x56354f25a290 2024-05-17T22:00:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c1a39640 1 -- 172.21.15.83:0/3571890831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc003a50 con 0x56354f25a290 2024-05-17T22:00:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- 172.21.15.83:0/3571890831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 msgr2=0x56354f255da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001600 tx=0x7f96bc001ac0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- 172.21.15.83:0/3571890831 shutdown_connections 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f25a290 0x56354f255da0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f1e6100 0x56354f256c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 --2- 172.21.15.83:0/3571890831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e5ac0 0x56354f1e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- 172.21.15.83:0/3571890831 >> 172.21.15.83:0/3571890831 conn(0x56354f187e40 msgr2=0x56354f19fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- 172.21.15.83:0/3571890831 shutdown_connections 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- 172.21.15.83:0/3571890831 wait complete. 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 Processor -- start 2024-05-17T22:00:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.280+0000 7f96c975a080 1 -- start start 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 0x56354f25c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e6100 0x56354f1caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f25a290 0x56354f1cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56354f2550e0 con 0x56354f1e5ac0 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56354f1c9a60 con 0x56354f1e6100 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56354f255380 con 0x56354f25a290 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 0x56354f25c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 0x56354f25c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52294/0 (socket says 172.21.15.83:52294) 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 -- 172.21.15.83:0/3489157278 learned_addr learned my addr 172.21.15.83:0/3489157278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c2a3b640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e6100 0x56354f1caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 -- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f25a290 msgr2=0x56354f1cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56354f25a290 0x56354f1cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 -- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e6100 msgr2=0x56354f1caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56354f1e6100 0x56354f1caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 -- 172.21.15.83:0/3489157278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354f25ab50 con 0x56354f1e5ac0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c323c640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 0x56354f25c960 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006fd0 tx=0x7f96b40057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 -- 172.21.15.83:0/3489157278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4017070 con 0x56354f1e5ac0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 -- 172.21.15.83:0/3489157278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4012410 con 0x56354f1e5ac0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56354f25adf0 con 0x56354f1e5ac0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 -- 172.21.15.83:0/3489157278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4012800 con 0x56354f1e5ac0 2024-05-17T22:00:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354f25b330 con 0x56354f1e5ac0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 -- 172.21.15.83:0/3489157278 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96b401a070 con 0x56354f1e5ac0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96a406dde0 0x7f96a40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c2a3b640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96a406dde0 0x7f96a40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c0a37640 1 -- 172.21.15.83:0/3489157278 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f96b408d260 con 0x56354f1e5ac0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x56354f25c430 0x56354f1e1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) -- 0x56354f1e1740 con 0x56354f25c430 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c2a3b640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96a406dde0 0x7f96a40702f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0ac000 tx=0x7f96ac003030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.284+0000 7f96c3a3d640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x56354f25c430 0x56354f1e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.288+0000 7f96c3a3d640 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x56354f25c430 0x56354f1e1200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3489157278 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v56'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96bc01b040 con 0x56354f25c430 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x56354f25c430 msgr2=0x56354f1e1200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x56354f25c430 0x56354f1e1200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96a406dde0 msgr2=0x7f96a40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f96a406dde0 0x7f96a40702f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0ac000 tx=0x7f96ac003030 comp rx=0 tx=0).stop 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 msgr2=0x56354f25c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 --2- 172.21.15.83:0/3489157278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56354f1e5ac0 0x56354f25c960 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96b4006fd0 tx=0x7f96b40057d0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c3a3d640 1 -- 172.21.15.83:0/3489157278 reap_dead start 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 shutdown_connections 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 >> 172.21.15.83:0/3489157278 conn(0x56354f187e40 msgr2=0x56354f251f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 shutdown_connections 2024-05-17T22:00:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.292+0000 7f96c975a080 1 -- 172.21.15.83:0/3489157278 wait complete. 2024-05-17T22:00:07.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:07.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-17T22:00:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 Processor -- start 2024-05-17T22:00:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 -- start start 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9b809c0a0 0x7fe9b80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b80cdff0 con 0x7fe9b809c570 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b80ce290 con 0x7fe9b80d1f50 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bf64d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b80ceea0 con 0x7fe9b809c0a0 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bcbe8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe9b809c570 0x7fe9b809b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54966/0 (socket says 172.21.15.83:54966) 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/4180732191 learned_addr learned my addr 172.21.15.83:0/4180732191 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/4180732191 >> v2:172.21.15.155:3300/0 conn(0x7fe9b809c0a0 0x7fe9b80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362186772 0 0) 0x7fe9b80cdff0 con 0x7fe9b809c570 2024-05-17T22:00:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a4003530 con 0x7fe9b809c570 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.436+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4074387130 0 0) 0x7fe9a4003530 con 0x7fe9b809c570 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 >> v2:172.21.15.155:3300/0 conn(0x7fe9b809c0a0 msgr2=0x7fe9b80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 --2- 172.21.15.83:0/4180732191 >> v2:172.21.15.155:3300/0 conn(0x7fe9b809c0a0 0x7fe9b80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 >> v1:172.21.15.155:6789/0 conn(0x7fe9b80d1f50 legacy=0x7fe9b80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b80ce570 con 0x7fe9b809c570 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe9ac001700 con 0x7fe9b809c570 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 >> v1:172.21.15.83:6789/0 conn(0x7fe9b809c570 legacy=0x7fe9b809b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a4003a60 0x7fe9a4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a4005260 0x7fe9a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80ceea0 con 0x7fe9a4004810 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a400c1f0 con 0x7fe9a4003a60 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a400c3f0 con 0x7fe9a4005260 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a4003a60 0x7fe9a4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a4005260 0x7fe9a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a4005260 msgr2=0x7fe9a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9a4005260 0x7fe9a400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a4003a60 msgr2=0x7fe9a4003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9a4003a60 0x7fe9a4003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/4180732191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a4003530 con 0x7fe9a4004810 2024-05-17T22:00:07.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bdbea640 1 -- 172.21.15.83:0/4180732191 reap_dead start 2024-05-17T22:00:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9a4004c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c240 tx=0x7fe9ac01f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5148e2ab4b10a20d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac007cf0 con 0x7fe9a4004810 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac005ae0 con 0x7fe9a4004810 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9abfff640 1 -- 172.21.15.83:0/4180732191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac007070 con 0x7fe9a4004810 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 -- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 msgr2=0x7fe9a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9a4004c20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c240 tx=0x7fe9ac01f2f0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 -- 172.21.15.83:0/4180732191 shutdown_connections 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/4180732191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9a4004c20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 -- 172.21.15.83:0/4180732191 >> 172.21.15.83:0/4180732191 conn(0x7fe9b80b5100 msgr2=0x7fe9b80b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 -- 172.21.15.83:0/4180732191 shutdown_connections 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.440+0000 7fe9bf64d640 1 -- 172.21.15.83:0/4180732191 wait complete. 2024-05-17T22:00:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 Processor -- start 2024-05-17T22:00:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- start start 2024-05-17T22:00:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9b806f390 0x7fe9b806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9b806dde0 0x7fe9b81f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80d2360 con 0x7fe9a4004810 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80ceb30 con 0x7fe9b806dde0 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80d24e0 con 0x7fe9b806f390 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52304/0 (socket says 172.21.15.83:52304) 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 -- 172.21.15.83:0/1841730004 learned_addr learned my addr 172.21.15.83:0/1841730004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 -- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9b806f390 msgr2=0x7fe9b806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9b806f390 0x7fe9b806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 -- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9b806dde0 msgr2=0x7fe9b81f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9b806dde0 0x7fe9b81f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9b806dde0 0x7fe9b81f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 -- 172.21.15.83:0/1841730004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b80b2360 con 0x7fe9a4004810 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e8f0 tx=0x7fe9b000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9a9ffb640 1 -- 172.21.15.83:0/1841730004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0003b90 con 0x7fe9a4004810 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9a9ffb640 1 -- 172.21.15.83:0/1841730004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b0018070 con 0x7fe9a4004810 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b80b12c0 con 0x7fe9a4004810 2024-05-17T22:00:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9a9ffb640 1 -- 172.21.15.83:0/1841730004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00050d0 con 0x7fe9a4004810 2024-05-17T22:00:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.444+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b80b1880 con 0x7fe9a4004810 2024-05-17T22:00:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9a9ffb640 1 -- 172.21.15.83:0/1841730004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9b0005270 con 0x7fe9a4004810 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9a9ffb640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 0x7fe99c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9a9ffb640 1 -- 172.21.15.83:0/1841730004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe9b008f300 con 0x7fe9a4004810 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 0x7fe99c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 0x7fe99c071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ae000 tx=0x7fe9b80ce460 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 Processor -- start 2024-05-17T22:00:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 -- start start 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 0x55dcc9b1e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dcc9930300 con 0x55dcc99ab6d0 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dcc991fea0 con 0x55dcc99cdc10 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dcc9923ee0 con 0x55dcc99bf450 2024-05-17T22:00:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9bdbea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9bdbea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52310/0 (socket says 172.21.15.83:52310) 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.448+0000 7fe9bdbea640 1 -- 172.21.15.83:0/3825288717 learned_addr learned my addr 172.21.15.83:0/3825288717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 0x55dcc9b1e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 -- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 msgr2=0x55dcc9b1eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b1eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 -- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 msgr2=0x55dcc9b1e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 0x55dcc9b1e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 -- 172.21.15.83:0/3825288717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc98fae80 con 0x55dcc99ab6d0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 0x55dcc9b1e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b1eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9b808ced0 tx=0x7fe9b4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4313289a318b473 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9a2ffd640 1 -- 172.21.15.83:0/3825288717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4019070 con 0x55dcc99ab6d0 2024-05-17T22:00:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9a2ffd640 1 -- 172.21.15.83:0/3825288717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b4004220 con 0x55dcc99ab6d0 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9a2ffd640 1 -- 172.21.15.83:0/3825288717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4011850 con 0x55dcc99ab6d0 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3825288717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 msgr2=0x55dcc9b1eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9b808ced0 tx=0x7fe9b4009190 comp rx=0 tx=0).stop 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3825288717 shutdown_connections 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b1eff0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b1eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3825288717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99cdc10 0x55dcc9b1e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3825288717 >> 172.21.15.83:0/3825288717 conn(0x55dcc95e2590 msgr2=0x55dcc9ad3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3825288717 shutdown_connections 2024-05-17T22:00:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3825288717 wait complete. 2024-05-17T22:00:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- start start 2024-05-17T22:00:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 Processor -- start 2024-05-17T22:00:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b26270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dcc98e7bf0 con 0x55dcc99bf450 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dcc99dfa10 con 0x55dcc99ab6d0 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dcc991c410 con 0x55dcc99cdc10 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9c68c7000 10 client.0 _ll_register_callbacks cb 0x55dcc9ac0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bcbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bcbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52316/0 (socket says 172.21.15.83:52316) 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.452+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/2595390803 learned_addr learned my addr 172.21.15.83:0/2595390803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 msgr2=0x55dcc9b26270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b26270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b26270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 msgr2=0x55dcc9b257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/2595390803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc98d58f0 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac09bae0 tx=0x7fe9ac01c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd6227a538b593a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0cb060 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe9ac004d40 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b4004220 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9ac007520 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 10 client.5775 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55dcc98e04e0 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b4004220 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fe9ac0b0420 con 0x55dcc99bf450 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 10 client.5775 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 10 client.5775 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 20 client.5775 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95c6ac0 0x55dcc9b2fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 4 client.5775 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc98bbd70 con 0x55dcc95c6ac0 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95c6ac0 0x55dcc9b2fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 10 client.5775 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95c6ac0 0x55dcc9b2fbd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45731d157b29b68d server_cookie=a2d4bf91ef8ad11 in_seq=0 out_seq=0 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 1 -- 172.21.15.83:0/2595390803 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcc98bbd70 con 0x55dcc95c6ac0 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 10 client.5775 handle_command_reply: tid=0 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 client.5775 shutdown 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 msgr2=0x55dcc9b25d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac09bae0 tx=0x7fe9ac01c130 comp rx=0 tx=0).stop 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 shutdown_connections 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b26270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b25d30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95c6ac0 0x55dcc9b2fbd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/2595390803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe997fff640 0 client.5775 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 >> 172.21.15.83:0/2595390803 conn(0x55dcc95e2590 msgr2=0x55dcc9ad4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 shutdown_connections 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 1 -- 172.21.15.83:0/2595390803 wait complete. 2024-05-17T22:00:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.456+0000 7fe9c68c7000 20 client.5775 trim_cache size 0 max 16384 2024-05-17T22:00:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 Processor -- start 2024-05-17T22:00:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- start start 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b82028a0 con 0x55dcc99bf450 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b82018e0 con 0x55dcc99ab6d0 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b8201290 con 0x55dcc99cdc10 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52332/0 (socket says 172.21.15.83:52332) 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/3031453141 learned_addr learned my addr 172.21.15.83:0/3031453141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 msgr2=0x55dcc9b25d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b25d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 msgr2=0x55dcc9b2c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 -- 172.21.15.83:0/3031453141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc99a4bf0 con 0x55dcc99bf450 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0c9040 tx=0x7fe9ac0c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3643f1e0746a875b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9a27fc640 1 -- 172.21.15.83:0/3031453141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0b0da0 con 0x55dcc99bf450 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9a27fc640 1 -- 172.21.15.83:0/3031453141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0076c0 con 0x55dcc99bf450 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9a27fc640 1 -- 172.21.15.83:0/3031453141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0b0970 con 0x55dcc99bf450 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3031453141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 msgr2=0x55dcc9b257c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0c9040 tx=0x7fe9ac0c0000 comp rx=0 tx=0).stop 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3031453141 shutdown_connections 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99cdc10 0x55dcc9b25d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99bf450 0x55dcc9b257c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/3031453141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3031453141 >> 172.21.15.83:0/3031453141 conn(0x55dcc95e2590 msgr2=0x55dcc9b09080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3031453141 shutdown_connections 2024-05-17T22:00:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.476+0000 7fe9c68c7000 1 -- 172.21.15.83:0/3031453141 wait complete. 2024-05-17T22:00:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- start start 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 Processor -- start 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b4daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dcc98ed460 con 0x55dcc99cdc10 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dcc992f0d0 con 0x55dcc99ab6d0 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dcc99231f0 con 0x55dcc99bf450 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 10 client.0 _ll_register_callbacks cb 0x7fe9a4003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52336/0 (socket says 172.21.15.83:52336) 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 -- 172.21.15.83:0/4274288443 learned_addr learned my addr 172.21.15.83:0/4274288443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bcbe8640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b4daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 -- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 msgr2=0x55dcc9b4daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b4daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 -- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 msgr2=0x55dcc9b2b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 -- 172.21.15.83:0/4274288443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc98dcdf0 con 0x55dcc99cdc10 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bdbea640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55dcc99616c0 tx=0x7fe9b40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ce411726f15eca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4019040 con 0x55dcc99cdc10 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fe9b4029ad0 con 0x55dcc99cdc10 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9ac0076c0 con 0x55dcc99cdc10 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9b4006de0 con 0x55dcc99cdc10 2024-05-17T22:00:07.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 10 client.5781 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55dcc98c74d0 con 0x55dcc99cdc10 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0076c0 con 0x55dcc99cdc10 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fe9b4011b00 con 0x55dcc99cdc10 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 10 client.5781 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 10 client.5781 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 20 client.5781 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95b4b20 0x55dcc9b4e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 4 client.5781 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55dcc9932250 con 0x55dcc95b4b20 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95b4b20 0x55dcc9b4e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 10 client.5781 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9bd3e9640 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95b4b20 0x55dcc9b4e520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665f11903d7c471 server_cookie=627d737aaa5cbe2e in_seq=0 out_seq=0 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 1 -- 172.21.15.83:0/4274288443 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcc9932250 con 0x55dcc95b4b20 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe997fff640 10 client.5781 handle_command_reply: tid=0 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.480+0000 7fe9c68c7000 1 client.5781 shutdown 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 msgr2=0x55dcc9b4dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55dcc99616c0 tx=0x7fe9b40046d0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 shutdown_connections 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc99cdc10 0x55dcc9b4dfe0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dcc99bf450 0x55dcc9b4daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55dcc95b4b20 0x55dcc9b4e520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 --2- 172.21.15.83:0/4274288443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dcc99ab6d0 0x55dcc9b2b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe997fff640 0 client.5781 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 >> 172.21.15.83:0/4274288443 conn(0x55dcc95e2590 msgr2=0x55dcc9b09810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 shutdown_connections 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 1 -- 172.21.15.83:0/4274288443 wait complete. 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9c68c7000 20 client.5781 trim_cache size 0 max 16384 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "26099743-579f-43c8-a724-aeb99362c6e6", 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:00:07.481 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:07.482 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-05-17T22:00:07.484+0000 7f7d26e02640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5739=0-4194304@1} caps={5739=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5739 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55d1ee694b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:00:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 msgr2=0x7fe99c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 0x7fe99c071910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ae000 tx=0x7fe9b80ce460 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 msgr2=0x7fe9b80b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e8f0 tx=0x7fe9b000edb0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 shutdown_connections 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe99c06f400 0x7fe99c071910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9b806dde0 0x7fe9b81f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9b806f390 0x7fe9b806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 --2- 172.21.15.83:0/1841730004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a4004810 0x7fe9b80b0a70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 >> 172.21.15.83:0/1841730004 conn(0x7fe9b80b5100 msgr2=0x7fe9b80b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 shutdown_connections 2024-05-17T22:00:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.484+0000 7fe9bf64d640 1 -- 172.21.15.83:0/1841730004 wait complete. 2024-05-17T22:00:07.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:07.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-17T22:00:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.640+0000 7f5244dd4640 1 Processor -- start 2024-05-17T22:00:07.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.640+0000 7f5244dd4640 1 -- start start 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cf830 0x7f52400b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52400cdf40 0x7f524015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f524015d530 0x7f524006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f52400b5640 con 0x7f524015d530 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52400b58e0 con 0x7f52400cdf40 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f5244dd4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f524006f060 con 0x7f52400cf830 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f524015d530 0x7f524006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f524015d530 0x7f524006f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52346/0 (socket says 172.21.15.83:52346) 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 -- 172.21.15.83:0/978233279 learned_addr learned my addr 172.21.15.83:0/978233279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523dd9b640 1 --2- 172.21.15.83:0/978233279 >> v2:172.21.15.155:3300/0 conn(0x7f52400cdf40 0x7f524015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523e59c640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cf830 0x7f52400b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 -- 172.21.15.83:0/978233279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cf830 msgr2=0x7f52400b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cf830 0x7f52400b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 -- 172.21.15.83:0/978233279 >> v2:172.21.15.155:3300/0 conn(0x7f52400cdf40 msgr2=0x7f524015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 --2- 172.21.15.83:0/978233279 >> v2:172.21.15.155:3300/0 conn(0x7f52400cdf40 0x7f524015cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 -- 172.21.15.83:0/978233279 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524006f340 con 0x7f524015d530 2024-05-17T22:00:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523ed9d640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524015d530 0x7f524006f860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f524006d7e0 tx=0x7f523801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d686fa8b189162 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523d59a640 1 -- 172.21.15.83:0/978233279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380a5780 con 0x7f524015d530 2024-05-17T22:00:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523d59a640 1 -- 172.21.15.83:0/978233279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52380a5920 con 0x7f524015d530 2024-05-17T22:00:07.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.644+0000 7f523d59a640 1 -- 172.21.15.83:0/978233279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380a5cf0 con 0x7f524015d530 2024-05-17T22:00:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- 172.21.15.83:0/978233279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524015d530 msgr2=0x7f524006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524015d530 0x7f524006f860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f524006d7e0 tx=0x7f523801c9f0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- 172.21.15.83:0/978233279 shutdown_connections 2024-05-17T22:00:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524015d530 0x7f524006f860 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- 172.21.15.83:0/978233279 >> v2:172.21.15.155:3300/0 conn(0x7f52400cdf40 0x7f524015cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- 172.21.15.83:0/978233279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cf830 0x7f52400b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- 172.21.15.83:0/978233279 >> 172.21.15.83:0/978233279 conn(0x7f524006cd90 msgr2=0x7f52400b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- 172.21.15.83:0/978233279 shutdown_connections 2024-05-17T22:00:07.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- 172.21.15.83:0/978233279 wait complete. 2024-05-17T22:00:07.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 Processor -- start 2024-05-17T22:00:07.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- start start 2024-05-17T22:00:07.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 0x7f524014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524015d530 0x7f524014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5240072e80 con 0x7f52400cf830 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52400707c0 con 0x7f524015d530 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.648+0000 7f5244dd4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5240073000 con 0x7f52400cdf40 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52348/0 (socket says 172.21.15.83:52348) 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 -- 172.21.15.83:0/3016453482 learned_addr learned my addr 172.21.15.83:0/3016453482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523e59c640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 0x7f524014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 -- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 msgr2=0x7f524014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 0x7f524014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 -- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524015d530 msgr2=0x7f524014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523ed9d640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524015d530 0x7f524014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524015d530 0x7f524014ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 -- 172.21.15.83:0/3016453482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52400d0e50 con 0x7f52400cf830 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523e59c640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 0x7f524014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523dd9b640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f523400d5a0 tx=0x7f523400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 -- 172.21.15.83:0/3016453482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340125f0 con 0x7f52400cf830 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 -- 172.21.15.83:0/3016453482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5234019070 con 0x7f52400cf830 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52400d1130 con 0x7f52400cf830 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 -- 172.21.15.83:0/3016453482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340129b0 con 0x7f52400cf830 2024-05-17T22:00:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52400d16f0 con 0x7f52400cf830 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 -- 172.21.15.83:0/3016453482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5234012b50 con 0x7f52400cf830 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f522effd640 1 -- 172.21.15.83:0/3016453482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f523408f6b0 con 0x7f52400cf830 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523e59c640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.652+0000 7f523e59c640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5230006fd0 tx=0x7f5230008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 Processor -- start 2024-05-17T22:00:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- start start 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x55905163ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x55905163d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590514dba90 con 0x5590514c4090 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55905144f120 con 0x5590514c3880 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55905143ecc0 con 0x5590514d6f80 2024-05-17T22:00:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52362/0 (socket says 172.21.15.83:52362) 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 -- 172.21.15.83:0/3531266327 learned_addr learned my addr 172.21.15.83:0/3531266327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523dd9b640 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x55905163d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523e59c640 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x55905163ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 -- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 msgr2=0x55905163ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x55905163ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 -- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 msgr2=0x55905163d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x55905163d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 -- 172.21.15.83:0/3531266327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590513fb110 con 0x5590514c4090 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f523ed9d640 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f523801b0a0 tx=0x7f5238004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d0251b9f2881d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f521ffff640 1 -- 172.21.15.83:0/3531266327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380a5780 con 0x5590514c4090 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f521ffff640 1 -- 172.21.15.83:0/3531266327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52380a5920 con 0x5590514c4090 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f521ffff640 1 -- 172.21.15.83:0/3531266327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380a5cf0 con 0x5590514c4090 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- 172.21.15.83:0/3531266327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 msgr2=0x55905163d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f523801b0a0 tx=0x7f5238004ac0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- 172.21.15.83:0/3531266327 shutdown_connections 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x55905163d8a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x55905163d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 --2- 172.21.15.83:0/3531266327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x55905163ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- 172.21.15.83:0/3531266327 >> 172.21.15.83:0/3531266327 conn(0x559051512010 msgr2=0x5590515f25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- 172.21.15.83:0/3531266327 shutdown_connections 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- 172.21.15.83:0/3531266327 wait complete. 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.656+0000 7f524c04e000 1 -- start start 2024-05-17T22:00:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 Processor -- start 2024-05-17T22:00:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c3880 0x5590515f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c4090 0x5590515f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590513faec0 con 0x5590514d6f80 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559051442d00 con 0x5590514c4090 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559051108680 con 0x5590514c3880 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 10 client.0 _ll_register_callbacks cb 0x5590514f61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52370/0 (socket says 172.21.15.83:52370) 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 -- 172.21.15.83:0/982791639 learned_addr learned my addr 172.21.15.83:0/982791639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523e59c640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c3880 0x5590515f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523dd9b640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c4090 0x5590515f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 -- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c3880 msgr2=0x5590515f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c3880 0x5590515f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 -- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c4090 msgr2=0x5590515f9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c4090 0x5590515f9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 -- 172.21.15.83:0/982791639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590513fd890 con 0x5590514d6f80 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523ed9d640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f523801c820 tx=0x7f523801b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a992688d6fd55c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380c8060 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f52380039a0 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52380a5920 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52380ce050 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 10 client.5793 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5590513ff650 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52380a5920 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f52380d1020 con 0x5590514d6f80 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 10 client.5793 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 10 client.5793 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 20 client.5793 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510e5930 0x559051643e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 4 client.5793 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590513dac40 con 0x5590510e5930 2024-05-17T22:00:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523e59c640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510e5930 0x559051643e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 10 client.5793 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f523e59c640 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510e5930 0x559051643e90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55e5dc637477cb4 server_cookie=3eb54bec6642bb7c in_seq=0 out_seq=0 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 1 -- 172.21.15.83:0/982791639 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590513dac40 con 0x5590510e5930 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.660+0000 7f521d7fa640 10 client.5793 handle_command_reply: tid=0 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 client.5793 shutdown 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 msgr2=0x5590515febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f523801c820 tx=0x7f523801b230 comp rx=0 tx=0).stop 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 shutdown_connections 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590515febe0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510e5930 0x559051643e90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c4090 0x5590515f9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 --2- 172.21.15.83:0/982791639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c3880 0x5590515f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f521d7fa640 0 client.5793 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 >> 172.21.15.83:0/982791639 conn(0x559051512010 msgr2=0x5590515f25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 shutdown_connections 2024-05-17T22:00:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 1 -- 172.21.15.83:0/982791639 wait complete. 2024-05-17T22:00:07.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.664+0000 7f524c04e000 20 client.5793 trim_cache size 0 max 16384 2024-05-17T22:00:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 Processor -- start 2024-05-17T22:00:07.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 -- start start 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x559051654420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x5590515f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52402011a0 con 0x5590514c4090 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f524015f350 con 0x5590514c3880 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f524015e6b0 con 0x5590514d6f80 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f523dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52378/0 (socket says 172.21.15.83:52378) 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.680+0000 7f523dd9b640 1 -- 172.21.15.83:0/2704010862 learned_addr learned my addr 172.21.15.83:0/2704010862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523e59c640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x559051654420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523ed9d640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x5590515f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 -- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 msgr2=0x5590515f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x5590515f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 -- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 msgr2=0x559051654420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x559051654420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 -- 172.21.15.83:0/2704010862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590513de8c0 con 0x5590514c4090 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523ed9d640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x5590515f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523dd9b640 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5234057ea0 tx=0x7f523405b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9252da9531cfcfb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f521f7fe640 1 -- 172.21.15.83:0/2704010862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523405be20 con 0x5590514c4090 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f521f7fe640 1 -- 172.21.15.83:0/2704010862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52340601d0 con 0x5590514c4090 2024-05-17T22:00:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f521f7fe640 1 -- 172.21.15.83:0/2704010862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234064c80 con 0x5590514c4090 2024-05-17T22:00:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- 172.21.15.83:0/2704010862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 msgr2=0x559051654960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5234057ea0 tx=0x7f523405b650 comp rx=0 tx=0).stop 2024-05-17T22:00:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- 172.21.15.83:0/2704010862 shutdown_connections 2024-05-17T22:00:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514d6f80 0x5590515f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514c4090 0x559051654960 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- 172.21.15.83:0/2704010862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x559051654420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- 172.21.15.83:0/2704010862 >> 172.21.15.83:0/2704010862 conn(0x559051512010 msgr2=0x55905163c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- 172.21.15.83:0/2704010862 shutdown_connections 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- 172.21.15.83:0/2704010862 wait complete. 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- start start 2024-05-17T22:00:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 Processor -- start 2024-05-17T22:00:07.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x5590516af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c4090 0x5590516afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f524015ed00 con 0x5590514d6f80 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559051405f10 con 0x5590514c3880 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55905143ecc0 con 0x5590514c4090 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f524c04e000 10 client.0 _ll_register_callbacks cb 0x7f5238009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52382/0 (socket says 172.21.15.83:52382) 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.684+0000 7f523ed9d640 1 -- 172.21.15.83:0/1096707819 learned_addr learned my addr 172.21.15.83:0/1096707819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523dd9b640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c4090 0x5590516afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523e59c640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x5590516af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 -- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c4090 msgr2=0x5590516afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c4090 0x5590516afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 -- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 msgr2=0x5590516af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x5590516af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 -- 172.21.15.83:0/1096707819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590513f9fa0 con 0x5590514d6f80 2024-05-17T22:00:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523ed9d640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5238002240 tx=0x7f52380a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f7b43de2d68245 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380d7060 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340601d0 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f52380039a0 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52380dd050 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 10 client.5799 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5590510f9220 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52340601d0 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f52380e0020 con 0x5590514d6f80 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 10 client.5799 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 10 client.5799 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 20 client.5799 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510d3990 0x55905166b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 4 client.5799 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:00:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559051451070 con 0x5590510d3990 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523e59c640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510d3990 0x55905166b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 10 client.5799 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f523e59c640 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510d3990 0x55905166b220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3d70802657c768 server_cookie=b13580ba1a5926ba in_seq=0 out_seq=0 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 1 -- 172.21.15.83:0/1096707819 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559051451070 con 0x5590510d3990 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 10 client.5799 handle_command_reply: tid=0 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 client.5799 shutdown 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 msgr2=0x5590516b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5238002240 tx=0x7f52380a7070 comp rx=0 tx=0).stop 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 shutdown_connections 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590514d6f80 0x5590516b01a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5590514c4090 0x5590516afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5590510d3990 0x55905166b220 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 --2- 172.21.15.83:0/1096707819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5590514c3880 0x5590516af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f521d7fa640 0 client.5799 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:07.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 >> 172.21.15.83:0/1096707819 conn(0x559051512010 msgr2=0x559051653410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 shutdown_connections 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 1 -- 172.21.15.83:0/1096707819 wait complete. 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f524c04e000 20 client.5799 trim_cache size 0 max 16384 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:00:07.687 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 msgr2=0x7f5220071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5230006fd0 tx=0x7f5230008040 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 msgr2=0x7f524014d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f523400d5a0 tx=0x7f523400da60 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 shutdown_connections 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524015d530 0x7f524014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52400cf830 0x7f524014d890 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 --2- 172.21.15.83:0/3016453482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52400cdf40 0x7f524014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 >> 172.21.15.83:0/3016453482 conn(0x7f524006cd90 msgr2=0x7f524015c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 shutdown_connections 2024-05-17T22:00:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.688+0000 7f5244dd4640 1 -- 172.21.15.83:0/3016453482 wait complete. 2024-05-17T22:00:07.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:00:07.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 Processor -- start 2024-05-17T22:00:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 -- start start 2024-05-17T22:00:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f415d2f0 0x7f26f4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26f40b3df0 0x7f26f4072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f415c830 con 0x7f26f40b39e0 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f26f41608a0 con 0x7f26f40b3df0 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fc64e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26f4160b80 con 0x7f26f415d2f0 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fa3ea640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f415d2f0 0x7f26f4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26f40b3df0 0x7f26f4072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26f40b3df0 0x7f26f4072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:56252/0 (socket says 172.21.15.83:56252) 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26f9be9640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f26f40b39e0 0x7f26f415fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:38676/0 (socket says 172.21.15.83:38676) 2024-05-17T22:00:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 -- 172.21.15.83:0/1249826747 learned_addr learned my addr 172.21.15.83:0/1249826747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 -- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f415d2f0 msgr2=0x7f26f4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 --2- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f415d2f0 0x7f26f4159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 -- 172.21.15.83:0/1249826747 >> v1:172.21.15.155:6789/0 conn(0x7f26f40b39e0 legacy=0x7f26f415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.844+0000 7f26fabeb640 1 -- 172.21.15.83:0/1249826747 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f415caf0 con 0x7f26f40b3df0 2024-05-17T22:00:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fabeb640 1 --2- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f40b3df0 0x7f26f4072e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009e50 tx=0x7f26ec01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b945dce050af08 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26f93e8640 1 -- 172.21.15.83:0/1249826747 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0a5700 con 0x7f26f40b3df0 2024-05-17T22:00:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26f93e8640 1 -- 172.21.15.83:0/1249826747 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ec0a58a0 con 0x7f26f40b3df0 2024-05-17T22:00:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26f93e8640 1 -- 172.21.15.83:0/1249826747 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0a5c70 con 0x7f26f40b3df0 2024-05-17T22:00:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 -- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f40b3df0 msgr2=0x7f26f4072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 --2- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f40b3df0 0x7f26f4072e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009e50 tx=0x7f26ec01ca30 comp rx=0 tx=0).stop 2024-05-17T22:00:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 -- 172.21.15.83:0/1249826747 shutdown_connections 2024-05-17T22:00:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 --2- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f40b3df0 0x7f26f4072e30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 --2- 172.21.15.83:0/1249826747 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f415d2f0 0x7f26f4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 -- 172.21.15.83:0/1249826747 >> 172.21.15.83:0/1249826747 conn(0x7f26f406cd90 msgr2=0x7f26f40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 -- 172.21.15.83:0/1249826747 shutdown_connections 2024-05-17T22:00:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.848+0000 7f26fc64e640 1 -- 172.21.15.83:0/1249826747 wait complete. 2024-05-17T22:00:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 Processor -- start 2024-05-17T22:00:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- start start 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40b3df0 0x7f26f41592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 0x7f26f413ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26f40b0b80 con 0x7f26f40b39e0 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26f415a160 con 0x7f26f415d2f0 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26f40b0d00 con 0x7f26f40b3df0 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52396/0 (socket says 172.21.15.83:52396) 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 -- 172.21.15.83:0/3391288942 learned_addr learned my addr 172.21.15.83:0/3391288942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26f9be9640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40b3df0 0x7f26f41592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fabeb640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 0x7f26f413ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 -- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40b3df0 msgr2=0x7f26f41592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40b3df0 0x7f26f41592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 -- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 msgr2=0x7f26f413ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 0x7f26f413ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 -- 172.21.15.83:0/3391288942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f4159820 con 0x7f26f40b39e0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fabeb640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 0x7f26f413ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fa3ea640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26f0010a80 tx=0x7f26f0010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f001e070 con 0x7f26f40b39e0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f0011450 con 0x7f26f40b39e0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f415e320 con 0x7f26f40b39e0 2024-05-17T22:00:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0011800 con 0x7f26f40b39e0 2024-05-17T22:00:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f415e8e0 con 0x7f26f40b39e0 2024-05-17T22:00:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.852+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f4072e30 con 0x7f26f40b39e0 2024-05-17T22:00:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.856+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26f0021070 con 0x7f26f40b39e0 2024-05-17T22:00:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.856+0000 7f26eaffd640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 0x7f26d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.856+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f26f0091370 con 0x7f26f40b39e0 2024-05-17T22:00:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.856+0000 7f26f9be9640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 0x7f26d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.856+0000 7f26f9be9640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 0x7f26d8071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f26f4156020 tx=0x7f26e4007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:07.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:07.860+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f26f4159820 con 0x7f26f40b39e0 2024-05-17T22:00:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.056+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f4155d70 con 0x7f26f40b39e0 2024-05-17T22:00:08.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.056+0000 7f26eaffd640 1 -- 172.21.15.83:0/3391288942 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5184 (secure 0 0 0) 0x7f26f005dc10 con 0x7f26f40b39e0 2024-05-17T22:00:08.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:08.051 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T21:59:59.970206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-05-17T22:00:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 msgr2=0x7f26d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 0x7f26d8071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f26f4156020 tx=0x7f26e4007cb0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 msgr2=0x7f26f413d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26f0010a80 tx=0x7f26f0010e80 comp rx=0 tx=0).stop 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 shutdown_connections 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26d806f400 0x7f26d8071910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26f415d2f0 0x7f26f413ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26f40b3df0 0x7f26f41592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 --2- 172.21.15.83:0/3391288942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f40b39e0 0x7f26f413d620 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 >> 172.21.15.83:0/3391288942 conn(0x7f26f406cd90 msgr2=0x7f26f40d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 shutdown_connections 2024-05-17T22:00:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.060+0000 7f26fc64e640 1 -- 172.21.15.83:0/3391288942 wait complete. 2024-05-17T22:00:08.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-17T22:00:08.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.212+0000 7fee383b0640 1 Processor -- start 2024-05-17T22:00:08.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.212+0000 7fee383b0640 1 -- start start 2024-05-17T22:00:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee383b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300b4810 0x7fee300d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee383b0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee300d2000 0x7fee300d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee383b0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee300d4170 con 0x7fee300d0290 2024-05-17T22:00:08.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee383b0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fee300d4450 con 0x7fee300d2000 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee383b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee300d4730 con 0x7fee300b4810 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee300d2000 0x7fee300d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fee300d2000 0x7fee300d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52408/0 (socket says 172.21.15.83:52408) 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 -- 172.21.15.83:0/2787991802 learned_addr learned my addr 172.21.15.83:0/2787991802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3614c640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300b4810 0x7fee300d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 -- 172.21.15.83:0/2787991802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300b4810 msgr2=0x7fee300d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300b4810 0x7fee300d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 -- 172.21.15.83:0/2787991802 >> v1:172.21.15.155:6789/0 conn(0x7fee300d0290 legacy=0x7fee3015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 -- 172.21.15.83:0/2787991802 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee300d4a10 con 0x7fee300d2000 2024-05-17T22:00:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3694d640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee300d2410 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e50 tx=0x7fee2801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81acac2ed83b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3514a640 1 -- 172.21.15.83:0/2787991802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280a5920 con 0x7fee300d2000 2024-05-17T22:00:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3514a640 1 -- 172.21.15.83:0/2787991802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee280a5ac0 con 0x7fee300d2000 2024-05-17T22:00:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.216+0000 7fee3514a640 1 -- 172.21.15.83:0/2787991802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280ad3f0 con 0x7fee300d2000 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- 172.21.15.83:0/2787991802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 msgr2=0x7fee300d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee300d2410 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e50 tx=0x7fee2801ca30 comp rx=0 tx=0).stop 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- 172.21.15.83:0/2787991802 shutdown_connections 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee300d2410 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- 172.21.15.83:0/2787991802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300b4810 0x7fee300d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- 172.21.15.83:0/2787991802 >> 172.21.15.83:0/2787991802 conn(0x7fee3006e010 msgr2=0x7fee3006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- 172.21.15.83:0/2787991802 shutdown_connections 2024-05-17T22:00:08.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- 172.21.15.83:0/2787991802 wait complete. 2024-05-17T22:00:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 Processor -- start 2024-05-17T22:00:08.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- start start 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 0x7fee3014f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300d0290 0x7fee301550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee300b9670 con 0x7fee300d2000 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee300b94f0 con 0x7fee300b4810 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee383b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee300b97f0 con 0x7fee300d0290 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee3694d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee3694d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52424/0 (socket says 172.21.15.83:52424) 2024-05-17T22:00:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.220+0000 7fee3694d640 1 -- 172.21.15.83:0/3206711602 learned_addr learned my addr 172.21.15.83:0/3206711602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3614c640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 0x7fee3014f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3594b640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300d0290 0x7fee301550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 -- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300d0290 msgr2=0x7fee301550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300d0290 0x7fee301550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 -- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 msgr2=0x7fee3014f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 0x7fee3014f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 -- 172.21.15.83:0/3206711602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee30150220 con 0x7fee300d2000 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3614c640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 0x7fee3014f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3694d640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fee2801a660 tx=0x7fee280197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 -- 172.21.15.83:0/3206711602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280be020 con 0x7fee300d2000 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 -- 172.21.15.83:0/3206711602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee280b8440 con 0x7fee300d2000 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3014f980 con 0x7fee300d2000 2024-05-17T22:00:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 -- 172.21.15.83:0/3206711602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280a5920 con 0x7fee300d2000 2024-05-17T22:00:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3014ff40 con 0x7fee300d2000 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 -- 172.21.15.83:0/3206711602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee280c1070 con 0x7fee300d2000 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 0x7fee14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee26ffd640 1 -- 172.21.15.83:0/3206711602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fee281298c0 con 0x7fee300d2000 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3614c640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 0x7fee14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3614c640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 0x7fee14071910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0097a0 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3f62a000 1 Processor -- start 2024-05-17T22:00:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.224+0000 7fee3f62a000 1 -- start start 2024-05-17T22:00:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f7a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ce1e18fa0 con 0x557ce1e2a460 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557ce1d8d2e0 con 0x557ce1e34e00 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557ce1d7ce80 con 0x557ce1e14030 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52434/0 (socket says 172.21.15.83:52434) 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 -- 172.21.15.83:0/3216161310 learned_addr learned my addr 172.21.15.83:0/3216161310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3614c640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f7a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 -- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 msgr2=0x557ce1f7aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3694d640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 -- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 msgr2=0x557ce1f7a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f7a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 -- 172.21.15.83:0/3216161310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1e27b80 con 0x557ce1e2a460 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3614c640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f7a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3594b640 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fee200015f0 tx=0x7fee20001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381472992c8e69b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee13fff640 1 -- 172.21.15.83:0/3216161310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20016020 con 0x557ce1e2a460 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee13fff640 1 -- 172.21.15.83:0/3216161310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2000db40 con 0x557ce1e2a460 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee13fff640 1 -- 172.21.15.83:0/3216161310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20001d00 con 0x557ce1e2a460 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- 172.21.15.83:0/3216161310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 msgr2=0x557ce1f7a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fee200015f0 tx=0x7fee20001ab0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- 172.21.15.83:0/3216161310 shutdown_connections 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f7aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f7a560 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 --2- 172.21.15.83:0/3216161310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f7a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- 172.21.15.83:0/3216161310 >> 172.21.15.83:0/3216161310 conn(0x557ce1a3e760 msgr2=0x557ce1f37270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- 172.21.15.83:0/3216161310 shutdown_connections 2024-05-17T22:00:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- 172.21.15.83:0/3216161310 wait complete. 2024-05-17T22:00:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 -- start start 2024-05-17T22:00:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.228+0000 7fee3f62a000 1 Processor -- start 2024-05-17T22:00:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ce1d38eb0 con 0x557ce1e2a460 2024-05-17T22:00:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557ce1d80ec0 con 0x557ce1e34e00 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557ce1a46820 con 0x557ce1e14030 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 10 client.0 _ll_register_callbacks cb 0x557ce1f1cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52448/0 (socket says 172.21.15.83:52448) 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 -- 172.21.15.83:0/1780065437 learned_addr learned my addr 172.21.15.83:0/1780065437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3614c640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 -- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 msgr2=0x557ce1f812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3694d640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 -- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 msgr2=0x557ce1f81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f81d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 -- 172.21.15.83:0/1780065437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce1dc5ef0 con 0x557ce1e2a460 2024-05-17T22:00:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3594b640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fee20001af0 tx=0x7fee20000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db77938ea11645f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20028060 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fee2000d940 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee2000db40 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee2002e050 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 10 client.5814 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557ce1dba910 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2000db40 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fee20031020 con 0x557ce1e2a460 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 10 client.5814 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 10 client.5814 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 20 client.5814 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:08.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a23ad0 0x557ce1f8b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 4 client.5814 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce1d18cf0 con 0x557ce1a23ad0 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3614c640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a23ad0 0x557ce1f8b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 10 client.5814 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3614c640 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a23ad0 0x557ce1f8b680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a05911fbee06421 server_cookie=3692336696910a67 in_seq=0 out_seq=0 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 1 -- 172.21.15.83:0/1780065437 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557ce1d18cf0 con 0x557ce1a23ad0 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 10 client.5814 handle_command_reply: tid=0 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 client.5814 shutdown 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 msgr2=0x557ce1f817e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fee20001af0 tx=0x7fee20000bd0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 shutdown_connections 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a23ad0 0x557ce1f8b680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1e34e00 0x557ce1f81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee117fa640 0 client.5814 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f817e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1780065437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 >> 172.21.15.83:0/1780065437 conn(0x557ce1a3e760 msgr2=0x557ce1f2f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 shutdown_connections 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 1 -- 172.21.15.83:0/1780065437 wait complete. 2024-05-17T22:00:08.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.232+0000 7fee3f62a000 20 client.5814 trim_cache size 0 max 16384 2024-05-17T22:00:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 Processor -- start 2024-05-17T22:00:08.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- start start 2024-05-17T22:00:08.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1f79450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f79990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee301512d0 con 0x557ce1e2a460 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee30150630 con 0x557ce1a23ad0 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee30150c80 con 0x557ce1e14030 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52460/0 (socket says 172.21.15.83:52460) 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 -- 172.21.15.83:0/1687517679 learned_addr learned my addr 172.21.15.83:0/1687517679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3594b640 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f79990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3614c640 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1f79450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 -- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 msgr2=0x557ce1f79990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f79990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 -- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 msgr2=0x557ce1f79450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1f79450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 -- 172.21.15.83:0/1687517679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1d3a670 con 0x557ce1e2a460 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3694d640 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fee280f6160 tx=0x7fee280f2150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f432590f5a24799a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee137fe640 1 -- 172.21.15.83:0/1687517679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280a5ac0 con 0x557ce1e2a460 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee137fe640 1 -- 172.21.15.83:0/1687517679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee280f5ce0 con 0x557ce1e2a460 2024-05-17T22:00:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee137fe640 1 -- 172.21.15.83:0/1687517679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280f68d0 con 0x557ce1e2a460 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- 172.21.15.83:0/1687517679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 msgr2=0x557ce1f2fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fee280f6160 tx=0x7fee280f2150 comp rx=0 tx=0).stop 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- 172.21.15.83:0/1687517679 shutdown_connections 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e2a460 0x557ce1f2fb00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e14030 0x557ce1f79990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 --2- 172.21.15.83:0/1687517679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1f79450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- 172.21.15.83:0/1687517679 >> 172.21.15.83:0/1687517679 conn(0x557ce1a3e760 msgr2=0x557ce1f78d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- 172.21.15.83:0/1687517679 shutdown_connections 2024-05-17T22:00:08.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.248+0000 7fee3f62a000 1 -- 172.21.15.83:0/1687517679 wait complete. 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- start start 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 Processor -- start 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1ff5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e2a460 0x557ce1ff61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ce1d38eb0 con 0x557ce1e14030 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557ce1d8c0b0 con 0x557ce1a23ad0 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557ce1e3c260 con 0x557ce1e2a460 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 10 client.0 _ll_register_callbacks cb 0x7fee20001490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52474/0 (socket says 172.21.15.83:52474) 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 -- 172.21.15.83:0/2232092388 learned_addr learned my addr 172.21.15.83:0/2232092388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3614c640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1ff5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 -- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e2a460 msgr2=0x557ce1ff61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e2a460 0x557ce1ff61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 -- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 msgr2=0x557ce1ff5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1ff5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 -- 172.21.15.83:0/2232092388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce1d39100 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3594b640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d6a0 tx=0x7fee20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9d3c9b4f7a8c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20037060 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fee2001ba40 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee280f5ce0 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee2003d050 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 10 client.5820 fetch_fsmap learned FSMap version 32 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557ce1e01d80 con 0x557ce1e14030 2024-05-17T22:00:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee280f5ce0 con 0x557ce1e14030 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7fee20031030 con 0x557ce1e14030 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 10 client.5820 fetch_fsmap finished waiting for FSMap version 32 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 10 client.5820 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 20 client.5820 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a3d180 0x557ce1fb1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 4 client.5820 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x557ce1d8f230 con 0x557ce1a3d180 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3614c640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a3d180 0x557ce1fb1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 10 client.5820 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee3614c640 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a3d180 0x557ce1fb1190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831b6c256e4683b8 server_cookie=eadf748b3d3e943c in_seq=0 out_seq=0 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.252+0000 7fee117fa640 1 -- 172.21.15.83:0/2232092388 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x557ce1d8f230 con 0x557ce1a3d180 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee117fa640 10 client.5820 handle_command_reply: tid=0 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 client.5820 shutdown 2024-05-17T22:00:08.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 msgr2=0x557ce1ff5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d6a0 tx=0x7fee20009040 comp rx=0 tx=0).stop 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 shutdown_connections 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557ce1e2a460 0x557ce1ff61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce1e14030 0x557ce1ff5c90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x557ce1a3d180 0x557ce1fb1190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 --2- 172.21.15.83:0/2232092388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ce1a23ad0 0x557ce1ff5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee117fa640 0 client.5820 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 >> 172.21.15.83:0/2232092388 conn(0x557ce1a3e760 msgr2=0x557ce1f794d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 shutdown_connections 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 1 -- 172.21.15.83:0/2232092388 wait complete. 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee3f62a000 20 client.5820 trim_cache size 0 max 16384 2024-05-17T22:00:08.252 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 3200176370, 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir_test/file" 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:08.253 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 msgr2=0x7fee14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 0x7fee14071910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0097a0 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 msgr2=0x7fee301540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fee2801a660 tx=0x7fee280197d0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 shutdown_connections 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee300d2000 0x7fee301540e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee300d0290 0x7fee301550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1406f400 0x7fee14071910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 --2- 172.21.15.83:0/3206711602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee300b4810 0x7fee3014f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 >> 172.21.15.83:0/3206711602 conn(0x7fee3006e010 msgr2=0x7fee30156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 shutdown_connections 2024-05-17T22:00:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.256+0000 7fee383b0640 1 -- 172.21.15.83:0/3206711602 wait complete. 2024-05-17T22:00:08.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:00:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.400+0000 7feed65e4640 1 Processor -- start 2024-05-17T22:00:08.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.400+0000 7feed65e4640 1 -- start start 2024-05-17T22:00:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed65e4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feed006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed65e4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed006d0e0 con 0x7feed015ec80 2024-05-17T22:00:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed65e4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00af930 con 0x7feed00af3e0 2024-05-17T22:00:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed65e4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feed015e7f0 con 0x7feed006c790 2024-05-17T22:00:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feecf7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feed015ec80 0x7feed00b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54974/0 (socket says 172.21.15.83:54974) 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feecf7fe640 1 -- 172.21.15.83:0/1057328416 learned_addr learned my addr 172.21.15.83:0/1057328416 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659225629 0 0) 0x7feec4001120 con 0x7feed015ec80 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feed006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8003530 con 0x7feed015ec80 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2007397653 0 0) 0x7feeb8003530 con 0x7feed015ec80 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 msgr2=0x7feed006cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feed006cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 >> v1:172.21.15.155:6789/0 conn(0x7feed00af3e0 legacy=0x7feed00b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015e1a0 con 0x7feed015ec80 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7feec40018d0 con 0x7feed015ec80 2024-05-17T22:00:08.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 >> v1:172.21.15.83:6789/0 conn(0x7feed015ec80 legacy=0x7feed00b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb8003a40 0x7feeb8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feeb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeb8004e70 0x7feeb80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feed015e7f0 con 0x7feeb8004e70 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800bc50 con 0x7feeb8003a40 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feeb800be50 con 0x7feed006c790 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeb8004e70 0x7feeb80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feecf7fe640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feeb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feecffff640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb8003a40 0x7feeb8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 -- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 msgr2=0x7feeb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed006c790 0x7feeb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 -- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb8003a40 msgr2=0x7feeb8003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb8003a40 0x7feeb8003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feed4b81640 1 -- 172.21.15.83:0/1057328416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb8003530 con 0x7feeb8004e70 2024-05-17T22:00:08.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.404+0000 7feecffff640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feeb8003a40 0x7feeb8003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:08.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed4b81640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeb8004e70 0x7feeb80052c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feec8007360 tx=0x7feec801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb7521bb0058509 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80b2070 con 0x7feeb8004e70 2024-05-17T22:00:08.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec80a5d60 con 0x7feeb8004e70 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feeceffd640 1 -- 172.21.15.83:0/1057328416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8009be0 con 0x7feeb8004e70 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- 172.21.15.83:0/1057328416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeb8004e70 msgr2=0x7feeb80052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 --2- 172.21.15.83:0/1057328416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeb8004e70 0x7feeb80052c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feec8007360 tx=0x7feec801ce60 comp rx=0 tx=0).stop 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed4b81640 1 -- 172.21.15.83:0/1057328416 reap_dead start 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- 172.21.15.83:0/1057328416 shutdown_connections 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- 172.21.15.83:0/1057328416 >> 172.21.15.83:0/1057328416 conn(0x7feed009b8b0 msgr2=0x7feed00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- 172.21.15.83:0/1057328416 shutdown_connections 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- 172.21.15.83:0/1057328416 wait complete. 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 Processor -- start 2024-05-17T22:00:08.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.408+0000 7feed65e4640 1 -- start start 2024-05-17T22:00:08.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed01563c0 0x7feed01542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 0x7feed01e8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feed006cba0 con 0x7feed0155fb0 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7feed00afc10 con 0x7feed01e8480 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7feed006cd20 con 0x7feed01563c0 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52494/0 (socket says 172.21.15.83:52494) 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 -- 172.21.15.83:0/4040685710 learned_addr learned my addr 172.21.15.83:0/4040685710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecf7fe640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed01563c0 0x7feed01542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed4b81640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 0x7feed01e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 -- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed01563c0 msgr2=0x7feed01542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed01563c0 0x7feed01542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 -- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 msgr2=0x7feed01e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 0x7feed01e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 -- 172.21.15.83:0/4040685710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed01e8dd0 con 0x7feed0155fb0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed4b81640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 0x7feed01e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecffff640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feec000d850 tx=0x7feec000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0005480 con 0x7feed0155fb0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0017070 con 0x7feed0155fb0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed0156930 con 0x7feed0155fb0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec000b870 con 0x7feed0155fb0 2024-05-17T22:00:08.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed01509d0 con 0x7feed0155fb0 2024-05-17T22:00:08.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed00b39e0 con 0x7feed0155fb0 2024-05-17T22:00:08.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feec000ba10 con 0x7feed0155fb0 2024-05-17T22:00:08.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 0x7feebc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:08.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7feec008f160 con 0x7feed0155fb0 2024-05-17T22:00:08.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecf7fe640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 0x7feebc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:08.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.412+0000 7feecf7fe640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 0x7feebc071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec4000fc0 tx=0x7feec4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:08.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.416+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7feed01e8dd0 con 0x7feed0155fb0 2024-05-17T22:00:08.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.596+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7feed009c0c0 con 0x7feed0155fb0 2024-05-17T22:00:08.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feeccff9640 1 -- 172.21.15.83:0/4040685710 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7feec0001d10 con 0x7feed0155fb0 2024-05-17T22:00:08.592 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:08.592 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 msgr2=0x7feebc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 0x7feebc071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec4000fc0 tx=0x7feec4009040 comp rx=0 tx=0).stop 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 msgr2=0x7feed0153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feec000d850 tx=0x7feec000dd10 comp rx=0 tx=0).stop 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 shutdown_connections 2024-05-17T22:00:08.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7feebc06f400 0x7feebc071910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feed01e8480 0x7feed01e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feed01563c0 0x7feed01542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 --2- 172.21.15.83:0/4040685710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feed0155fb0 0x7feed0153d70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 >> 172.21.15.83:0/4040685710 conn(0x7feed009b8b0 msgr2=0x7feed0153030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 shutdown_connections 2024-05-17T22:00:08.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:08.600+0000 7feed65e4640 1 -- 172.21.15.83:0/4040685710 wait complete. 2024-05-17T22:00:08.611 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-05-17T22:00:09.897 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:00:09.904+0000 7fb429809640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T22:00:13.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:00:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.776+0000 7fec96fa7640 1 Processor -- start 2024-05-17T22:00:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.776+0000 7fec96fa7640 1 -- start start 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec96fa7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 0x7fec90144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec96fa7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec9015e080 0x7fec90072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec96fa7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9015ea40 con 0x7fec9015e490 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec96fa7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fec9015ed20 con 0x7fec9015e080 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec96fa7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fec9015f000 con 0x7fec900d1040 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec9015e080 0x7fec90072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec95544640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fec9015e490 0x7fec901616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54984/0 (socket says 172.21.15.83:54984) 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec95544640 1 -- 172.21.15.83:0/3183045912 learned_addr learned my addr 172.21.15.83:0/3183045912 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec94d43640 1 --2- 172.21.15.83:0/3183045912 >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 0x7fec90144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 -- 172.21.15.83:0/3183045912 >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 msgr2=0x7fec90144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:13.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 --2- 172.21.15.83:0/3183045912 >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 0x7fec90144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 -- 172.21.15.83:0/3183045912 >> v1:172.21.15.83:6789/0 conn(0x7fec9015e490 legacy=0x7fec901616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 -- 172.21.15.83:0/3183045912 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9006ef30 con 0x7fec9015e080 2024-05-17T22:00:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8f7fe640 1 -- 172.21.15.83:0/3183045912 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221422066 0 0) 0x7fec9015ea40 con 0x7fec9015e490 2024-05-17T22:00:13.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec94d43640 1 --2- 172.21.15.83:0/3183045912 >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 0x7fec90144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:13.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8ffff640 1 --2- 172.21.15.83:0/3183045912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec90072e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fec90071d60 tx=0x7fec7c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa6742e21922067 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:13.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8f7fe640 1 -- 172.21.15.83:0/3183045912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b1070 con 0x7fec9015e080 2024-05-17T22:00:13.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8f7fe640 1 -- 172.21.15.83:0/3183045912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0ac5f0 con 0x7fec9015e080 2024-05-17T22:00:13.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.780+0000 7fec8f7fe640 1 -- 172.21.15.83:0/3183045912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0ac9c0 con 0x7fec9015e080 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- 172.21.15.83:0/3183045912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 msgr2=0x7fec90072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- 172.21.15.83:0/3183045912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec90072e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fec90071d60 tx=0x7fec7c01c480 comp rx=0 tx=0).stop 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- 172.21.15.83:0/3183045912 shutdown_connections 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- 172.21.15.83:0/3183045912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec90072e30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- 172.21.15.83:0/3183045912 >> v2:172.21.15.155:3300/0 conn(0x7fec900d1040 0x7fec90144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- 172.21.15.83:0/3183045912 >> 172.21.15.83:0/3183045912 conn(0x7fec9006cd90 msgr2=0x7fec900b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:13.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- 172.21.15.83:0/3183045912 shutdown_connections 2024-05-17T22:00:13.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- 172.21.15.83:0/3183045912 wait complete. 2024-05-17T22:00:13.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 Processor -- start 2024-05-17T22:00:13.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- start start 2024-05-17T22:00:13.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec900d1040 0x7fec900b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 0x7fec9015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec9016abe0 con 0x7fec9015e080 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.784+0000 7fec96fa7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fec9016aa60 con 0x7fec9015e490 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec96fa7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fec9016ad60 con 0x7fec900d1040 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52500/0 (socket says 172.21.15.83:52500) 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 -- 172.21.15.83:0/57561633 learned_addr learned my addr 172.21.15.83:0/57561633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec95544640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 0x7fec9015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec94d43640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec900d1040 0x7fec900b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 -- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec900d1040 msgr2=0x7fec900b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec900d1040 0x7fec900b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 -- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 msgr2=0x7fec9015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 0x7fec9015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 -- 172.21.15.83:0/57561633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9013e320 con 0x7fec9015e080 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec95544640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 0x7fec9015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8ffff640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fec7c01ab30 tx=0x7fec7c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b1040 con 0x7fec9015e080 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0bb070 con 0x7fec9015e080 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec90168180 con 0x7fec9015e080 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b5650 con 0x7fec9015e080 2024-05-17T22:00:13.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec90159e00 con 0x7fec9015e080 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec90167ef0 con 0x7fec9015e080 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec7c0b57f0 con 0x7fec9015e080 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 0x7fec74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fec7c128fa0 con 0x7fec9015e080 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec94d43640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 0x7fec74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:13.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.788+0000 7fec94d43640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 0x7fec74071940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec84006fd0 tx=0x7fec84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:13.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.792+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fec7c0a9070 con 0x7fec9015e080 2024-05-17T22:00:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.988+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fec90144080 con 0x7fec9015e080 2024-05-17T22:00:13.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.988+0000 7fec8d7fa640 1 -- 172.21.15.83:0/57561633 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fec7c016070 con 0x7fec9015e080 2024-05-17T22:00:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:13.983 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-17T22:00:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 msgr2=0x7fec74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 0x7fec74071940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec84006fd0 tx=0x7fec84008040 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 msgr2=0x7fec9015c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fec7c01ab30 tx=0x7fec7c019220 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 shutdown_connections 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fec7406f430 0x7fec74071940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec9015e490 0x7fec9015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec9015e080 0x7fec9015c650 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 --2- 172.21.15.83:0/57561633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec900d1040 0x7fec900b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 >> 172.21.15.83:0/57561633 conn(0x7fec9006cd90 msgr2=0x7fec9015f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 shutdown_connections 2024-05-17T22:00:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:13.992+0000 7fec96fa7640 1 -- 172.21.15.83:0/57561633 wait complete. 2024-05-17T22:00:14.005 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-05-17T22:00:14.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:14.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-17T22:00:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.164+0000 7f20ee2fb640 1 Processor -- start 2024-05-17T22:00:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.164+0000 7f20ee2fb640 1 -- start start 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ee2fb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20e8159da0 0x7f20e815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ee2fb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e8073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ee2fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20e80737a0 con 0x7f20e806ee70 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ee2fb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f20e80b0e90 con 0x7f20e8159da0 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ee2fb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20e80cf050 con 0x7f20e8072e50 2024-05-17T22:00:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20e8159da0 0x7f20e815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ec898640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f20e806ee70 0x7f20e806f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55000/0 (socket says 172.21.15.83:55000) 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20ec898640 1 -- 172.21.15.83:0/160599814 learned_addr learned my addr 172.21.15.83:0/160599814 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20df7fe640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e8073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e7fff640 1 -- 172.21.15.83:0/160599814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 msgr2=0x7f20e8073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e7fff640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e8073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e7fff640 1 -- 172.21.15.83:0/160599814 >> v1:172.21.15.83:6789/0 conn(0x7f20e806ee70 legacy=0x7f20e806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e7fff640 1 -- 172.21.15.83:0/160599814 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e80cf2f0 con 0x7f20e8159da0 2024-05-17T22:00:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.168+0000 7f20e77fe640 1 -- 172.21.15.83:0/160599814 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357231116 0 0) 0x7f20e00014a0 con 0x7f20e806ee70 2024-05-17T22:00:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20e7fff640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e8159da0 0x7f20e815cfc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38b2e46deed1f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20e77fe640 1 -- 172.21.15.83:0/160599814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80b0070 con 0x7f20e8159da0 2024-05-17T22:00:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20e77fe640 1 -- 172.21.15.83:0/160599814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d80ab600 con 0x7f20e8159da0 2024-05-17T22:00:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20e77fe640 1 -- 172.21.15.83:0/160599814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80ab9d0 con 0x7f20e8159da0 2024-05-17T22:00:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- 172.21.15.83:0/160599814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e8159da0 msgr2=0x7f20e815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e8159da0 0x7f20e815cfc0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801c480 comp rx=0 tx=0).stop 2024-05-17T22:00:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- 172.21.15.83:0/160599814 shutdown_connections 2024-05-17T22:00:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e8073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/160599814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e8159da0 0x7f20e815cfc0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- 172.21.15.83:0/160599814 >> 172.21.15.83:0/160599814 conn(0x7f20e806cd90 msgr2=0x7f20e80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- 172.21.15.83:0/160599814 shutdown_connections 2024-05-17T22:00:14.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- 172.21.15.83:0/160599814 wait complete. 2024-05-17T22:00:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 Processor -- start 2024-05-17T22:00:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.172+0000 7f20ee2fb640 1 -- start start 2024-05-17T22:00:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8159da0 0x7f20e814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8070520 con 0x7f20e806ee70 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20e80cf870 con 0x7f20e8159da0 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20e80b0810 con 0x7f20e8072e50 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52524/0 (socket says 172.21.15.83:52524) 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 -- 172.21.15.83:0/3630797341 learned_addr learned my addr 172.21.15.83:0/3630797341 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20df7fe640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 -- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 msgr2=0x7f20e814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 -- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8159da0 msgr2=0x7f20e814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ec898640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8159da0 0x7f20e814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8159da0 0x7f20e814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 -- 172.21.15.83:0/3630797341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e815e0c0 con 0x7f20e806ee70 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e57fa640 1 -- 172.21.15.83:0/3630797341 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80b0040 con 0x7f20e806ee70 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e57fa640 1 -- 172.21.15.83:0/3630797341 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d8004030 con 0x7f20e806ee70 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e80b5820 con 0x7f20e806ee70 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20e57fa640 1 -- 172.21.15.83:0/3630797341 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004400 con 0x7f20e806ee70 2024-05-17T22:00:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.176+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e8154d50 con 0x7f20e806ee70 2024-05-17T22:00:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20e57fa640 1 -- 172.21.15.83:0/3630797341 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f20d8003620 con 0x7f20e806ee70 2024-05-17T22:00:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20e57fa640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 0x7f20c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20e57fa640 1 -- 172.21.15.83:0/3630797341 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f20d81298a0 con 0x7f20e806ee70 2024-05-17T22:00:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 0x7f20c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 0x7f20c8071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20e8157b70 tx=0x7f20d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 Processor -- start 2024-05-17T22:00:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 -- start start 2024-05-17T22:00:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197888df20 0x561978a14770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561978824f90 con 0x5619788a1c30 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561978814b30 con 0x56197889a240 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561978818b70 con 0x56197888df20 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52532/0 (socket says 172.21.15.83:52532) 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 -- 172.21.15.83:0/2922094389 learned_addr learned my addr 172.21.15.83:0/2922094389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20ec898640 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20e7fff640 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197888df20 0x561978a14770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 -- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197888df20 msgr2=0x561978a14770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197888df20 0x561978a14770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 -- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 msgr2=0x561978a151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 -- 172.21.15.83:0/2922094389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197885f390 con 0x5619788a1c30 2024-05-17T22:00:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d9d0 tx=0x7f20d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd7461e6954386 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20ddffb640 1 -- 172.21.15.83:0/2922094389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0023070 con 0x5619788a1c30 2024-05-17T22:00:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20ddffb640 1 -- 172.21.15.83:0/2922094389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d001a040 con 0x5619788a1c30 2024-05-17T22:00:14.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.180+0000 7f20ddffb640 1 -- 172.21.15.83:0/2922094389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0019050 con 0x5619788a1c30 2024-05-17T22:00:14.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- 172.21.15.83:0/2922094389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 msgr2=0x561978a14cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f20d000d9d0 tx=0x7f20d0010000 comp rx=0 tx=0).stop 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- 172.21.15.83:0/2922094389 shutdown_connections 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619788a1c30 0x561978a14cb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- 172.21.15.83:0/2922094389 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197888df20 0x561978a14770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- 172.21.15.83:0/2922094389 >> 172.21.15.83:0/2922094389 conn(0x56197898a850 msgr2=0x5619789c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- 172.21.15.83:0/2922094389 shutdown_connections 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- 172.21.15.83:0/2922094389 wait complete. 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- start start 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 Processor -- start 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x5619789d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x5619789d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5619787c1e70 con 0x56197888df20 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5619787e3ed0 con 0x56197889a240 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561978817e80 con 0x5619788a1c30 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20f5575000 10 client.0 _ll_register_callbacks cb 0x5619789b6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52546/0 (socket says 172.21.15.83:52546) 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 -- 172.21.15.83:0/2662452769 learned_addr learned my addr 172.21.15.83:0/2662452769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20ec898640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x5619789d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x5619789d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 -- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 msgr2=0x5619789d5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x5619789d5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 -- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 msgr2=0x5619789d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x5619789d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 -- 172.21.15.83:0/2662452769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619787ca9c0 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20ec898640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x5619789d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20e7fff640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20d801c4c0 tx=0x7f20d80abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c93cf3465d977b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.184+0000 7f20c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80fecf0 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f20d80f5410 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d001a040 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20d80093e0 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 10 client.5850 fetch_fsmap learned FSMap version 33 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5619787d1a50 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d001a040 con 0x56197888df20 2024-05-17T22:00:14.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f20d80f1b20 con 0x56197888df20 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 10 client.5850 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 10 client.5850 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 20 client.5850 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784bb860 0x561978a1a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 4 client.5850 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619787b0cc0 con 0x5619784bb860 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784bb860 0x561978a1a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 10 client.5850 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20df7fe640 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784bb860 0x561978a1a9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f9818f656e4a70 server_cookie=6e7f0489f00ba77a in_seq=0 out_seq=0 2024-05-17T22:00:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 1 -- 172.21.15.83:0/2662452769 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619787b0cc0 con 0x5619784bb860 2024-05-17T22:00:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 10 client.5850 handle_command_reply: tid=0 2024-05-17T22:00:14.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 client.5850 shutdown 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 msgr2=0x5619789d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f20d801c4c0 tx=0x7f20d80abf90 comp rx=0 tx=0).stop 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 shutdown_connections 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x5619789d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784bb860 0x561978a1a9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x5619789d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 --2- 172.21.15.83:0/2662452769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789d0740 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20c6ffd640 0 client.5850 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 >> 172.21.15.83:0/2662452769 conn(0x56197898a850 msgr2=0x5619789c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 shutdown_connections 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 1 -- 172.21.15.83:0/2662452769 wait complete. 2024-05-17T22:00:14.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.188+0000 7f20f5575000 20 client.5850 trim_cache size 0 max 16384 2024-05-17T22:00:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 Processor -- start 2024-05-17T22:00:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- start start 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x561978a2a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8202b10 con 0x56197888df20 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81527d0 con 0x56197889a240 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f20e8156610 con 0x5619788a1c30 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52560/0 (socket says 172.21.15.83:52560) 2024-05-17T22:00:14.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 -- 172.21.15.83:0/3479075394 learned_addr learned my addr 172.21.15.83:0/3479075394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 -- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 msgr2=0x561978a2a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x561978a2a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 -- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 msgr2=0x561978a2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 -- 172.21.15.83:0/3479075394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619789d62e0 con 0x56197888df20 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20e7fff640 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f20d80f6140 tx=0x7f20d80f9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2493ce53b54f0774 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20dd7fa640 1 -- 172.21.15.83:0/3479075394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80f68b0 con 0x56197888df20 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20dd7fa640 1 -- 172.21.15.83:0/3479075394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d80f61b0 con 0x56197888df20 2024-05-17T22:00:14.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20dd7fa640 1 -- 172.21.15.83:0/3479075394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80fecf0 con 0x56197888df20 2024-05-17T22:00:14.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- 172.21.15.83:0/3479075394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 msgr2=0x561978a29f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f20d80f6140 tx=0x7f20d80f9000 comp rx=0 tx=0).stop 2024-05-17T22:00:14.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- 172.21.15.83:0/3479075394 shutdown_connections 2024-05-17T22:00:14.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5619788a1c30 0x561978a2a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56197889a240 0x561978a2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 --2- 172.21.15.83:0/3479075394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x561978a29f60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- 172.21.15.83:0/3479075394 >> 172.21.15.83:0/3479075394 conn(0x56197898a850 msgr2=0x5619789cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- 172.21.15.83:0/3479075394 shutdown_connections 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.208+0000 7f20f5575000 1 -- 172.21.15.83:0/3479075394 wait complete. 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- start start 2024-05-17T22:00:14.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 Processor -- start 2024-05-17T22:00:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197889a240 0x5619789cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619788a1c30 0x561978a47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561978827b00 con 0x56197888df20 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56197881d1a0 con 0x5619788a1c30 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5619787d5120 con 0x56197889a240 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 10 client.0 _ll_register_callbacks cb 0x7f20d80f2800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52562/0 (socket says 172.21.15.83:52562) 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 -- 172.21.15.83:0/1578282200 learned_addr learned my addr 172.21.15.83:0/1578282200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 -- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197889a240 msgr2=0x5619789cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20df7fe640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197889a240 0x5619789cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20ec898640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619788a1c30 0x561978a47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197889a240 0x5619789cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 -- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619788a1c30 msgr2=0x561978a47ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619788a1c30 0x561978a47ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 -- 172.21.15.83:0/1578282200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619787d1070 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20e7fff640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f20d80f2230 tx=0x7f20d8127d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a52f23d6df1d7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d80f68b0 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f20d80f5620 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d80f61b0 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20d80fa560 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 10 client.5856 fetch_fsmap learned FSMap version 33 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5619787cd900 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d80f61b0 con 0x56197888df20 2024-05-17T22:00:14.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f20d80f4e80 con 0x56197888df20 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 10 client.5856 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 10 client.5856 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 20 client.5856 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784a98c0 0x561978a486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 4 client.5856 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x5619787ed520 con 0x5619784a98c0 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20df7fe640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784a98c0 0x561978a486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20c6ffd640 10 client.5856 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.212+0000 7f20df7fe640 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784a98c0 0x561978a486a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82393e0fef503019 server_cookie=c4a6934a95890d3d in_seq=0 out_seq=0 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20c6ffd640 1 -- 172.21.15.83:0/1578282200 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5619787ed520 con 0x5619784a98c0 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20c6ffd640 10 client.5856 handle_command_reply: tid=0 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 client.5856 shutdown 2024-05-17T22:00:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 msgr2=0x5619789cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f20d80f2230 tx=0x7f20d8127d30 comp rx=0 tx=0).stop 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 shutdown_connections 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5619784a98c0 0x561978a486a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619788a1c30 0x561978a47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56197889a240 0x5619789cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 --2- 172.21.15.83:0/1578282200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56197888df20 0x5619789cb340 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20c6ffd640 0 client.5856 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 >> 172.21.15.83:0/1578282200 conn(0x56197898a850 msgr2=0x5619789d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 shutdown_connections 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 1 -- 172.21.15.83:0/1578282200 wait complete. 2024-05-17T22:00:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20f5575000 20 client.5856 trim_cache size 0 max 16384 2024-05-17T22:00:14.213 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:00:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "eccc295f-195f-4200-ab00-ca2ab307488e", 2024-05-17T22:00:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:00:14.213 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:14.213 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f7d26e02640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-17T22:00:06.309114+0000 1=1+0) caps={5739=pAsLsXsFs/-@2},l=5739(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55d1ee694b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 msgr2=0x7f20c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 0x7f20c8071910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20e8157b70 tx=0x7f20d0008040 comp rx=0 tx=0).stop 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 msgr2=0x7f20e80b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801acc0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 shutdown_connections 2024-05-17T22:00:14.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f20c806f400 0x7f20c8071910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20e8159da0 0x7f20e814e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20e8072e50 0x7f20e814e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 --2- 172.21.15.83:0/3630797341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20e806ee70 0x7f20e80b6470 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 >> 172.21.15.83:0/3630797341 conn(0x7f20e806cd90 msgr2=0x7f20e806f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 shutdown_connections 2024-05-17T22:00:14.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.216+0000 7f20ee2fb640 1 -- 172.21.15.83:0/3630797341 wait complete. 2024-05-17T22:00:14.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:14.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-17T22:00:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.380+0000 7effded19640 1 Processor -- start 2024-05-17T22:00:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.380+0000 7effded19640 1 -- start start 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effded19640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effd80d2050 0x7effd80d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effded19640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd80d1210 con 0x7effd81600f0 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effded19640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd8160e50 con 0x7effd80d0e00 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effded19640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7effd8160c10 con 0x7effd80d2050 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effd80d2050 0x7effd80d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effdd2b6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7effd81600f0 0x7effd8160500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55010/0 (socket says 172.21.15.83:55010) 2024-05-17T22:00:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effdd2b6640 1 -- 172.21.15.83:0/2020736724 learned_addr learned my addr 172.21.15.83:0/2020736724 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd7fff640 1 -- 172.21.15.83:0/2020736724 >> v1:172.21.15.155:6789/0 conn(0x7effd80d0e00 legacy=0x7effd815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd7fff640 1 -- 172.21.15.83:0/2020736724 >> v1:172.21.15.83:6789/0 conn(0x7effd81600f0 legacy=0x7effd8160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd7fff640 1 -- 172.21.15.83:0/2020736724 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd80d0320 con 0x7effd80d2050 2024-05-17T22:00:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd7fff640 1 --2- 172.21.15.83:0/2020736724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd80d2460 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a286305da58c876d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd77fe640 1 -- 172.21.15.83:0/2020736724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b0070 con 0x7effd80d2050 2024-05-17T22:00:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd77fe640 1 -- 172.21.15.83:0/2020736724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0ab600 con 0x7effd80d2050 2024-05-17T22:00:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.384+0000 7effd77fe640 1 -- 172.21.15.83:0/2020736724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0ab9d0 con 0x7effd80d2050 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- 172.21.15.83:0/2020736724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 msgr2=0x7effd80d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 --2- 172.21.15.83:0/2020736724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd80d2460 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097c0 tx=0x7effcc01c410 comp rx=0 tx=0).stop 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- 172.21.15.83:0/2020736724 shutdown_connections 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 --2- 172.21.15.83:0/2020736724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd80d2460 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- 172.21.15.83:0/2020736724 >> 172.21.15.83:0/2020736724 conn(0x7effd806dfd0 msgr2=0x7effd806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- 172.21.15.83:0/2020736724 shutdown_connections 2024-05-17T22:00:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- 172.21.15.83:0/2020736724 wait complete. 2024-05-17T22:00:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 Processor -- start 2024-05-17T22:00:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.388+0000 7effded19640 1 -- start start 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd80d0e00 0x7effd80c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd81600f0 0x7effd81474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7effd80b4e80 con 0x7effd80d2050 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effd80b4d00 con 0x7effd80d0e00 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effd81645d0 con 0x7effd81600f0 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52578/0 (socket says 172.21.15.83:52578) 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 -- 172.21.15.83:0/306102249 learned_addr learned my addr 172.21.15.83:0/306102249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 -- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd81600f0 msgr2=0x7effd81474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd81600f0 0x7effd81474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 -- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd80d0e00 msgr2=0x7effd80c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effdcab5640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd80d0e00 0x7effd80c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd80d0e00 0x7effd80c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 -- 172.21.15.83:0/306102249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd814b030 con 0x7effd80d2050 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd7fff640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7effcc0ae040 tx=0x7effcc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 -- 172.21.15.83:0/306102249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b0050 con 0x7effd80d2050 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 -- 172.21.15.83:0/306102249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0b1040 con 0x7effd80d2050 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 -- 172.21.15.83:0/306102249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd814a420 con 0x7effd80d2050 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 -- 172.21.15.83:0/306102249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b5870 con 0x7effd80d2050 2024-05-17T22:00:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effded19640 1 -- 172.21.15.83:0/306102249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd814a9e0 con 0x7effd80d2050 2024-05-17T22:00:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 -- 172.21.15.83:0/306102249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7effcc0be030 con 0x7effd80d2050 2024-05-17T22:00:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 0x7effc4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effd57fa640 1 -- 172.21.15.83:0/306102249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7effcc128d60 con 0x7effd80d2050 2024-05-17T22:00:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.392+0000 7effdcab5640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 0x7effc4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 0x7effc4071940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7effc800a750 tx=0x7effc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 Processor -- start 2024-05-17T22:00:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- start start 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0485360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568e0323a90 con 0x5568e031ef80 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568e0297120 con 0x5568e030c090 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5568e0286cc0 con 0x5568e030b880 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52594/0 (socket says 172.21.15.83:52594) 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 -- 172.21.15.83:0/1868326852 learned_addr learned my addr 172.21.15.83:0/1868326852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 -- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 msgr2=0x5568e0485360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0485360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 -- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 msgr2=0x5568e04858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 -- 172.21.15.83:0/1868326852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e0243110 con 0x5568e031ef80 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effdcab5640 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7effc800c940 tx=0x7effc800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3365e7d50117ba97 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effbe7fc640 1 -- 172.21.15.83:0/1868326852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8019040 con 0x5568e031ef80 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effbe7fc640 1 -- 172.21.15.83:0/1868326852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8022070 con 0x5568e031ef80 2024-05-17T22:00:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effbe7fc640 1 -- 172.21.15.83:0/1868326852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8013dc0 con 0x5568e031ef80 2024-05-17T22:00:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- 172.21.15.83:0/1868326852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 msgr2=0x5568e0484e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7effc800c940 tx=0x7effc800f000 comp rx=0 tx=0).stop 2024-05-17T22:00:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- 172.21.15.83:0/1868326852 shutdown_connections 2024-05-17T22:00:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0485360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 --2- 172.21.15.83:0/1868326852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0484e20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- 172.21.15.83:0/1868326852 >> 172.21.15.83:0/1868326852 conn(0x5568e035a010 msgr2=0x5568e043a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- 172.21.15.83:0/1868326852 shutdown_connections 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.396+0000 7effe5f93000 1 -- 172.21.15.83:0/1868326852 wait complete. 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- start start 2024-05-17T22:00:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 Processor -- start 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0441a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e0441f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568e0242ec0 con 0x5568e031ef80 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5568e028ad00 con 0x5568e030c090 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5568dff50680 con 0x5568e030b880 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 10 client.0 _ll_register_callbacks cb 0x5568e033e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52610/0 (socket says 172.21.15.83:52610) 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 -- 172.21.15.83:0/1367973559 learned_addr learned my addr 172.21.15.83:0/1367973559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdcab5640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0441a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effd7fff640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e0441f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 -- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 msgr2=0x5568e0441a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0441a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 -- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 msgr2=0x5568e0441f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e0441f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 -- 172.21.15.83:0/1367973559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e0245890 con 0x5568e031ef80 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdcab5640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0441a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdd2b6640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7effd000c160 tx=0x7effd000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfc1d2d79d00c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0004d70 con 0x5568e031ef80 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7effd0015ad0 con 0x5568e031ef80 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc8022070 con 0x5568e031ef80 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7effd0015d10 con 0x5568e031ef80 2024-05-17T22:00:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 10 client.5871 fetch_fsmap learned FSMap version 33 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5568e0247650 con 0x5568e031ef80 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8022070 con 0x5568e031ef80 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7effd001c050 con 0x5568e031ef80 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 10 client.5871 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 10 client.5871 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 20 client.5871 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff2d930 0x5568e048be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 4 client.5871 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e0222c40 con 0x5568dff2d930 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.400+0000 7effdcab5640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff2d930 0x5568e048be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effdcab5640 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff2d930 0x5568e048be90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f57b593f17f4d8 server_cookie=4e97e69172e671c6 in_seq=0 out_seq=0 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effb77fe640 10 client.5871 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effb77fe640 1 -- 172.21.15.83:0/1367973559 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568e0222c40 con 0x5568dff2d930 2024-05-17T22:00:14.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effb77fe640 10 client.5871 handle_command_reply: tid=0 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 client.5871 shutdown 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 msgr2=0x5568e0446be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7effd000c160 tx=0x7effd000c620 comp rx=0 tx=0).stop 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 shutdown_connections 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e031ef80 0x5568e0446be0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff2d930 0x5568e048be90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e0441f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effb77fe640 0 client.5871 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 --2- 172.21.15.83:0/1367973559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e0441a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 >> 172.21.15.83:0/1367973559 conn(0x5568e035a010 msgr2=0x5568e043a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 shutdown_connections 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 1 -- 172.21.15.83:0/1367973559 wait complete. 2024-05-17T22:00:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.404+0000 7effe5f93000 20 client.5871 trim_cache size 0 max 16384 2024-05-17T22:00:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.420+0000 7effe5f93000 1 Processor -- start 2024-05-17T22:00:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.420+0000 7effe5f93000 1 -- start start 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e035a010 0x5568e04f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7effd82057f0 con 0x5568e030b880 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effd8153730 con 0x5568e030c090 2024-05-17T22:00:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7effd814e760 con 0x5568e035a010 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52614/0 (socket says 172.21.15.83:52614) 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 -- 172.21.15.83:0/2199999686 learned_addr learned my addr 172.21.15.83:0/2199999686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effdcab5640 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e035a010 0x5568e04f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 -- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e035a010 msgr2=0x5568e04f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e035a010 0x5568e04f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effdd2b6640 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 -- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 msgr2=0x5568e04f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 -- 172.21.15.83:0/2199999686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e02268c0 con 0x5568e030b880 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effd7fff640 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7effcc0f16f0 tx=0x7effcc0f1530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9bad34aea4fefa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effbdffb640 1 -- 172.21.15.83:0/2199999686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0f4de0 con 0x5568e030b880 2024-05-17T22:00:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effbdffb640 1 -- 172.21.15.83:0/2199999686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0f5180 con 0x5568e030b880 2024-05-17T22:00:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effbdffb640 1 -- 172.21.15.83:0/2199999686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0f5d70 con 0x5568e030b880 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- 172.21.15.83:0/2199999686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 msgr2=0x5568e04f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7effcc0f16f0 tx=0x7effcc0f1530 comp rx=0 tx=0).stop 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- 172.21.15.83:0/2199999686 shutdown_connections 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e030c090 0x5568e04f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030b880 0x5568e04f40f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 --2- 172.21.15.83:0/2199999686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e035a010 0x5568e04f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- 172.21.15.83:0/2199999686 >> 172.21.15.83:0/2199999686 conn(0x5568e043e400 msgr2=0x5568e04409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- 172.21.15.83:0/2199999686 shutdown_connections 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- 172.21.15.83:0/2199999686 wait complete. 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 -- start start 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.424+0000 7effe5f93000 1 Processor -- start 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e043c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e035a010 0x5568e04b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568e024df10 con 0x5568e030c090 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7effd81530e0 con 0x5568e035a010 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5568e0346d50 con 0x5568e030b880 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 10 client.0 _ll_register_callbacks cb 0x7effd0007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52626/0 (socket says 172.21.15.83:52626) 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 -- 172.21.15.83:0/2796423070 learned_addr learned my addr 172.21.15.83:0/2796423070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effdd2b6640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e035a010 0x5568e04b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effdcab5640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e043c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 -- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 msgr2=0x5568e043c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e043c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 -- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e035a010 msgr2=0x5568e04b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e035a010 0x5568e04b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 -- 172.21.15.83:0/2796423070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e0241fa0 con 0x5568e030c090 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effd7fff640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7effcc0f5370 tx=0x7effcc0b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc40617fdc86d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0f5670 con 0x5568e030c090 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7effcc0fabb0 con 0x5568e030c090 2024-05-17T22:00:14.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc0f5180 con 0x5568e030c090 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7effcc0f9a20 con 0x5568e030c090 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 10 client.5877 fetch_fsmap learned FSMap version 33 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5568dff41220 con 0x5568e030c090 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0f5180 con 0x5568e030c090 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7effcc0f48d0 con 0x5568e030c090 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 10 client.5877 fetch_fsmap finished waiting for FSMap version 33 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 10 client.5877 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 20 client.5877 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff46fe0 0x5568e04b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 4 client.5877 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568e0299070 con 0x5568dff46fe0 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effdcab5640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff46fe0 0x5568e04b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 10 client.5877 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effdcab5640 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff46fe0 0x5568e04b5740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114e66871f210b6d server_cookie=cc80c13d001d74e4 in_seq=0 out_seq=0 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 1 -- 172.21.15.83:0/2796423070 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5568e0299070 con 0x5568dff46fe0 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 10 client.5877 handle_command_reply: tid=0 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 client.5877 shutdown 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 msgr2=0x5568e043cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7effcc0f5370 tx=0x7effcc0b8000 comp rx=0 tx=0).stop 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 shutdown_connections 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x5568dff46fe0 0x5568e04b5740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5568e035a010 0x5568e04b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5568e030c090 0x5568e043cc30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effb77fe640 0 client.5877 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 --2- 172.21.15.83:0/2796423070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5568e030b880 0x5568e043c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 >> 172.21.15.83:0/2796423070 conn(0x5568e043e400 msgr2=0x5568e04413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 shutdown_connections 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 1 -- 172.21.15.83:0/2796423070 wait complete. 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.428+0000 7effe5f93000 20 client.5877 trim_cache size 0 max 16384 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "eccc295f-195f-4200-ab00-ca2ab307488e": { 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "eccc295f-195f-4200-ab00-ca2ab307488e", 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair,force" 2024-05-17T22:00:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:14.429 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 msgr2=0x7effc4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 0x7effc4071940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7effc800a750 tx=0x7effc8008040 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 msgr2=0x7effd814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7effcc0ae040 tx=0x7effcc01ac50 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 shutdown_connections 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effd81600f0 0x7effd81474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effd80d2050 0x7effd814b430 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7effc406f430 0x7effc4071940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 --2- 172.21.15.83:0/306102249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effd80d0e00 0x7effd80c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 >> 172.21.15.83:0/306102249 conn(0x7effd806dfd0 msgr2=0x7effd8155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 shutdown_connections 2024-05-17T22:00:14.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:14.432+0000 7effded19640 1 -- 172.21.15.83:0/306102249 wait complete. 2024-05-17T22:00:14.443 INFO:tasks.cephfs.filesystem:scrub status for tag:eccc295f-195f-4200-ab00-ca2ab307488e - {'path': '/', 'tag': 'eccc295f-195f-4200-ab00-ca2ab307488e', 'options': 'recursive,repair,force'} 2024-05-17T22:00:30.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:00:30.166 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:00:44.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:44.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-17T22:00:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.616+0000 7fc3df496640 1 Processor -- start 2024-05-17T22:00:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.616+0000 7fc3df496640 1 -- start start 2024-05-17T22:00:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3df496640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3df496640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3df496640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d80b5640 con 0x7fc3d815d530 2024-05-17T22:00:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3df496640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d80b58e0 con 0x7fc3d80cf830 2024-05-17T22:00:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3df496640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d806f060 con 0x7fc3d80cdf40 2024-05-17T22:00:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dd232640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dda33640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc3d815d530 0x7fc3d806f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:39364/0 (socket says 172.21.15.83:39364) 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dda33640 1 -- 172.21.15.83:0/3785887208 learned_addr learned my addr 172.21.15.83:0/3785887208 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3785887208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:53094/0 (socket says 172.21.15.83:53094) 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3785887208 >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:58162/0 (socket says 172.21.15.83:58162) 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 -- 172.21.15.83:0/3785887208 >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 msgr2=0x7fc3d80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3785887208 >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 -- 172.21.15.83:0/3785887208 >> v1:172.21.15.155:6789/0 conn(0x7fc3d815d530 legacy=0x7fc3d806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dca31640 1 -- 172.21.15.83:0/3785887208 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d806f340 con 0x7fc3d80cdf40 2024-05-17T22:00:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.620+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3785887208 >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:00:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3785887208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0097c0 tx=0x7fc3cc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee884874307c1457 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3d7fff640 1 -- 172.21.15.83:0/3785887208 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0b1070 con 0x7fc3d80cdf40 2024-05-17T22:00:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3d7fff640 1 -- 172.21.15.83:0/3785887208 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3cc0ac410 con 0x7fc3d80cdf40 2024-05-17T22:00:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3d7fff640 1 -- 172.21.15.83:0/3785887208 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0ac7e0 con 0x7fc3d80cdf40 2024-05-17T22:00:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- 172.21.15.83:0/3785887208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 msgr2=0x7fc3d815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 --2- 172.21.15.83:0/3785887208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0097c0 tx=0x7fc3cc01c4a0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- 172.21.15.83:0/3785887208 shutdown_connections 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 --2- 172.21.15.83:0/3785887208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cdf40 0x7fc3d815cff0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 --2- 172.21.15.83:0/3785887208 >> v2:172.21.15.155:3300/0 conn(0x7fc3d80cf830 0x7fc3d80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- 172.21.15.83:0/3785887208 >> 172.21.15.83:0/3785887208 conn(0x7fc3d806cd90 msgr2=0x7fc3d80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- 172.21.15.83:0/3785887208 shutdown_connections 2024-05-17T22:00:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- 172.21.15.83:0/3785887208 wait complete. 2024-05-17T22:00:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 Processor -- start 2024-05-17T22:00:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.624+0000 7fc3df496640 1 -- start start 2024-05-17T22:00:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d80cdf40 0x7fc3d80afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cf830 0x7fc3d8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8073210 con 0x7fc3d815d530 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8073090 con 0x7fc3d80cdf40 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d8073390 con 0x7fc3d80cf830 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53446/0 (socket says 172.21.15.83:53446) 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 -- 172.21.15.83:0/3376861106 learned_addr learned my addr 172.21.15.83:0/3376861106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d80cdf40 0x7fc3d80afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cf830 0x7fc3d8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 -- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cf830 msgr2=0x7fc3d8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cf830 0x7fc3d8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 -- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d80cdf40 msgr2=0x7fc3d80afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d80cdf40 0x7fc3d80afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 -- 172.21.15.83:0/3376861106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d80b03b0 con 0x7fc3d815d530 2024-05-17T22:00:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c470 tx=0x7fc3d000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3d5ffb640 1 -- 172.21.15.83:0/3376861106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0017070 con 0x7fc3d815d530 2024-05-17T22:00:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3d5ffb640 1 -- 172.21.15.83:0/3376861106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0005cf0 con 0x7fc3d815d530 2024-05-17T22:00:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d80b1e40 con 0x7fc3d815d530 2024-05-17T22:00:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3d5ffb640 1 -- 172.21.15.83:0/3376861106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00072f0 con 0x7fc3d815d530 2024-05-17T22:00:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d80b2400 con 0x7fc3d815d530 2024-05-17T22:00:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3d5ffb640 1 -- 172.21.15.83:0/3376861106 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc3d001a070 con 0x7fc3d815d530 2024-05-17T22:00:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.628+0000 7fc3d5ffb640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 0x7fc3c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3d5ffb640 1 -- 172.21.15.83:0/3376861106 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc3d008e5b0 con 0x7fc3d815d530 2024-05-17T22:00:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 0x7fc3c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 0x7fc3c4071940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009430 tx=0x7fc3c800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 Processor -- start 2024-05-17T22:00:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- start start 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f70e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f71360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c066e0fa90 con 0x55c066df8090 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c066d83120 con 0x55c066df7880 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c066d72cc0 con 0x55c066e0af80 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53460/0 (socket says 172.21.15.83:53460) 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 -- 172.21.15.83:0/3088025868 learned_addr learned my addr 172.21.15.83:0/3088025868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f71360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f70e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 -- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 msgr2=0x55c066f70e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f70e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 -- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 msgr2=0x55c066f71360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f71360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 -- 172.21.15.83:0/3088025868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c066d2f110 con 0x55c066df8090 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f71360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3d008cb00 tx=0x7fc3d005a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affa31116570e542 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3beffd640 1 -- 172.21.15.83:0/3088025868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0074390 con 0x55c066df8090 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3beffd640 1 -- 172.21.15.83:0/3088025868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d005ae30 con 0x55c066df8090 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3beffd640 1 -- 172.21.15.83:0/3088025868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007690 con 0x55c066df8090 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- 172.21.15.83:0/3088025868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 msgr2=0x55c066f718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3d008cb00 tx=0x7fc3d005a940 comp rx=0 tx=0).stop 2024-05-17T22:00:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- 172.21.15.83:0/3088025868 shutdown_connections 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f718a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f71360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3088025868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- 172.21.15.83:0/3088025868 >> 172.21.15.83:0/3088025868 conn(0x55c066e46010 msgr2=0x55c066f265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- 172.21.15.83:0/3088025868 shutdown_connections 2024-05-17T22:00:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.632+0000 7fc3e6710000 1 -- 172.21.15.83:0/3088025868 wait complete. 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- start start 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 Processor -- start 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f2da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f32be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c066d2eec0 con 0x55c066df8090 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c066d76d00 con 0x55c066df7880 2024-05-17T22:00:44.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c066a3c680 con 0x55c066e0af80 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 10 client.0 _ll_register_callbacks cb 0x55c066e2a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53462/0 (socket says 172.21.15.83:53462) 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 -- 172.21.15.83:0/3537979086 learned_addr learned my addr 172.21.15.83:0/3537979086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f2da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 -- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 msgr2=0x55c066f32be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f32be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 -- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 msgr2=0x55c066f2da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f2da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 -- 172.21.15.83:0/3537979086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c066d31890 con 0x55c066df8090 2024-05-17T22:00:44.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dda33640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0097c0 tx=0x7fc3cc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a0c87a914b52d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0b1040 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d005ae30 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc3cc0a9040 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc0c2050 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 10 client.5886 fetch_fsmap learned FSMap version 34 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55c066d33650 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d005ae30 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fc3cc0c5020 con 0x55c066df8090 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 10 client.5886 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 10 client.5886 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 20 client.5886 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:44.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a19930 0x55c066f77e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 4 client.5886 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.636+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c066d0ec40 con 0x55c066a19930 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a19930 0x55c066f77e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3b7fff640 10 client.5886 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a19930 0x55c066f77e90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1565f108d9cf9f34 server_cookie=161e26a8d2800031 in_seq=0 out_seq=0 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3537979086 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c066d0ec40 con 0x55c066a19930 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3b7fff640 10 client.5886 handle_command_reply: tid=0 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 client.5886 shutdown 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 msgr2=0x55c066f2dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0097c0 tx=0x7fc3cc0046d0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 shutdown_connections 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066f32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066f2dfb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a19930 0x55c066f77e90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3537979086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066f2da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 >> 172.21.15.83:0/3537979086 conn(0x55c066e46010 msgr2=0x55c066f265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3b7fff640 0 client.5886 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 shutdown_connections 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 1 -- 172.21.15.83:0/3537979086 wait complete. 2024-05-17T22:00:44.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.640+0000 7fc3e6710000 20 client.5886 trim_cache size 0 max 16384 2024-05-17T22:00:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.656+0000 7fc3e6710000 1 Processor -- start 2024-05-17T22:00:44.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.656+0000 7fc3e6710000 1 -- start start 2024-05-17T22:00:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066df7880 0x55c066f29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 0x55c066f87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8205970 con 0x55c066e0af80 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8204a30 con 0x55c066df8090 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d81f9ff0 con 0x55c066df7880 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53478/0 (socket says 172.21.15.83:53478) 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 -- 172.21.15.83:0/1899794236 learned_addr learned my addr 172.21.15.83:0/1899794236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dca31640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 0x55c066f87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 -- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066df7880 msgr2=0x55c066f29260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dd232640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066df7880 0x55c066f29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066df7880 0x55c066f29260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 -- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 msgr2=0x55c066f87410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 0x55c066f87410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 -- 172.21.15.83:0/1899794236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c066d128c0 con 0x55c066e0af80 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dca31640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 0x55c066f87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3dda33640 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0056e60 tx=0x7fc3d008e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04bcf04c0dd2f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3be7fc640 1 -- 172.21.15.83:0/1899794236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0079070 con 0x55c066e0af80 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3be7fc640 1 -- 172.21.15.83:0/1899794236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d005ae30 con 0x55c066e0af80 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3be7fc640 1 -- 172.21.15.83:0/1899794236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0074390 con 0x55c066e0af80 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- 172.21.15.83:0/1899794236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 msgr2=0x55c066f87950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0056e60 tx=0x7fc3d008e070 comp rx=0 tx=0).stop 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- 172.21.15.83:0/1899794236 shutdown_connections 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066e0af80 0x55c066f87950 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df8090 0x55c066f87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 --2- 172.21.15.83:0/1899794236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066df7880 0x55c066f29260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- 172.21.15.83:0/1899794236 >> 172.21.15.83:0/1899794236 conn(0x55c066e46010 msgr2=0x55c066f70520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- 172.21.15.83:0/1899794236 shutdown_connections 2024-05-17T22:00:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- 172.21.15.83:0/1899794236 wait complete. 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 -- start start 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.660+0000 7fc3e6710000 1 Processor -- start 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066fe3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066fe4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c066d1a0f0 con 0x55c066df8090 2024-05-17T22:00:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d80ab3d0 con 0x55c066df7880 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c066d7b330 con 0x55c066e0af80 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 10 client.0 _ll_register_callbacks cb 0x7fc3cc0af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53490/0 (socket says 172.21.15.83:53490) 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 -- 172.21.15.83:0/3237996769 learned_addr learned my addr 172.21.15.83:0/3237996769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066fe3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 -- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 msgr2=0x55c066fe4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066fe4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 -- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 msgr2=0x55c066fe3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066fe3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 -- 172.21.15.83:0/3237996769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c066d2dfa0 con 0x55c066df8090 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dca31640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc01ab50 tx=0x7fc3cc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3837763ca1bc64c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0cc020 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc3cc003830 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3d005ae30 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc3cc0b1040 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 10 client.5892 fetch_fsmap learned FSMap version 34 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55c066a2d220 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d005ae30 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fc3cc0c5040 con 0x55c066df8090 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 10 client.5892 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 10 client.5892 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 20 client.5892 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a07990 0x55c066f281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 4 client.5892 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c066d85070 con 0x55c066a07990 2024-05-17T22:00:44.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a07990 0x55c066f281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 10 client.5892 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3dd232640 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a07990 0x55c066f281e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f515c760fd67411 server_cookie=456981a38dd7718f in_seq=0 out_seq=0 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 1 -- 172.21.15.83:0/3237996769 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c066d85070 con 0x55c066a07990 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 10 client.5892 handle_command_reply: tid=0 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 client.5892 shutdown 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 msgr2=0x55c066fe4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc01ab50 tx=0x7fc3cc01ace0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 shutdown_connections 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c066e0af80 0x55c066fe4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c066df8090 0x55c066fe4340 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55c066a07990 0x55c066f281e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 --2- 172.21.15.83:0/3237996769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c066df7880 0x55c066fe3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3b7fff640 0 client.5892 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 >> 172.21.15.83:0/3237996769 conn(0x55c066e46010 msgr2=0x55c066f87410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 shutdown_connections 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 1 -- 172.21.15.83:0/3237996769 wait complete. 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.664+0000 7fc3e6710000 20 client.5892 trim_cache size 0 max 16384 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:00:44.663 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 msgr2=0x7fc3c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 0x7fc3c4071940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009430 tx=0x7fc3c800d040 comp rx=0 tx=0).stop 2024-05-17T22:00:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 msgr2=0x7fc3d81fc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c470 tx=0x7fc3d000c930 comp rx=0 tx=0).stop 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 shutdown_connections 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3d815d530 0x7fc3d81fc6e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d80cf830 0x7fc3d8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3c406f430 0x7fc3c4071940 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 --2- 172.21.15.83:0/3376861106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d80cdf40 0x7fc3d80afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 >> 172.21.15.83:0/3376861106 conn(0x7fc3d806cd90 msgr2=0x7fc3d815bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 shutdown_connections 2024-05-17T22:00:44.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.668+0000 7fc3df496640 1 -- 172.21.15.83:0/3376861106 wait complete. 2024-05-17T22:00:44.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:00:44.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 Processor -- start 2024-05-17T22:00:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 -- start start 2024-05-17T22:00:44.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3e40d1930 0x7ff3e40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3e40b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e40d2280 con 0x7ff3e41580f0 2024-05-17T22:00:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e4158500 con 0x7ff3e40d1930 2024-05-17T22:00:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3eac5f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e415efc0 con 0x7ff3e40b3120 2024-05-17T22:00:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e89fb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3e41580f0 0x7ff3e40af370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48930/0 (socket says 172.21.15.83:48930) 2024-05-17T22:00:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e89fb640 1 -- 172.21.15.83:0/850091851 learned_addr learned my addr 172.21.15.83:0/850091851 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/850091851 >> v2:172.21.15.155:3300/0 conn(0x7ff3e40d1930 0x7ff3e40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423219593 0 0) 0x7ff3d4001120 con 0x7ff3e41580f0 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3cc003530 con 0x7ff3e41580f0 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e91fc640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3e40b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732593447 0 0) 0x7ff3cc003530 con 0x7ff3e41580f0 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 msgr2=0x7ff3e40b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3e40b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 >> v2:172.21.15.155:3300/0 conn(0x7ff3e40d1930 msgr2=0x7ff3e40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 --2- 172.21.15.83:0/850091851 >> v2:172.21.15.155:3300/0 conn(0x7ff3e40d1930 0x7ff3e40d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e40b4920 con 0x7ff3e41580f0 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ff3d40018d0 con 0x7ff3e41580f0 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 >> v1:172.21.15.83:6789/0 conn(0x7ff3e41580f0 legacy=0x7ff3e40af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3cc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3cc004d50 0x7ff3cc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e4158500 con 0x7ff3cc003a10 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e415efc0 con 0x7ff3cc004d50 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc00bb30 con 0x7ff3e40b3120 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3cc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.832+0000 7ff3e91fc640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3cc004d50 0x7ff3cc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 -- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 msgr2=0x7ff3cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e40b3120 0x7ff3cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 -- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3cc004d50 msgr2=0x7ff3cc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:44.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3cc004d50 0x7ff3cc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 -- 172.21.15.83:0/850091851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc003530 con 0x7ff3cc003a10 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3cc003e80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c4c0 tx=0x7ff3d401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da72f84dfa58dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400c0e0 con 0x7ff3cc003a10 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d40a9070 con 0x7ff3cc003a10 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e37fe640 1 -- 172.21.15.83:0/850091851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40a5dd0 con 0x7ff3cc003a10 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 -- 172.21.15.83:0/850091851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 msgr2=0x7ff3cc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/850091851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3cc003e80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400c4c0 tx=0x7ff3d401cbf0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3e91fc640 1 -- 172.21.15.83:0/850091851 reap_dead start 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 -- 172.21.15.83:0/850091851 shutdown_connections 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 -- 172.21.15.83:0/850091851 >> 172.21.15.83:0/850091851 conn(0x7ff3e409b8b0 msgr2=0x7ff3e40b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 -- 172.21.15.83:0/850091851 shutdown_connections 2024-05-17T22:00:44.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.836+0000 7ff3eac5f640 1 -- 172.21.15.83:0/850091851 wait complete. 2024-05-17T22:00:44.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 Processor -- start 2024-05-17T22:00:44.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 -- start start 2024-05-17T22:00:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3e40b61c0 0x7ff3e4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e4156380 0x7ff3e40cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40afa80 con 0x7ff3cc003a10 2024-05-17T22:00:44.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e40b4ee0 con 0x7ff3e40b61c0 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3eac5f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e40afc00 con 0x7ff3e4156380 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53512/0 (socket says 172.21.15.83:53512) 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 -- 172.21.15.83:0/364598228 learned_addr learned my addr 172.21.15.83:0/364598228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3e40b61c0 0x7ff3e4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e91fc640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e4156380 0x7ff3e40cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 -- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e4156380 msgr2=0x7ff3e40cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e4156380 0x7ff3e40cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 -- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3e40b61c0 msgr2=0x7ff3e4155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3e40b61c0 0x7ff3e4155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 -- 172.21.15.83:0/364598228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e4156790 con 0x7ff3cc003a10 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e89fb640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff3d401c6b0 tx=0x7ff3cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4001120 con 0x7ff3cc003a10 2024-05-17T22:00:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d40a5970 con 0x7ff3cc003a10 2024-05-17T22:00:44.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.840+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40a5d20 con 0x7ff3cc003a10 2024-05-17T22:00:44.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e40cb750 con 0x7ff3cc003a10 2024-05-17T22:00:44.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e40cbf30 con 0x7ff3cc003a10 2024-05-17T22:00:44.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e40d1d40 con 0x7ff3cc003a10 2024-05-17T22:00:44.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3d40047f0 con 0x7ff3cc003a10 2024-05-17T22:00:44.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3e17fa640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 0x7ff3d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:44.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7ff3d41286c0 con 0x7ff3cc003a10 2024-05-17T22:00:44.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 0x7ff3d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:44.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.844+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 0x7ff3d0071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff3e41fc540 tx=0x7ff3d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:44.848+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff3d40c1070 con 0x7ff3cc003a10 2024-05-17T22:00:45.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.032+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e4158ba0 con 0x7ff3cc003a10 2024-05-17T22:00:45.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3e17fa640 1 -- 172.21.15.83:0/364598228 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7ff3e4156790 con 0x7ff3cc003a10 2024-05-17T22:00:45.027 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:45.027 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T22:00:17.620242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:45.028 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-05-17T22:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 msgr2=0x7ff3d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 0x7ff3d0071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff3e41fc540 tx=0x7ff3d8008040 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 msgr2=0x7ff3e41543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff3d401c6b0 tx=0x7ff3cc005840 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 shutdown_connections 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3e4156380 0x7ff3e40cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3d006f400 0x7ff3d0071910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3e40b61c0 0x7ff3e4155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 --2- 172.21.15.83:0/364598228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3cc003a10 0x7ff3e41543d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 >> 172.21.15.83:0/364598228 conn(0x7ff3e409b8b0 msgr2=0x7ff3e406c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 shutdown_connections 2024-05-17T22:00:45.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.036+0000 7ff3eac5f640 1 -- 172.21.15.83:0/364598228 wait complete. 2024-05-17T22:00:45.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-17T22:00:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 Processor -- start 2024-05-17T22:00:45.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 -- start start 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fef300cf830 0x7fef300b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef3015d700 0x7fef301663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef300b5640 con 0x7fef300cdf40 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fef30070870 con 0x7fef3015d700 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef34ebf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fef300b58e0 con 0x7fef300cf830 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef2ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef3015d700 0x7fef301663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef2dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef300cdf40 0x7fef3015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48942/0 (socket says 172.21.15.83:48942) 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.188+0000 7fef2dd9b640 1 -- 172.21.15.83:0/3152929242 learned_addr learned my addr 172.21.15.83:0/3152929242 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2e59c640 1 --2- 172.21.15.83:0/3152929242 >> v2:172.21.15.155:3300/0 conn(0x7fef300cf830 0x7fef300b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2ed9d640 1 -- 172.21.15.83:0/3152929242 >> v2:172.21.15.155:3300/0 conn(0x7fef300cf830 msgr2=0x7fef300b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2ed9d640 1 --2- 172.21.15.83:0/3152929242 >> v2:172.21.15.155:3300/0 conn(0x7fef300cf830 0x7fef300b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2ed9d640 1 -- 172.21.15.83:0/3152929242 >> v1:172.21.15.83:6789/0 conn(0x7fef300cdf40 legacy=0x7fef3015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2d59a640 1 -- 172.21.15.83:0/3152929242 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213429501 0 0) 0x7fef300b5640 con 0x7fef300cdf40 2024-05-17T22:00:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2ed9d640 1 -- 172.21.15.83:0/3152929242 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3006f060 con 0x7fef3015d700 2024-05-17T22:00:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2ed9d640 1 --2- 172.21.15.83:0/3152929242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3015d700 0x7fef301663d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fef3006d6e0 tx=0x7fef2801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5e785156e1cef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2d59a640 1 -- 172.21.15.83:0/3152929242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280b2070 con 0x7fef3015d700 2024-05-17T22:00:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2d59a640 1 -- 172.21.15.83:0/3152929242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef280a5840 con 0x7fef3015d700 2024-05-17T22:00:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef2d59a640 1 -- 172.21.15.83:0/3152929242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef280a5c10 con 0x7fef3015d700 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 -- 172.21.15.83:0/3152929242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3015d700 msgr2=0x7fef301663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 --2- 172.21.15.83:0/3152929242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3015d700 0x7fef301663d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fef3006d6e0 tx=0x7fef2801c930 comp rx=0 tx=0).stop 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 -- 172.21.15.83:0/3152929242 shutdown_connections 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 --2- 172.21.15.83:0/3152929242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3015d700 0x7fef301663d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 --2- 172.21.15.83:0/3152929242 >> v2:172.21.15.155:3300/0 conn(0x7fef300cf830 0x7fef300b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 -- 172.21.15.83:0/3152929242 >> 172.21.15.83:0/3152929242 conn(0x7fef3006cd90 msgr2=0x7fef300b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 -- 172.21.15.83:0/3152929242 shutdown_connections 2024-05-17T22:00:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.192+0000 7fef34ebf640 1 -- 172.21.15.83:0/3152929242 wait complete. 2024-05-17T22:00:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 Processor -- start 2024-05-17T22:00:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- start start 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef300cf830 0x7fef300ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef3015d700 0x7fef3015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef301692b0 con 0x7fef300cdf40 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef30169130 con 0x7fef300cf830 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fef30169430 con 0x7fef3015d700 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53526/0 (socket says 172.21.15.83:53526) 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 -- 172.21.15.83:0/485159424 learned_addr learned my addr 172.21.15.83:0/485159424 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 -- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef3015d700 msgr2=0x7fef3015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef300cf830 0x7fef300ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef3015d700 0x7fef3015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 -- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef300cf830 msgr2=0x7fef300ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef300cf830 0x7fef300ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 -- 172.21.15.83:0/485159424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3015aa30 con 0x7fef300cdf40 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef2e59c640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fef2000d620 tx=0x7fef2000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef1effd640 1 -- 172.21.15.83:0/485159424 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2000b620 con 0x7fef300cdf40 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef1effd640 1 -- 172.21.15.83:0/485159424 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef20019070 con 0x7fef300cdf40 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef1effd640 1 -- 172.21.15.83:0/485159424 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2000ba00 con 0x7fef300cdf40 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3015c900 con 0x7fef300cdf40 2024-05-17T22:00:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.196+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef30156930 con 0x7fef300cdf40 2024-05-17T22:00:45.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef1effd640 1 -- 172.21.15.83:0/485159424 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fef2000bba0 con 0x7fef300cdf40 2024-05-17T22:00:45.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef1effd640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 0x7fef10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 0x7fef10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef1effd640 1 -- 172.21.15.83:0/485159424 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fef2008f640 con 0x7fef300cdf40 2024-05-17T22:00:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 0x7fef10071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef30076440 tx=0x7fef24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 Processor -- start 2024-05-17T22:00:45.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 -- start start 2024-05-17T22:00:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x559193996020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x559193996aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559193834fa0 con 0x559193846460 2024-05-17T22:00:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5591937a92e0 con 0x559193830030 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559193798e80 con 0x559193850e00 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53538/0 (socket says 172.21.15.83:53538) 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.200+0000 7fef2dd9b640 1 -- 172.21.15.83:0/1352442887 learned_addr learned my addr 172.21.15.83:0/1352442887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2e59c640 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x559193996020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 -- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 msgr2=0x559193996020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x559193996020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 -- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 msgr2=0x559193996aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x559193996aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 -- 172.21.15.83:0/1352442887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559193843b80 con 0x559193846460 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef24005c20 tx=0x7fef24010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9eddd1fd720571 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef0ffff640 1 -- 172.21.15.83:0/1352442887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24005e50 con 0x559193846460 2024-05-17T22:00:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef0ffff640 1 -- 172.21.15.83:0/1352442887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2400da20 con 0x559193846460 2024-05-17T22:00:45.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef0ffff640 1 -- 172.21.15.83:0/1352442887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2400ddf0 con 0x559193846460 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- 172.21.15.83:0/1352442887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 msgr2=0x559193996560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef24005c20 tx=0x7fef24010000 comp rx=0 tx=0).stop 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- 172.21.15.83:0/1352442887 shutdown_connections 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x559193996aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x559193996560 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- 172.21.15.83:0/1352442887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x559193996020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- 172.21.15.83:0/1352442887 >> 172.21.15.83:0/1352442887 conn(0x55919345a760 msgr2=0x559193953270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- 172.21.15.83:0/1352442887 shutdown_connections 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- 172.21.15.83:0/1352442887 wait complete. 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- start start 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 Processor -- start 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193846460 0x55919399d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x55919399dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559193754eb0 con 0x559193830030 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55919379cec0 con 0x559193846460 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559193462820 con 0x559193850e00 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef3c139000 10 client.0 _ll_register_callbacks cb 0x559193938d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53548/0 (socket says 172.21.15.83:53548) 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.204+0000 7fef2e59c640 1 -- 172.21.15.83:0/4015415206 learned_addr learned my addr 172.21.15.83:0/4015415206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2ed9d640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x55919399dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 -- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 msgr2=0x55919399dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x55919399dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193846460 0x55919399d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 -- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193846460 msgr2=0x55919399d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193846460 0x55919399d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 -- 172.21.15.83:0/4015415206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591937e1ef0 con 0x559193830030 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2e59c640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef2000b0e0 tx=0x7fef2005b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb6fc212febce5b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2005c5c0 con 0x559193830030 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fef20011b90 con 0x559193830030 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2400da20 con 0x559193830030 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef2005bec0 con 0x559193830030 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 10 client.5913 fetch_fsmap learned FSMap version 34 2024-05-17T22:00:45.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5591937d6910 con 0x559193830030 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2400da20 con 0x559193830030 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fef20013070 con 0x559193830030 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 10 client.5913 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 10 client.5913 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 20 client.5913 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919343fad0 0x5591939a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 4 client.5913 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559193734cf0 con 0x55919343fad0 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919343fad0 0x5591939a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 10 client.5913 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919343fad0 0x5591939a7680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27985bf9691bee89 server_cookie=2b4b53178f4bbb00 in_seq=0 out_seq=0 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 1 -- 172.21.15.83:0/4015415206 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559193734cf0 con 0x55919343fad0 2024-05-17T22:00:45.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 10 client.5913 handle_command_reply: tid=0 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 client.5913 shutdown 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 msgr2=0x55919399d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef2000b0e0 tx=0x7fef2005b940 comp rx=0 tx=0).stop 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 shutdown_connections 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x55919399dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919343fad0 0x5591939a7680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193846460 0x55919399d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 --2- 172.21.15.83:0/4015415206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919399d2a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 >> 172.21.15.83:0/4015415206 conn(0x55919345a760 msgr2=0x55919394b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef0d7fa640 0 client.5913 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 shutdown_connections 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 1 -- 172.21.15.83:0/4015415206 wait complete. 2024-05-17T22:00:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.208+0000 7fef3c139000 20 client.5913 trim_cache size 0 max 16384 2024-05-17T22:00:45.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.220+0000 7fef3c139000 1 Processor -- start 2024-05-17T22:00:45.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- start start 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193846460 0x55919394e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193850e00 0x55919399d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef30205450 con 0x559193830030 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef30204570 con 0x559193850e00 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fef301f90f0 con 0x559193846460 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53562/0 (socket says 172.21.15.83:53562) 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 -- 172.21.15.83:0/975052993 learned_addr learned my addr 172.21.15.83:0/975052993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 -- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193846460 msgr2=0x55919394e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193846460 0x55919394e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 -- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193850e00 msgr2=0x55919399d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193850e00 0x55919399d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 -- 172.21.15.83:0/975052993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591939491c0 con 0x559193830030 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef2e59c640 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fef2001d000 tx=0x7fef2008f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25498a3b68fecf99 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef0f7fe640 1 -- 172.21.15.83:0/975052993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20064ce0 con 0x559193830030 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef0f7fe640 1 -- 172.21.15.83:0/975052993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2005c5c0 con 0x559193830030 2024-05-17T22:00:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef0f7fe640 1 -- 172.21.15.83:0/975052993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20011d90 con 0x559193830030 2024-05-17T22:00:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- 172.21.15.83:0/975052993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 msgr2=0x55919394e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fef2001d000 tx=0x7fef2008f070 comp rx=0 tx=0).stop 2024-05-17T22:00:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- 172.21.15.83:0/975052993 shutdown_connections 2024-05-17T22:00:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193850e00 0x55919399d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193846460 0x55919394e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 --2- 172.21.15.83:0/975052993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193830030 0x55919394e480 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- 172.21.15.83:0/975052993 >> 172.21.15.83:0/975052993 conn(0x55919345a760 msgr2=0x55919394dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- 172.21.15.83:0/975052993 shutdown_connections 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- 172.21.15.83:0/975052993 wait complete. 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 -- start start 2024-05-17T22:00:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.224+0000 7fef3c139000 1 Processor -- start 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x5591939a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x5591939c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559193858260 con 0x559193846460 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef301f9740 con 0x559193830030 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55919371dc10 con 0x559193850e00 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 10 client.0 _ll_register_callbacks cb 0x7fef2005bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53568/0 (socket says 172.21.15.83:53568) 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 -- 172.21.15.83:0/3005112487 learned_addr learned my addr 172.21.15.83:0/3005112487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 -- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 msgr2=0x5591939c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x5591939c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 -- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 msgr2=0x5591939a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x5591939a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 -- 172.21.15.83:0/3005112487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559193756670 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2dd9b640 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef24005bc0 tx=0x7fef24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5cc880176d2d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2400ddb0 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fef240306f0 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2005c5c0 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef24030930 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 10 client.5919 fetch_fsmap learned FSMap version 34 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559193755100 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2005c5c0 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fef24030ad0 con 0x559193846460 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 10 client.5919 fetch_fsmap finished waiting for FSMap version 34 2024-05-17T22:00:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 10 client.5919 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 20 client.5919 populate_metadata read hostname 'smithi083' 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919342db30 0x5591939c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 4 client.5919 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 --> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5591937ab230 con 0x55919342db30 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2e59c640 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919342db30 0x5591939c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 10 client.5919 ms_handle_connect on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef2e59c640 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919342db30 0x5591939c5e50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71472603a70683f3 server_cookie=45e8043bef3d3ba6 in_seq=0 out_seq=0 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 1 -- 172.21.15.83:0/3005112487 <== mds.0 v2:172.21.15.83:6836/3998902165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591937ab230 con 0x55919342db30 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 10 client.5919 handle_command_reply: tid=0 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 client.5919 shutdown 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 msgr2=0x5591939a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fef24005bc0 tx=0x7fef24005800 comp rx=0 tx=0).stop 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 shutdown_connections 2024-05-17T22:00:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559193850e00 0x5591939c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559193846460 0x5591939a32f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.83:6836/3998902165,v1:172.21.15.83:6837/3998902165] conn(0x55919342db30 0x5591939c5e50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 --2- 172.21.15.83:0/3005112487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559193830030 0x5591939a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef0d7fa640 0 client.5919 ms_handle_reset on v2:172.21.15.83:6836/3998902165 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 >> 172.21.15.83:0/3005112487 conn(0x55919345a760 msgr2=0x55919394e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 shutdown_connections 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 1 -- 172.21.15.83:0/3005112487 wait complete. 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef3c139000 20 client.5919 trim_cache size 0 max 16384 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:45.227 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-05-17T22:00:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 msgr2=0x7fef10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 0x7fef10071910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef30076440 tx=0x7fef24009040 comp rx=0 tx=0).stop 2024-05-17T22:00:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 msgr2=0x7fef30152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fef2000d620 tx=0x7fef2000dae0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 shutdown_connections 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef3015d700 0x7fef3015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef1006f400 0x7fef10071910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef300cf830 0x7fef300ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 --2- 172.21.15.83:0/485159424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef300cdf40 0x7fef30152f10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 >> 172.21.15.83:0/485159424 conn(0x7fef3006cd90 msgr2=0x7fef30154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 shutdown_connections 2024-05-17T22:00:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.228+0000 7fef34ebf640 1 -- 172.21.15.83:0/485159424 wait complete. 2024-05-17T22:00:45.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:00:45.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 Processor -- start 2024-05-17T22:00:45.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 -- start start 2024-05-17T22:00:45.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f170415ffa0 0x7f17040d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f170406e3d0 0x7f17040d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17040d34e0 con 0x7f17040d1150 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f17040d3780 con 0x7f170406e3d0 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170c319640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f17040d3a60 con 0x7f170415ffa0 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a8b6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f170406e3d0 0x7f17040d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f17098b4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f17040d1150 0x7f170406dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48948/0 (socket says 172.21.15.83:48948) 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f17098b4640 1 -- 172.21.15.83:0/600791022 learned_addr learned my addr 172.21.15.83:0/600791022 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a0b5640 1 --2- 172.21.15.83:0/600791022 >> v2:172.21.15.155:3300/0 conn(0x7f170415ffa0 0x7f17040d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a8b6640 1 -- 172.21.15.83:0/600791022 >> v2:172.21.15.155:3300/0 conn(0x7f170415ffa0 msgr2=0x7f17040d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a8b6640 1 --2- 172.21.15.83:0/600791022 >> v2:172.21.15.155:3300/0 conn(0x7f170415ffa0 0x7f17040d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a8b6640 1 -- 172.21.15.83:0/600791022 >> v1:172.21.15.83:6789/0 conn(0x7f17040d1150 legacy=0x7f170406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.388+0000 7f170a8b6640 1 -- 172.21.15.83:0/600791022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17040d3d40 con 0x7f170406e3d0 2024-05-17T22:00:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170a8b6640 1 --2- 172.21.15.83:0/600791022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f170406e3d0 0x7f17040d4260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f170408e380 tx=0x7f16fc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f72b33d696928 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f17090b3640 1 -- 172.21.15.83:0/600791022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0b2070 con 0x7f170406e3d0 2024-05-17T22:00:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f17090b3640 1 -- 172.21.15.83:0/600791022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc0a57f0 con 0x7f170406e3d0 2024-05-17T22:00:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f17090b3640 1 -- 172.21.15.83:0/600791022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0a5bc0 con 0x7f170406e3d0 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- 172.21.15.83:0/600791022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f170406e3d0 msgr2=0x7f17040d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 --2- 172.21.15.83:0/600791022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f170406e3d0 0x7f17040d4260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f170408e380 tx=0x7f16fc01c930 comp rx=0 tx=0).stop 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- 172.21.15.83:0/600791022 shutdown_connections 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 --2- 172.21.15.83:0/600791022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f170406e3d0 0x7f17040d4260 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 --2- 172.21.15.83:0/600791022 >> v2:172.21.15.155:3300/0 conn(0x7f170415ffa0 0x7f17040d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- 172.21.15.83:0/600791022 >> 172.21.15.83:0/600791022 conn(0x7f17040d28f0 msgr2=0x7f17040d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- 172.21.15.83:0/600791022 shutdown_connections 2024-05-17T22:00:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- 172.21.15.83:0/600791022 wait complete. 2024-05-17T22:00:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 Processor -- start 2024-05-17T22:00:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.392+0000 7f170c319640 1 -- start start 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17040d1150 0x7f1704164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170415ffa0 0x7f170415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17040d5340 con 0x7f17041697b0 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f17040d51c0 con 0x7f17040d1150 2024-05-17T22:00:45.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f17040d54c0 con 0x7f170415ffa0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53572/0 (socket says 172.21.15.83:53572) 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 -- 172.21.15.83:0/1760669896 learned_addr learned my addr 172.21.15.83:0/1760669896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a0b5640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17040d1150 0x7f1704164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f17098b4640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170415ffa0 0x7f170415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 -- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170415ffa0 msgr2=0x7f170415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170415ffa0 0x7f170415cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 -- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17040d1150 msgr2=0x7f1704164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17040d1150 0x7f1704164b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 -- 172.21.15.83:0/1760669896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041f91f0 con 0x7f17041697b0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170a8b6640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16fc019540 tx=0x7f16fc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0b2050 con 0x7f17041697b0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc0be070 con 0x7f17041697b0 2024-05-17T22:00:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17041f94d0 con 0x7f17041697b0 2024-05-17T22:00:45.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0ad870 con 0x7f17041697b0 2024-05-17T22:00:45.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041f9a90 con 0x7f17041697b0 2024-05-17T22:00:45.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1704165100 con 0x7f17041697b0 2024-05-17T22:00:45.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.396+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16fc0ada10 con 0x7f17041697b0 2024-05-17T22:00:45.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.400+0000 7f16faffd640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 0x7f16e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.400+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f16fc12a3e0 con 0x7f17041697b0 2024-05-17T22:00:45.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.400+0000 7f170a0b5640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 0x7f16e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.400+0000 7f170a0b5640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 0x7f16e8071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f170000b930 tx=0x7f1700008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.404+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f16fc016520 con 0x7f17041697b0 2024-05-17T22:00:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.600+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f17040d4260 con 0x7f17041697b0 2024-05-17T22:00:45.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.600+0000 7f16faffd640 1 -- 172.21.15.83:0/1760669896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f16fc0f7360 con 0x7f17041697b0 2024-05-17T22:00:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:45.595 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 msgr2=0x7f16e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 0x7f16e8071910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f170000b930 tx=0x7f1700008040 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 msgr2=0x7f1704161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16fc019540 tx=0x7f16fc0196d0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 shutdown_connections 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041697b0 0x7f1704161e80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f170415ffa0 0x7f170415cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f16e806f400 0x7f16e8071910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 --2- 172.21.15.83:0/1760669896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17040d1150 0x7f1704164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 >> 172.21.15.83:0/1760669896 conn(0x7f17040d28f0 msgr2=0x7f170416a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:45.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 shutdown_connections 2024-05-17T22:00:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:45.604+0000 7f170c319640 1 -- 172.21.15.83:0/1760669896 wait complete. 2024-05-17T22:00:45.615 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-17T22:00:45.615 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:00:45.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:00:45.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:00:45.747 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:00:45.747 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:00:45.747 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:00:45.747 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:45.752 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:00:45.752 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:45.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:00:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 Processor -- start 2024-05-17T22:00:45.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 -- start start 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2b00b08a0 0x7fa2b015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b006f2d0 con 0x7fa2b00b44c0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b006f5a0 con 0x7fa2b006eec0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2b4e9e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b015c3b0 con 0x7fa2b00b08a0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2aed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa2b00b44c0 0x7fa2b0072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:48962/0 (socket says 172.21.15.83:48962) 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2aed9d640 1 -- 172.21.15.83:0/1105744457 learned_addr learned my addr 172.21.15.83:0/1105744457 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435739878 0 0) 0x7fa2b006f2d0 con 0x7fa2b00b44c0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa29c003530 con 0x7fa2b00b44c0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2a7fff640 1 --2- 172.21.15.83:0/1105744457 >> v2:172.21.15.155:3300/0 conn(0x7fa2b00b08a0 0x7fa2b015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 867393547 0 0) 0x7fa29c003530 con 0x7fa2b00b44c0 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 >> v2:172.21.15.155:3300/0 conn(0x7fa2b00b08a0 msgr2=0x7fa2b015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 --2- 172.21.15.83:0/1105744457 >> v2:172.21.15.155:3300/0 conn(0x7fa2b00b08a0 0x7fa2b015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 >> v1:172.21.15.155:6789/0 conn(0x7fa2b006eec0 legacy=0x7fa2b01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b006f880 con 0x7fa2b00b44c0 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.000+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fa2a8001a80 con 0x7fa2b00b44c0 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 >> v1:172.21.15.83:6789/0 conn(0x7fa2b00b44c0 legacy=0x7fa2b0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c003a60 0x7fa29c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 0x7fa29c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c0053b0 0x7fa29c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015c3b0 con 0x7fa29c004810 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c00e550 con 0x7fa29c0053b0 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa29c00e750 con 0x7fa29c003a60 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 0x7fa29c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2aed9d640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c0053b0 0x7fa29c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2ae59c640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c003a60 0x7fa29c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 -- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c003a60 msgr2=0x7fa29c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa29c003a60 0x7fa29c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 -- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c0053b0 msgr2=0x7fa29c00dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c0053b0 0x7fa29c00dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2aed9d640 1 -- 172.21.15.83:0/1105744457 reap_dead start 2024-05-17T22:00:45.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 -- 172.21.15.83:0/1105744457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c003530 con 0x7fa29c004810 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2a7fff640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 0x7fa29c004c20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00b56c0 tx=0x7fa29801c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93754a1b24d59f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980b3070 con 0x7fa29c004810 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2980ae680 con 0x7fa29c004810 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.004+0000 7fa2add9b640 1 -- 172.21.15.83:0/1105744457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29800b610 con 0x7fa29c004810 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 msgr2=0x7fa29c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:45.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 0x7fa29c004c20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00b56c0 tx=0x7fa29801c610 comp rx=0 tx=0).stop 2024-05-17T22:00:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/1105744457 shutdown_connections 2024-05-17T22:00:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/1105744457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa29c004810 0x7fa29c004c20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/1105744457 >> 172.21.15.83:0/1105744457 conn(0x7fa2b006cd90 msgr2=0x7fa2b00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/1105744457 shutdown_connections 2024-05-17T22:00:46.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/1105744457 wait complete. 2024-05-17T22:00:46.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 Processor -- start 2024-05-17T22:00:46.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- start start 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c004810 0x7fa2b0152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2b015ec90 0x7fa2b0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0160190 con 0x7fa2b0140960 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b00b4ce0 con 0x7fa29c004810 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2b4e9e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b0160310 con 0x7fa2b015ec90 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53598/0 (socket says 172.21.15.83:53598) 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.008+0000 7fa2aed9d640 1 -- 172.21.15.83:0/2926541826 learned_addr learned my addr 172.21.15.83:0/2926541826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 -- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2b015ec90 msgr2=0x7fa2b0153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2b015ec90 0x7fa2b0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 -- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c004810 msgr2=0x7fa2b0152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2ae59c640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c004810 0x7fa2b0152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c004810 0x7fa2b0152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 -- 172.21.15.83:0/2926541826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b015dbf0 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2aed9d640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00cea50 tx=0x7fa2a8007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800abe0 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a8018070 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b015ded0 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80133f0 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b015e490 con 0x7fa2b0140960 2024-05-17T22:00:46.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b01606a0 con 0x7fa2b0140960 2024-05-17T22:00:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa2a8013590 con 0x7fa2b0140960 2024-05-17T22:00:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 0x7fa290071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2ae59c640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 0x7fa290071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fa2a808f9f0 con 0x7fa2b0140960 2024-05-17T22:00:46.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.012+0000 7fa2ae59c640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 0x7fa290071940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2b006b180 tx=0x7fa2a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:46.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.016+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa2a805c8e0 con 0x7fa2b0140960 2024-05-17T22:00:46.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.176+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2b00b0f10 con 0x7fa2b0140960 2024-05-17T22:00:46.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.176+0000 7fa2a6ffd640 1 -- 172.21.15.83:0/2926541826 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7fa2a805c1e0 con 0x7fa2b0140960 2024-05-17T22:00:46.170 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 msgr2=0x7fa290071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 0x7fa290071940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2b006b180 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 msgr2=0x7fa2b0140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00cea50 tx=0x7fa2a8007b20 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 shutdown_connections 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0140960 0x7fa2b0140d70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2b015ec90 0x7fa2b0153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa29006f430 0x7fa290071940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 --2- 172.21.15.83:0/2926541826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa29c004810 0x7fa2b0152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 >> 172.21.15.83:0/2926541826 conn(0x7fa2b006cd90 msgr2=0x7fa2b0072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:46.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 shutdown_connections 2024-05-17T22:00:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.180+0000 7fa2b4e9e640 1 -- 172.21.15.83:0/2926541826 wait complete. 2024-05-17T22:00:46.191 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:00:46.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:46.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:00:46.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[39467]: fuse finished with error 0 and tester_r 0 2024-05-17T22:00:46.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:00:46.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:00:46.202 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:00:46.367 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:00:46.367 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:00:46.367 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:46.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:00:46.374 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:00:46.374 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:00:46.374 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:46.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:00:46.415 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:00:46.416 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:00:46.416 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:46.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:00:46.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:46.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 Processor -- start 2024-05-17T22:00:46.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 -- start start 2024-05-17T22:00:46.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43a006ffd0 0x7f43a00703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 0x7f43a01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a01465d0 con 0x7f43a01461c0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f43a0160ef0 con 0x7f43a006ffd0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a6e4d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f43a006ef30 con 0x7f43a01605a0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43a006ffd0 0x7f43a00703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f43a006ffd0 0x7f43a00703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53612/0 (socket says 172.21.15.83:53612) 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 -- 172.21.15.83:0/2978245247 learned_addr learned my addr 172.21.15.83:0/2978245247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a53ea640 1 --2- 172.21.15.83:0/2978245247 >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 0x7f43a01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 -- 172.21.15.83:0/2978245247 >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 msgr2=0x7f43a01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 --2- 172.21.15.83:0/2978245247 >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 0x7f43a01609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 -- 172.21.15.83:0/2978245247 >> v1:172.21.15.155:6789/0 conn(0x7f43a01461c0 legacy=0x7f43a015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f4397fff640 1 -- 172.21.15.83:0/2978245247 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a0070920 con 0x7f43a006ffd0 2024-05-17T22:00:46.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.620+0000 7f43a53ea640 1 --2- 172.21.15.83:0/2978245247 >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 0x7f43a01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:46.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f4397fff640 1 --2- 172.21.15.83:0/2978245247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a006ffd0 0x7f43a00703e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f43a0072c00 tx=0x7f439001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2b4fab824d9083 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:46.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43977fe640 1 -- 172.21.15.83:0/2978245247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900aa610 con 0x7f43a006ffd0 2024-05-17T22:00:46.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43977fe640 1 -- 172.21.15.83:0/2978245247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43900af070 con 0x7f43a006ffd0 2024-05-17T22:00:46.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43977fe640 1 -- 172.21.15.83:0/2978245247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900aa9d0 con 0x7f43a006ffd0 2024-05-17T22:00:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 -- 172.21.15.83:0/2978245247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a006ffd0 msgr2=0x7f43a00703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/2978245247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a006ffd0 0x7f43a00703e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f43a0072c00 tx=0x7f439001c4f0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 -- 172.21.15.83:0/2978245247 shutdown_connections 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/2978245247 >> v2:172.21.15.155:3300/0 conn(0x7f43a01605a0 0x7f43a01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/2978245247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a006ffd0 0x7f43a00703e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 -- 172.21.15.83:0/2978245247 >> 172.21.15.83:0/2978245247 conn(0x7f43a006cd90 msgr2=0x7f43a00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 -- 172.21.15.83:0/2978245247 shutdown_connections 2024-05-17T22:00:46.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.624+0000 7f43a6e4d640 1 -- 172.21.15.83:0/2978245247 wait complete. 2024-05-17T22:00:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 Processor -- start 2024-05-17T22:00:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- start start 2024-05-17T22:00:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 0x7f43a01452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01461c0 0x7f43a0138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43a015e880 con 0x7f43a01605a0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f43a006fab0 con 0x7f43a01461c0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f43a00d10a0 con 0x7f43a006ffd0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53622/0 (socket says 172.21.15.83:53622) 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 -- 172.21.15.83:0/361733646 learned_addr learned my addr 172.21.15.83:0/361733646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a4be9640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 0x7f43a01452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f4397fff640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01461c0 0x7f43a0138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 -- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 msgr2=0x7f43a01452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 0x7f43a01452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 -- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01461c0 msgr2=0x7f43a0138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01461c0 0x7f43a0138e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 -- 172.21.15.83:0/361733646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00b12c0 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a4be9640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 0x7f43a01452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a53ea640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f439800cd50 tx=0x7f439800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800f680 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439800f840 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a00b15a0 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800fc30 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.628+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a013a7e0 con 0x7f43a01605a0 2024-05-17T22:00:46.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a015a7d0 con 0x7f43a01605a0 2024-05-17T22:00:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43980213f0 con 0x7f43a01605a0 2024-05-17T22:00:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43957fa640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 0x7f438c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f439808ff90 con 0x7f43a01605a0 2024-05-17T22:00:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43a4be9640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 0x7f438c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.632+0000 7f43a4be9640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 0x7f438c071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f43a00b11f0 tx=0x7f439c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.636+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4398014030 con 0x7f43a01605a0 2024-05-17T22:00:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.832+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43a00703e0 con 0x7f43a01605a0 2024-05-17T22:00:46.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.832+0000 7f43957fa640 1 -- 172.21.15.83:0/361733646 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7f439805cf10 con 0x7f43a01605a0 2024-05-17T22:00:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:46.825 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T22:00:17.620242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:46.826 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 msgr2=0x7f438c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 0x7f438c071940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f43a00b11f0 tx=0x7f439c008040 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 msgr2=0x7f43a0139380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f439800cd50 tx=0x7f439800a650 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 shutdown_connections 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a01605a0 0x7f43a0139380 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f438c06f430 0x7f438c071940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01461c0 0x7f43a0138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 --2- 172.21.15.83:0/361733646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f43a006ffd0 0x7f43a01452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 >> 172.21.15.83:0/361733646 conn(0x7f43a006cd90 msgr2=0x7f43a00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 shutdown_connections 2024-05-17T22:00:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:46.836+0000 7f43a6e4d640 1 -- 172.21.15.83:0/361733646 wait complete. 2024-05-17T22:00:46.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 Processor -- start 2024-05-17T22:00:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 -- start start 2024-05-17T22:00:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d27c0 0x7fc9d015fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9d00d14c0 0x7fc9d00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9d0071110 con 0x7fc9d0070990 2024-05-17T22:00:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d0070500 con 0x7fc9d00d14c0 2024-05-17T22:00:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d8396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d00713b0 con 0x7fc9d00d27c0 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9d00d14c0 0x7fc9d00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9d00d14c0 0x7fc9d00b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43512/0 (socket says 172.21.15.83:43512) 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 -- 172.21.15.83:0/3053781868 learned_addr learned my addr 172.21.15.83:0/3053781868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d5931640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc9d0070990 0x7fc9d00d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55584/0 (socket says 172.21.15.83:55584) 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6132640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d27c0 0x7fc9d015fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 -- 172.21.15.83:0/3053781868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d27c0 msgr2=0x7fc9d015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d27c0 0x7fc9d015fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 -- 172.21.15.83:0/3053781868 >> v1:172.21.15.83:6789/0 conn(0x7fc9d0070990 legacy=0x7fc9d00d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d6933640 1 -- 172.21.15.83:0/3053781868 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d0070190 con 0x7fc9d00d14c0 2024-05-17T22:00:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.008+0000 7fc9d5130640 1 -- 172.21.15.83:0/3053781868 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415525006 0 0) 0x7fc9d0071110 con 0x7fc9d0070990 2024-05-17T22:00:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d6933640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d00d14c0 0x7fc9d00b1350 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0070680 tx=0x7fc9c801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4d6af1ca1e878e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d5130640 1 -- 172.21.15.83:0/3053781868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a55b0 con 0x7fc9d00d14c0 2024-05-17T22:00:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d5130640 1 -- 172.21.15.83:0/3053781868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a5750 con 0x7fc9d00d14c0 2024-05-17T22:00:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d5130640 1 -- 172.21.15.83:0/3053781868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a5b20 con 0x7fc9d00d14c0 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- 172.21.15.83:0/3053781868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d00d14c0 msgr2=0x7fc9d00b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d00d14c0 0x7fc9d00b1350 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0070680 tx=0x7fc9c801c8f0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- 172.21.15.83:0/3053781868 shutdown_connections 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d00d14c0 0x7fc9d00b1350 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 --2- 172.21.15.83:0/3053781868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d27c0 0x7fc9d015fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- 172.21.15.83:0/3053781868 >> 172.21.15.83:0/3053781868 conn(0x7fc9d00d1ca0 msgr2=0x7fc9d00d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- 172.21.15.83:0/3053781868 shutdown_connections 2024-05-17T22:00:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- 172.21.15.83:0/3053781868 wait complete. 2024-05-17T22:00:47.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 Processor -- start 2024-05-17T22:00:47.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.012+0000 7fc9d8396640 1 -- start start 2024-05-17T22:00:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d14c0 0x7fc9d015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9d00d27c0 0x7fc9d00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d006d330 con 0x7fc9d0070990 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d006d1b0 con 0x7fc9d00d27c0 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d006d4b0 con 0x7fc9d00d14c0 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43524/0 (socket says 172.21.15.83:43524) 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 -- 172.21.15.83:0/1035570409 learned_addr learned my addr 172.21.15.83:0/1035570409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d5931640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d14c0 0x7fc9d015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 -- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d14c0 msgr2=0x7fc9d015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d14c0 0x7fc9d015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6933640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9d00d27c0 0x7fc9d00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 -- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9d00d27c0 msgr2=0x7fc9d00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9d00d27c0 0x7fc9d00b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 -- 172.21.15.83:0/1035570409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01545c0 con 0x7fc9d0070990 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d6132640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000d5f0 tx=0x7fc9c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00125f0 con 0x7fc9d0070990 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c0019070 con 0x7fc9d0070990 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d0159610 con 0x7fc9d0070990 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00129b0 con 0x7fc9d0070990 2024-05-17T22:00:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d0154b60 con 0x7fc9d0070990 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d00d19e0 con 0x7fc9d0070990 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.016+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc9c0012b50 con 0x7fc9d0070990 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.020+0000 7fc9ceffd640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 0x7fc9bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.020+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc9c008f5a0 con 0x7fc9d0070990 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.020+0000 7fc9d5931640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 0x7fc9bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.020+0000 7fc9d5931640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 0x7fc9bc071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000f70 tx=0x7fc9c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.024+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc9c005c520 con 0x7fc9d0070990 2024-05-17T22:00:47.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.220+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9d00b5170 con 0x7fc9d0070990 2024-05-17T22:00:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9ceffd640 1 -- 172.21.15.83:0/1035570409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7fc9c005be20 con 0x7fc9d0070990 2024-05-17T22:00:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:47.216 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T22:00:17.620242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:47.216 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-05-17T22:00:47.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 msgr2=0x7fc9bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 0x7fc9bc071910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000f70 tx=0x7fc9c4009040 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 msgr2=0x7fc9d0155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000d5f0 tx=0x7fc9c000dab0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 shutdown_connections 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc9bc06f400 0x7fc9bc071910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9d00d27c0 0x7fc9d00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9d00d14c0 0x7fc9d015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 --2- 172.21.15.83:0/1035570409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9d0070990 0x7fc9d0155360 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 >> 172.21.15.83:0/1035570409 conn(0x7fc9d00d1ca0 msgr2=0x7fc9d015eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 shutdown_connections 2024-05-17T22:00:47.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.224+0000 7fc9d8396640 1 -- 172.21.15.83:0/1035570409 wait complete. 2024-05-17T22:00:47.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:00:47.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 Processor -- start 2024-05-17T22:00:47.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 -- start start 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f038c1602e0 0x7f038c06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f038c09bcc0 0x7f038c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038c160850 con 0x7f038c15d9d0 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f038c160b30 con 0x7f038c09bcc0 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f039452c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f038c160e10 con 0x7f038c1602e0 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0391ac7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f038c09bcc0 0x7f038c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0392ac9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f038c15d9d0 0x7f038c1638d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55590/0 (socket says 172.21.15.83:55590) 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0392ac9640 1 -- 172.21.15.83:0/2238145743 learned_addr learned my addr 172.21.15.83:0/2238145743 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f03922c8640 1 --2- 172.21.15.83:0/2238145743 >> v2:172.21.15.155:3300/0 conn(0x7f038c1602e0 0x7f038c06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0391ac7640 1 -- 172.21.15.83:0/2238145743 >> v2:172.21.15.155:3300/0 conn(0x7f038c1602e0 msgr2=0x7f038c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0391ac7640 1 --2- 172.21.15.83:0/2238145743 >> v2:172.21.15.155:3300/0 conn(0x7f038c1602e0 0x7f038c06c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0391ac7640 1 -- 172.21.15.83:0/2238145743 >> v1:172.21.15.83:6789/0 conn(0x7f038c15d9d0 legacy=0x7f038c1638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.396+0000 7f0391ac7640 1 -- 172.21.15.83:0/2238145743 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c1610f0 con 0x7f038c09bcc0 2024-05-17T22:00:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f0391ac7640 1 --2- 172.21.15.83:0/2238145743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c06ccd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f038c0d3530 tx=0x7f037c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a86d01c0794462 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f03912c6640 1 -- 172.21.15.83:0/2238145743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c0b1070 con 0x7f038c09bcc0 2024-05-17T22:00:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f03912c6640 1 -- 172.21.15.83:0/2238145743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c0ac5f0 con 0x7f038c09bcc0 2024-05-17T22:00:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f03912c6640 1 -- 172.21.15.83:0/2238145743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c0ac9c0 con 0x7f038c09bcc0 2024-05-17T22:00:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 -- 172.21.15.83:0/2238145743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 msgr2=0x7f038c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 --2- 172.21.15.83:0/2238145743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c06ccd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f038c0d3530 tx=0x7f037c01c480 comp rx=0 tx=0).stop 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 -- 172.21.15.83:0/2238145743 shutdown_connections 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 --2- 172.21.15.83:0/2238145743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c06ccd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 --2- 172.21.15.83:0/2238145743 >> v2:172.21.15.155:3300/0 conn(0x7f038c1602e0 0x7f038c06c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 -- 172.21.15.83:0/2238145743 >> 172.21.15.83:0/2238145743 conn(0x7f038c09b8b0 msgr2=0x7f038c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 -- 172.21.15.83:0/2238145743 shutdown_connections 2024-05-17T22:00:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.400+0000 7f039452c640 1 -- 172.21.15.83:0/2238145743 wait complete. 2024-05-17T22:00:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 Processor -- start 2024-05-17T22:00:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 -- start start 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f038c15d9d0 0x7f038c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 0x7f038c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f038c16cdd0 con 0x7f038c09bcc0 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f038c16cc50 con 0x7f038c15d9d0 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f038c16cf50 con 0x7f038c1602e0 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43534/0 (socket says 172.21.15.83:43534) 2024-05-17T22:00:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 -- 172.21.15.83:0/4220182380 learned_addr learned my addr 172.21.15.83:0/4220182380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0391ac7640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f038c15d9d0 0x7f038c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0392ac9640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 0x7f038c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 -- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 msgr2=0x7f038c158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 0x7f038c158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 -- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f038c15d9d0 msgr2=0x7f038c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f038c15d9d0 0x7f038c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 -- 172.21.15.83:0/4220182380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c154730 con 0x7f038c09bcc0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0392ac9640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 0x7f038c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f03922c8640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f038800d620 tx=0x7f038800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880125f0 con 0x7f038c09bcc0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0388019070 con 0x7f038c09bcc0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c1549a0 con 0x7f038c09bcc0 2024-05-17T22:00:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.404+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880129b0 con 0x7f038c09bcc0 2024-05-17T22:00:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c154f60 con 0x7f038c09bcc0 2024-05-17T22:00:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038c1638d0 con 0x7f038c09bcc0 2024-05-17T22:00:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0388012b50 con 0x7f038c09bcc0 2024-05-17T22:00:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f0382ffd640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 0x7f0370071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f0388090180 con 0x7f038c09bcc0 2024-05-17T22:00:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f0391ac7640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 0x7f0370071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.408+0000 7f0391ac7640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 0x7f0370071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f037c0ae000 tx=0x7f037c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.412+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f038805d150 con 0x7f038c09bcc0 2024-05-17T22:00:47.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.576+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f038c163610 con 0x7f038c09bcc0 2024-05-17T22:00:47.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.576+0000 7f0382ffd640 1 -- 172.21.15.83:0/4220182380 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16641 (secure 0 0 0) 0x7f038805ca50 con 0x7f038c09bcc0 2024-05-17T22:00:47.569 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:47.569 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:58.567137+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:59:54.592647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-17T21:59:55.893117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 msgr2=0x7f0370071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 0x7f0370071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f037c0ae000 tx=0x7f037c008040 comp rx=0 tx=0).stop 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 msgr2=0x7f038c14d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f038800d620 tx=0x7f038800dae0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 shutdown_connections 2024-05-17T22:00:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f038c1602e0 0x7f038c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f037006f400 0x7f0370071910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f038c15d9d0 0x7f038c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 --2- 172.21.15.83:0/4220182380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f038c09bcc0 0x7f038c14d2d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 >> 172.21.15.83:0/4220182380 conn(0x7f038c09b8b0 msgr2=0x7f038c156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 shutdown_connections 2024-05-17T22:00:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.580+0000 7f039452c640 1 -- 172.21.15.83:0/4220182380 wait complete. 2024-05-17T22:00:47.592 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:00:47.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:47.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 Processor -- start 2024-05-17T22:00:47.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 -- start start 2024-05-17T22:00:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9e6006ffd0 0x7f9e600703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e601609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e601465d0 con 0x7f9e601461c0 2024-05-17T22:00:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9e60160ef0 con 0x7f9e6006ffd0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e65c8a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6006ef30 con 0x7f9e601605a0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9e601461c0 0x7f9e6015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55600/0 (socket says 172.21.15.83:55600) 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5f7fe640 1 -- 172.21.15.83:0/1912953466 learned_addr learned my addr 172.21.15.83:0/1912953466 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5ffff640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e601609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5effd640 1 --2- 172.21.15.83:0/1912953466 >> v2:172.21.15.155:3300/0 conn(0x7f9e6006ffd0 0x7f9e600703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 176381356 0 0) 0x7f9e601465d0 con 0x7f9e601461c0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e48003530 con 0x7f9e601461c0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1618368206 0 0) 0x7f9e48003530 con 0x7f9e601461c0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 msgr2=0x7f9e601609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e601609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 >> v2:172.21.15.155:3300/0 conn(0x7f9e6006ffd0 msgr2=0x7f9e600703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 --2- 172.21.15.83:0/1912953466 >> v2:172.21.15.155:3300/0 conn(0x7f9e6006ffd0 0x7f9e600703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e60070920 con 0x7f9e601461c0 2024-05-17T22:00:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9e50001700 con 0x7f9e601461c0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 >> v1:172.21.15.83:6789/0 conn(0x7f9e601461c0 legacy=0x7f9e6015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e48003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e48003fe0 0x7f9e48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e48004d50 0x7f9e48005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60160ef0 con 0x7f9e48003fe0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006ef30 con 0x7f9e48004d50 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4800bad0 con 0x7f9e601605a0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5effd640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e48003fe0 0x7f9e48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.752+0000 7f9e5f7fe640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e48003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5ffff640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e48004d50 0x7f9e48005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 -- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 msgr2=0x7f9e48003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e601605a0 0x7f9e48003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 -- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e48004d50 msgr2=0x7f9e48005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:47.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e48004d50 0x7f9e48005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 -- 172.21.15.83:0/1912953466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e48003530 con 0x7f9e48003fe0 2024-05-17T22:00:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5effd640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e48003fe0 0x7f9e48004810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009ae0 tx=0x7f9e5401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2667aee628a0691 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540b1070 con 0x7f9e48003fe0 2024-05-17T22:00:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e540a4ab0 con 0x7f9e48003fe0 2024-05-17T22:00:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.756+0000 7f9e5e7fc640 1 -- 172.21.15.83:0/1912953466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e540096b0 con 0x7f9e48003fe0 2024-05-17T22:00:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- 172.21.15.83:0/1912953466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e48003fe0 msgr2=0x7f9e48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/1912953466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e48003fe0 0x7f9e48004810 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009ae0 tx=0x7f9e5401c720 comp rx=0 tx=0).stop 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e5ffff640 1 -- 172.21.15.83:0/1912953466 reap_dead start 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- 172.21.15.83:0/1912953466 shutdown_connections 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- 172.21.15.83:0/1912953466 >> 172.21.15.83:0/1912953466 conn(0x7f9e6006cd90 msgr2=0x7f9e600b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- 172.21.15.83:0/1912953466 shutdown_connections 2024-05-17T22:00:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- 172.21.15.83:0/1912953466 wait complete. 2024-05-17T22:00:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 Processor -- start 2024-05-17T22:00:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- start start 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e60140ef0 0x7f9e601f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6015f860 0x7f9e600b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6015e880 con 0x7f9e6015fc70 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6006fab0 con 0x7f9e6015f860 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e65c8a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9e600d0ca0 con 0x7f9e60140ef0 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43544/0 (socket says 172.21.15.83:43544) 2024-05-17T22:00:47.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.760+0000 7f9e5ffff640 1 -- 172.21.15.83:0/521857707 learned_addr learned my addr 172.21.15.83:0/521857707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5effd640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6015f860 0x7f9e600b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5f7fe640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e60140ef0 0x7f9e601f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 -- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e60140ef0 msgr2=0x7f9e601f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e60140ef0 0x7f9e601f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 -- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6015f860 msgr2=0x7f9e600b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6015f860 0x7f9e600b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 -- 172.21.15.83:0/521857707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e60141580 con 0x7f9e6015fc70 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5ffff640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9e60160500 tx=0x7f9e580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58011e10 con 0x7f9e6015fc70 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5801b070 con 0x7f9e6015fc70 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e60141300 con 0x7f9e6015fc70 2024-05-17T22:00:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e580039a0 con 0x7f9e6015fc70 2024-05-17T22:00:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e600af480 con 0x7f9e6015fc70 2024-05-17T22:00:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e600d1030 con 0x7f9e6015fc70 2024-05-17T22:00:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e58003b40 con 0x7f9e6015fc70 2024-05-17T22:00:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 0x7f9e4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5808fa90 con 0x7f9e6015fc70 2024-05-17T22:00:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5f7fe640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 0x7f9e4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.764+0000 7f9e5f7fe640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 0x7f9e4c071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b6ba0 tx=0x7f9e50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:47.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.772+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9e58025020 con 0x7f9e6015fc70 2024-05-17T22:00:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.968+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e600b41a0 con 0x7f9e6015fc70 2024-05-17T22:00:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.968+0000 7f9e47fff640 1 -- 172.21.15.83:0/521857707 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7f9e60141300 con 0x7f9e6015fc70 2024-05-17T22:00:47.962 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:47.962 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T22:00:17.620242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 msgr2=0x7f9e4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 0x7f9e4c071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9e600b6ba0 tx=0x7f9e50009040 comp rx=0 tx=0).stop 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 msgr2=0x7f9e601fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9e60160500 tx=0x7f9e580115a0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 shutdown_connections 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e6015fc70 0x7f9e601fa970 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9e4c06f400 0x7f9e4c071910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e6015f860 0x7f9e600b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 --2- 172.21.15.83:0/521857707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9e60140ef0 0x7f9e601f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:47.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 >> 172.21.15.83:0/521857707 conn(0x7f9e6006cd90 msgr2=0x7f9e6015ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:47.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 shutdown_connections 2024-05-17T22:00:47.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:47.972+0000 7f9e65c8a640 1 -- 172.21.15.83:0/521857707 wait complete. 2024-05-17T22:00:47.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:48.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 Processor -- start 2024-05-17T22:00:48.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 -- start start 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f64a406ffd0 0x7f64a40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a41465d0 con 0x7f64a41461c0 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a4070920 con 0x7f64a41605a0 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a9b7c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f64a406ef30 con 0x7f64a406ffd0 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f64a41461c0 0x7f64a415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55610/0 (socket says 172.21.15.83:55610) 2024-05-17T22:00:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a37fe640 1 -- 172.21.15.83:0/2591418442 learned_addr learned my addr 172.21.15.83:0/2591418442 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:48.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 --2- 172.21.15.83:0/2591418442 >> v2:172.21.15.155:3300/0 conn(0x7f64a406ffd0 0x7f64a40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767450845 0 0) 0x7f64a41465d0 con 0x7f64a41461c0 2024-05-17T22:00:48.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6490005740 con 0x7f64a41461c0 2024-05-17T22:00:48.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1467215907 0 0) 0x7f6490005740 con 0x7f64a41461c0 2024-05-17T22:00:48.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 >> v2:172.21.15.155:3300/0 conn(0x7f64a406ffd0 msgr2=0x7f64a40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 --2- 172.21.15.83:0/2591418442 >> v2:172.21.15.155:3300/0 conn(0x7f64a406ffd0 0x7f64a40703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 >> v1:172.21.15.155:6789/0 conn(0x7f64a41605a0 legacy=0x7f64a41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a406f210 con 0x7f64a41461c0 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f648c001700 con 0x7f64a41461c0 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 >> v1:172.21.15.83:6789/0 conn(0x7f64a41461c0 legacy=0x7f64a415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490005c20 0x7f6490006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 0x7f6490006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64900073e0 0x7f649000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406ef30 con 0x7f6490006a20 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f649000e380 con 0x7f6490005c20 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f649000e580 con 0x7f64900073e0 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 0x7f6490006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a37fe640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490005c20 0x7f6490006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 -- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64900073e0 msgr2=0x7f649000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64900073e0 0x7f649000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 -- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490005c20 msgr2=0x7f6490006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490005c20 0x7f6490006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f649bfff640 1 -- 172.21.15.83:0/2591418442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6490005740 con 0x7f6490006a20 2024-05-17T22:00:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a3fff640 1 -- 172.21.15.83:0/2591418442 reap_dead start 2024-05-17T22:00:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.144+0000 7f64a37fe640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490005c20 0x7f6490006050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f649bfff640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 0x7f6490006e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ae0 tx=0x7f649401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107ad4fb9e7e95c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b2070 con 0x7f6490006a20 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940a5660 con 0x7f6490006a20 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a2ffd640 1 -- 172.21.15.83:0/2591418442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940096b0 con 0x7f6490006a20 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 -- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 msgr2=0x7f6490006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 0x7f6490006e30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ae0 tx=0x7f649401ca70 comp rx=0 tx=0).stop 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 -- 172.21.15.83:0/2591418442 shutdown_connections 2024-05-17T22:00:48.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/2591418442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6490006a20 0x7f6490006e30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 -- 172.21.15.83:0/2591418442 >> 172.21.15.83:0/2591418442 conn(0x7f64a406cd90 msgr2=0x7f64a40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:48.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 -- 172.21.15.83:0/2591418442 shutdown_connections 2024-05-17T22:00:48.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.148+0000 7f64a9b7c640 1 -- 172.21.15.83:0/2591418442 wait complete. 2024-05-17T22:00:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 Processor -- start 2024-05-17T22:00:48.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- start start 2024-05-17T22:00:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490006a20 0x7f64a40aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 0x7f64a40b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406fab0 con 0x7f64a413da60 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406f7d0 con 0x7f6490006a20 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f64a40703e0 con 0x7f64a40b6090 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43564/0 (socket says 172.21.15.83:43564) 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 -- 172.21.15.83:0/3264870443 learned_addr learned my addr 172.21.15.83:0/3264870443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a37fe640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490006a20 0x7f64a40aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a3fff640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 0x7f64a40b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 -- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 msgr2=0x7f64a40b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 0x7f64a40b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 -- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490006a20 msgr2=0x7f64a40aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490006a20 0x7f64a40aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 -- 172.21.15.83:0/3264870443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a40b69e0 con 0x7f64a413da60 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a3fff640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 0x7f64a40b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f649bfff640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f649401c8a0 tx=0x7f6494019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b2040 con 0x7f64a413da60 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494004020 con 0x7f64a413da60 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940043f0 con 0x7f64a413da60 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a40b52d0 con 0x7f64a413da60 2024-05-17T22:00:48.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.152+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a40b5890 con 0x7f64a413da60 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a4070770 con 0x7f64a413da60 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64940bf030 con 0x7f64a413da60 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a0ff9640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 0x7f6484071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a37fe640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 0x7f6484071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f6494129a20 con 0x7f64a413da60 2024-05-17T22:00:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.156+0000 7f64a37fe640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 0x7f6484071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f648c001ce0 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.160+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f64940c6060 con 0x7f64a413da60 2024-05-17T22:00:48.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64a41453d0 con 0x7f64a413da60 2024-05-17T22:00:48.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a0ff9640 1 -- 172.21.15.83:0/3264870443 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7f6494016780 con 0x7f64a413da60 2024-05-17T22:00:48.344 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:48.345 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T21:59:56.927208+0000","modified":"2024-05-17T22:00:17.620242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":8,"addr":"172.21.15.83:6837/3998902165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3998902165},{"type":"v1","addr":"172.21.15.83:6837","nonce":3998902165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":4}]} 2024-05-17T22:00:48.345 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 34 2024-05-17T22:00:48.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 msgr2=0x7f6484071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 0x7f6484071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f648c001ce0 tx=0x7f648c009040 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 msgr2=0x7f64a414e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f649401c8a0 tx=0x7f6494019810 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 shutdown_connections 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64a40b6090 0x7f64a40b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64a413da60 0x7f64a414e860 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f648406f400 0x7f6484071910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 --2- 172.21.15.83:0/3264870443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6490006a20 0x7f64a40aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 >> 172.21.15.83:0/3264870443 conn(0x7f64a406cd90 msgr2=0x7f64a4140e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 shutdown_connections 2024-05-17T22:00:48.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.352+0000 7f64a9b7c640 1 -- 172.21.15.83:0/3264870443 wait complete. 2024-05-17T22:00:48.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:00:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.520+0000 7fa4cad8d640 1 Processor -- start 2024-05-17T22:00:48.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 -- start start 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 0x7fa4c406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c406e610 con 0x7fa4c415ffa0 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c40d34e0 con 0x7fa4c406dcc0 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4cad8d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c40d3780 con 0x7fa4c40d4260 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 0x7fa4c406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:38264/0 (socket says 172.21.15.83:38264) 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 0x7fa4c406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:37208/0 (socket says 172.21.15.83:37208) 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 -- 172.21.15.83:0/2444042107 learned_addr learned my addr 172.21.15.83:0/2444042107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:48.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c8b29640 1 --1- 172.21.15.83:0/2444042107 >> v1:172.21.15.155:6789/0 conn(0x7fa4c415ffa0 0x7fa4c40d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:46924/0 (socket says 172.21.15.83:46924) 2024-05-17T22:00:48.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 -- 172.21.15.83:0/2444042107 >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 msgr2=0x7fa4c406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 --2- 172.21.15.83:0/2444042107 >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 0x7fa4c406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 -- 172.21.15.83:0/2444042107 >> v1:172.21.15.155:6789/0 conn(0x7fa4c415ffa0 legacy=0x7fa4c40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.524+0000 7fa4c932a640 1 -- 172.21.15.83:0/2444042107 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40d3a60 con 0x7fa4c40d4260 2024-05-17T22:00:48.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4c932a640 1 --2- 172.21.15.83:0/2444042107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e50 tx=0x7fa4bc01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=916f3a1f74e74c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4c37fe640 1 -- 172.21.15.83:0/2444042107 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0a5780 con 0x7fa4c40d4260 2024-05-17T22:00:48.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4c37fe640 1 -- 172.21.15.83:0/2444042107 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0a5920 con 0x7fa4c40d4260 2024-05-17T22:00:48.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4c37fe640 1 -- 172.21.15.83:0/2444042107 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0a5cf0 con 0x7fa4c40d4260 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2444042107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 msgr2=0x7fa4c40d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2444042107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009e50 tx=0x7fa4bc01ca30 comp rx=0 tx=0).stop 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2444042107 shutdown_connections 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2444042107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c40d4260 0x7fa4c40d4670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2444042107 >> v2:172.21.15.155:3300/0 conn(0x7fa4c406dcc0 0x7fa4c406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2444042107 >> 172.21.15.83:0/2444042107 conn(0x7fa4c40d28f0 msgr2=0x7fa4c40d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:48.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2444042107 shutdown_connections 2024-05-17T22:00:48.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.528+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2444042107 wait complete. 2024-05-17T22:00:48.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 Processor -- start 2024-05-17T22:00:48.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- start start 2024-05-17T22:00:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 0x7fa4c40bb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c40d4260 0x7fa4c4152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40d17e0 con 0x7fa4c415ffa0 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40d1660 con 0x7fa4c40d4260 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c416cb80 con 0x7fa4c406dcc0 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43576/0 (socket says 172.21.15.83:43576) 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 -- 172.21.15.83:0/2543331330 learned_addr learned my addr 172.21.15.83:0/2543331330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c8b29640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 0x7fa4c40bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c3fff640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c40d4260 0x7fa4c4152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 -- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 msgr2=0x7fa4c40bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 0x7fa4c40bb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 -- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c40d4260 msgr2=0x7fa4c4152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c40d4260 0x7fa4c4152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 -- 172.21.15.83:0/2543331330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40ba7e0 con 0x7fa4c415ffa0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c8b29640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 0x7fa4c40bb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c932a640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc01a660 tx=0x7fa4bc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0a5780 con 0x7fa4c415ffa0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0be070 con 0x7fa4c415ffa0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c40baac0 con 0x7fa4c415ffa0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0b88d0 con 0x7fa4c415ffa0 2024-05-17T22:00:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c4161200 con 0x7fa4c415ffa0 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.532+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c4068b80 con 0x7fa4c415ffa0 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.536+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa4bc0b8a70 con 0x7fa4c415ffa0 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.536+0000 7fa4c17fa640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 0x7fa4a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.536+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fa4bc12a720 con 0x7fa4c415ffa0 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.536+0000 7fa4c8b29640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 0x7fa4a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.536+0000 7fa4c8b29640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 0x7fa4a4071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000f70 tx=0x7fa4b4007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.540+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa4bc0c7020 con 0x7fa4c415ffa0 2024-05-17T22:00:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.700+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4c406e0d0 con 0x7fa4c415ffa0 2024-05-17T22:00:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4c17fa640 1 -- 172.21.15.83:0/2543331330 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16641 (secure 0 0 0) 0x7fa4bc016620 con 0x7fa4c415ffa0 2024-05-17T22:00:48.696 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:48.696 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":56,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T21:59:58.567137+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-17T21:59:54.592647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-17T21:59:55.893117+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 msgr2=0x7fa4a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 0x7fa4a4071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000f70 tx=0x7fa4b4007d50 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 msgr2=0x7fa4c41549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc01a660 tx=0x7fa4bc0197d0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 shutdown_connections 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4c415ffa0 0x7fa4c41549a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa4a406f430 0x7fa4a4071940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4c40d4260 0x7fa4c4152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 --2- 172.21.15.83:0/2543331330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4c406dcc0 0x7fa4c40bb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 >> 172.21.15.83:0/2543331330 conn(0x7fa4c40d28f0 msgr2=0x7fa4c40b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 shutdown_connections 2024-05-17T22:00:48.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.704+0000 7fa4cad8d640 1 -- 172.21.15.83:0/2543331330 wait complete. 2024-05-17T22:00:48.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 Processor -- start 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 -- start start 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2c9815dbb0 0x7f2c980d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980d20a0 con 0x7f2c980cde70 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980d2380 con 0x7f2c980d3580 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c9e6f1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9815dfc0 con 0x7f2c9815dbb0 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c97fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2c980cde70 0x7f2c980ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55616/0 (socket says 172.21.15.83:55616) 2024-05-17T22:00:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c97fff640 1 -- 172.21.15.83:0/135675605 learned_addr learned my addr 172.21.15.83:0/135675605 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:48.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c977fe640 1 --2- 172.21.15.83:0/135675605 >> v2:172.21.15.155:3300/0 conn(0x7f2c9815dbb0 0x7f2c980d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034476835 0 0) 0x7f2c980d20a0 con 0x7f2c980cde70 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c84003530 con 0x7f2c980cde70 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399315099 0 0) 0x7f2c84003530 con 0x7f2c980cde70 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 >> v2:172.21.15.155:3300/0 conn(0x7f2c9815dbb0 msgr2=0x7f2c980d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 --2- 172.21.15.83:0/135675605 >> v2:172.21.15.155:3300/0 conn(0x7f2c9815dbb0 0x7f2c980d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.864+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 >> v1:172.21.15.155:6789/0 conn(0x7f2c980d3580 legacy=0x7f2c980d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c980d2660 con 0x7f2c980cde70 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2c90001a80 con 0x7f2c980cde70 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 >> v1:172.21.15.83:6789/0 conn(0x7f2c980cde70 legacy=0x7f2c980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84003a60 0x7f2c84003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 0x7f2c84004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c84005260 0x7f2c8400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9815dfc0 con 0x7f2c84004810 2024-05-17T22:00:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8400c1f0 con 0x7f2c84003a60 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8400c3f0 con 0x7f2c84005260 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 0x7f2c84004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c97fff640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84003a60 0x7f2c84003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 -- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c84005260 msgr2=0x7f2c8400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c84005260 0x7f2c8400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 -- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84003a60 msgr2=0x7f2c84003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84003a60 0x7f2c84003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 -- 172.21.15.83:0/135675605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c84003530 con 0x7f2c84004810 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c97fff640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84003a60 0x7f2c84003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9cc8e640 1 -- 172.21.15.83:0/135675605 reap_dead start 2024-05-17T22:00:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c977fe640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 0x7f2c84004c20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009ab0 tx=0x7f2c8801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a96c79e97bcd05c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880b2070 con 0x7f2c84004810 2024-05-17T22:00:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c880a5680 con 0x7f2c84004810 2024-05-17T22:00:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c96ffd640 1 -- 172.21.15.83:0/135675605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c880096b0 con 0x7f2c84004810 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 msgr2=0x7f2c84004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 0x7f2c84004c20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009ab0 tx=0x7f2c8801ca90 comp rx=0 tx=0).stop 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/135675605 shutdown_connections 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/135675605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c84004810 0x7f2c84004c20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/135675605 >> 172.21.15.83:0/135675605 conn(0x7f2c98071180 msgr2=0x7f2c98071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/135675605 shutdown_connections 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.868+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/135675605 wait complete. 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 Processor -- start 2024-05-17T22:00:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- start start 2024-05-17T22:00:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84004810 0x7f2c98151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c981608e0 0x7f2c9806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980d2e60 con 0x7f2c9806e4b0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980d33a0 con 0x7f2c84004810 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2c980d2fe0 con 0x7f2c981608e0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43594/0 (socket says 172.21.15.83:43594) 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 -- 172.21.15.83:0/970672174 learned_addr learned my addr 172.21.15.83:0/970672174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c97fff640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84004810 0x7f2c98151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c977fe640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c981608e0 0x7f2c9806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 -- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c981608e0 msgr2=0x7f2c9806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c981608e0 0x7f2c9806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 -- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84004810 msgr2=0x7f2c98151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84004810 0x7f2c98151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 -- 172.21.15.83:0/970672174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9806e8c0 con 0x7f2c9806e4b0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9cc8e640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2c980d40a0 tx=0x7f2c8000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80007480 con 0x7f2c9806e4b0 2024-05-17T22:00:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80007640 con 0x7f2c9806e4b0 2024-05-17T22:00:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981521c0 con 0x7f2c9806e4b0 2024-05-17T22:00:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004260 con 0x7f2c9806e4b0 2024-05-17T22:00:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.872+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c980b1f60 con 0x7f2c9806e4b0 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c980ce670 con 0x7f2c9806e4b0 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2c80011070 con 0x7f2c9806e4b0 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c94ff9640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 0x7f2c78071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8008e290 con 0x7f2c9806e4b0 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c97fff640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 0x7f2c78071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.876+0000 7f2c97fff640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 0x7f2c78071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c900012f0 tx=0x7f2c90001260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:48.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:48.880+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2c9806e8c0 con 0x7f2c9806e4b0 2024-05-17T22:00:49.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.064+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2c980b2240 con 0x7f2c9806e4b0 2024-05-17T22:00:49.057 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:00:49.064+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:00:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c94ff9640 1 -- 172.21.15.83:0/970672174 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f2c80063020 con 0x7f2c9806e4b0 2024-05-17T22:00:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 msgr2=0x7f2c78071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 0x7f2c78071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c900012f0 tx=0x7f2c90001260 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 msgr2=0x7f2c980c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2c980d40a0 tx=0x7f2c8000c760 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 shutdown_connections 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c9806e4b0 0x7f2c980c70e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c981608e0 0x7f2c9806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c7806f430 0x7f2c78071940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 --2- 172.21.15.83:0/970672174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c84004810 0x7f2c98151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 >> 172.21.15.83:0/970672174 conn(0x7f2c98071180 msgr2=0x7f2c98071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 shutdown_connections 2024-05-17T22:00:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.068+0000 7f2c9e6f1640 1 -- 172.21.15.83:0/970672174 wait complete. 2024-05-17T22:00:49.070 INFO:tasks.ceph.mds.c.smithi083.stderr: -5497> 2024-05-17T21:59:59.504+0000 7f7d2ae0a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:00:49.076 INFO:tasks.ceph.mds.c.smithi083.stderr: -3659> 2024-05-17T22:00:07.484+0000 7f7d26e02640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5739=0-4194304@1} caps={5739=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5739 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55d1ee694b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:00:49.077 INFO:tasks.ceph.mds.c.smithi083.stderr: -2822> 2024-05-17T22:00:14.216+0000 7f7d26e02640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-05-17T22:00:06.309114+0000 1=1+0) caps={5739=pAsLsXsFs/-@2},l=5739(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55d1ee694b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:00:49.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:00:49.138 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:00:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 Processor -- start 2024-05-17T22:00:49.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 -- start start 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70400d5210 0x7f70400d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400d2000 0x7f70400d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70400d4170 con 0x7f70400b4810 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f70400d5620 con 0x7f70400d5210 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f704644d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70400d4450 con 0x7f70400d2000 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70400d5210 0x7f70400d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:49.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70400d5210 0x7f70400d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43618/0 (socket says 172.21.15.83:43618) 2024-05-17T22:00:49.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 -- 172.21.15.83:0/2533605117 learned_addr learned my addr 172.21.15.83:0/2533605117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:49.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 -- 172.21.15.83:0/2533605117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400d2000 msgr2=0x7f70400d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:49.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 --2- 172.21.15.83:0/2533605117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400d2000 0x7f70400d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 -- 172.21.15.83:0/2533605117 >> v1:172.21.15.155:6789/0 conn(0x7f70400b4810 legacy=0x7f704015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:49.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.244+0000 7f703f7fe640 1 -- 172.21.15.83:0/2533605117 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70400d4730 con 0x7f70400d5210 2024-05-17T22:00:49.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f703f7fe640 1 --2- 172.21.15.83:0/2533605117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f70400d0290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7034009870 tx=0x7f703401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed040f77971cf4d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:49.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f703effd640 1 -- 172.21.15.83:0/2533605117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340aa660 con 0x7f70400d5210 2024-05-17T22:00:49.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f703effd640 1 -- 172.21.15.83:0/2533605117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70340af070 con 0x7f70400d5210 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f703effd640 1 -- 172.21.15.83:0/2533605117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340aaa20 con 0x7f70400d5210 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- 172.21.15.83:0/2533605117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 msgr2=0x7f70400d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 --2- 172.21.15.83:0/2533605117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f70400d0290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7034009870 tx=0x7f703401c560 comp rx=0 tx=0).stop 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- 172.21.15.83:0/2533605117 shutdown_connections 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 --2- 172.21.15.83:0/2533605117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400d2000 0x7f70400d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 --2- 172.21.15.83:0/2533605117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f70400d0290 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- 172.21.15.83:0/2533605117 >> 172.21.15.83:0/2533605117 conn(0x7f704006e010 msgr2=0x7f704006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- 172.21.15.83:0/2533605117 shutdown_connections 2024-05-17T22:00:49.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- 172.21.15.83:0/2533605117 wait complete. 2024-05-17T22:00:49.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 Processor -- start 2024-05-17T22:00:49.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.248+0000 7f704644d640 1 -- start start 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400b4810 0x7f704015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70400d2000 0x7f7040152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70400b9670 con 0x7f70400d5210 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70400b94f0 con 0x7f70400d2000 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70400b97f0 con 0x7f70400b4810 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43622/0 (socket says 172.21.15.83:43622) 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 -- 172.21.15.83:0/36477522 learned_addr learned my addr 172.21.15.83:0/36477522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 -- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400b4810 msgr2=0x7f704015a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703ffff640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400b4810 0x7f704015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703f7fe640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70400d2000 0x7f7040152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400b4810 0x7f704015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 -- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70400d2000 msgr2=0x7f7040152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70400d2000 0x7f7040152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 -- 172.21.15.83:0/36477522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040153450 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f70449ea640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f703800b9e0 tx=0x7f703800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038019070 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703800cb40 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7040161720 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038014440 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7040161ce0 con 0x7f70400d5210 2024-05-17T22:00:49.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70400d2410 con 0x7f70400d5210 2024-05-17T22:00:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f703801c070 con 0x7f70400d5210 2024-05-17T22:00:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 0x7f702c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:49.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5795+0+0 (secure 0 0 0) 0x7f703808f0c0 con 0x7f70400d5210 2024-05-17T22:00:49.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703ffff640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 0x7f702c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:49.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.252+0000 7f703ffff640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 0x7f702c071940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f703000a790 tx=0x7f7030008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:49.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.256+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f703805beb0 con 0x7f70400d5210 2024-05-17T22:00:49.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:49.448+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f704015bd30 con 0x7f70400d5210 2024-05-17T22:00:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.068+0000 7f703cff9640 1 -- 172.21.15.83:0/36477522 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7f703805b7b0 con 0x7f70400d5210 2024-05-17T22:00:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 msgr2=0x7f702c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 0x7f702c071940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f703000a790 tx=0x7f7030008040 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 msgr2=0x7f704015d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f703800b9e0 tx=0x7f703800bea0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 shutdown_connections 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70400d5210 0x7f704015d260 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f702c06f430 0x7f702c071940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70400d2000 0x7f7040152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 --2- 172.21.15.83:0/36477522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70400b4810 0x7f704015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 >> 172.21.15.83:0/36477522 conn(0x7f704006e010 msgr2=0x7f70400ba170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 shutdown_connections 2024-05-17T22:00:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.072+0000 7f704644d640 1 -- 172.21.15.83:0/36477522 wait complete. 2024-05-17T22:00:50.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:00:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.248+0000 7f0cab88b640 1 Processor -- start 2024-05-17T22:00:50.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.248+0000 7f0cab88b640 1 -- start start 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0cab88b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ca4073560 0x7f0ca40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0cab88b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ca415df30 0x7f0ca415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0cab88b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca415ec60 con 0x7f0ca40b66c0 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0cab88b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca40b6230 con 0x7f0ca4073560 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0cab88b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca40b5be0 con 0x7f0ca415df30 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ca4073560 0x7f0ca40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ca4073560 0x7f0ca40b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43634/0 (socket says 172.21.15.83:43634) 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 -- 172.21.15.83:0/1135765564 learned_addr learned my addr 172.21.15.83:0/1135765564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca9e28640 1 --2- 172.21.15.83:0/1135765564 >> v2:172.21.15.155:3300/0 conn(0x7f0ca415df30 0x7f0ca415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 -- 172.21.15.83:0/1135765564 >> v2:172.21.15.155:3300/0 conn(0x7f0ca415df30 msgr2=0x7f0ca415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 --2- 172.21.15.83:0/1135765564 >> v2:172.21.15.155:3300/0 conn(0x7f0ca415df30 0x7f0ca415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 -- 172.21.15.83:0/1135765564 >> v1:172.21.15.155:6789/0 conn(0x7f0ca40b66c0 legacy=0x7f0ca4072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 -- 172.21.15.83:0/1135765564 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca40b5ec0 con 0x7f0ca4073560 2024-05-17T22:00:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0ca8e26640 1 --2- 172.21.15.83:0/1135765564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4073560 0x7f0ca40b0bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40b59f0 tx=0x7f0c9401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1c79ee90a4016f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.252+0000 7f0c9bfff640 1 -- 172.21.15.83:0/1135765564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940ab430 con 0x7f0ca4073560 2024-05-17T22:00:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0c9bfff640 1 -- 172.21.15.83:0/1135765564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c940b0070 con 0x7f0ca4073560 2024-05-17T22:00:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0c9bfff640 1 -- 172.21.15.83:0/1135765564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940ab7f0 con 0x7f0ca4073560 2024-05-17T22:00:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- 172.21.15.83:0/1135765564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4073560 msgr2=0x7f0ca40b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- 172.21.15.83:0/1135765564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4073560 0x7f0ca40b0bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40b59f0 tx=0x7f0c9401c520 comp rx=0 tx=0).stop 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- 172.21.15.83:0/1135765564 shutdown_connections 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- 172.21.15.83:0/1135765564 >> v2:172.21.15.155:3300/0 conn(0x7f0ca415df30 0x7f0ca415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- 172.21.15.83:0/1135765564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4073560 0x7f0ca40b0bc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- 172.21.15.83:0/1135765564 >> 172.21.15.83:0/1135765564 conn(0x7f0ca406cd90 msgr2=0x7f0ca40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- 172.21.15.83:0/1135765564 shutdown_connections 2024-05-17T22:00:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- 172.21.15.83:0/1135765564 wait complete. 2024-05-17T22:00:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 Processor -- start 2024-05-17T22:00:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- start start 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca40b66c0 0x7f0ca4143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca415df30 0x7f0ca415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b5060 con 0x7f0ca4157860 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b4ee0 con 0x7f0ca415df30 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.256+0000 7f0cab88b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca40b51e0 con 0x7f0ca40b66c0 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43648/0 (socket says 172.21.15.83:43648) 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 -- 172.21.15.83:0/2890017190 learned_addr learned my addr 172.21.15.83:0/2890017190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca8e26640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca415df30 0x7f0ca415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9627640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca40b66c0 0x7f0ca4143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 -- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca40b66c0 msgr2=0x7f0ca4143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca40b66c0 0x7f0ca4143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 -- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca415df30 msgr2=0x7f0ca415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca415df30 0x7f0ca415b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 -- 172.21.15.83:0/2890017190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca4144330 con 0x7f0ca4157860 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9e28640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c007e60 tx=0x7f0c9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c014420 con 0x7f0ca4157860 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c019070 con 0x7f0ca4157860 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca4144610 con 0x7f0ca4157860 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c014820 con 0x7f0ca4157860 2024-05-17T22:00:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca4159050 con 0x7f0ca4157860 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca41447e0 con 0x7f0ca4157860 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c9c0149c0 con 0x7f0ca4157860 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 0x7f0c88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9627640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 0x7f0c88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5795+0+0 (secure 0 0 0) 0x7f0c9c08faf0 con 0x7f0ca4157860 2024-05-17T22:00:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.260+0000 7f0ca9627640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 0x7f0c88071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0001570 tx=0x7f0ca0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.264+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0ca41447e0 con 0x7f0ca4157860 2024-05-17T22:00:50.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:50.416+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ca4158b60 con 0x7f0ca4157860 2024-05-17T22:00:51.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.216+0000 7f0c99ffb640 1 -- 172.21.15.83:0/2890017190 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7f0c9c05c3c0 con 0x7f0ca4157860 2024-05-17T22:00:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:00:51.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 msgr2=0x7f0c88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 0x7f0c88071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0001570 tx=0x7f0ca0009040 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 msgr2=0x7f0ca415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c007e60 tx=0x7f0c9c00e4b0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 shutdown_connections 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca4157860 0x7f0ca415b6b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f430 0x7f0c88071940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca415df30 0x7f0ca415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 --2- 172.21.15.83:0/2890017190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca40b66c0 0x7f0ca4143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 >> 172.21.15.83:0/2890017190 conn(0x7f0ca406cd90 msgr2=0x7f0ca40d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 shutdown_connections 2024-05-17T22:00:51.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.220+0000 7f0cab88b640 1 -- 172.21.15.83:0/2890017190 wait complete. 2024-05-17T22:00:51.239 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:00:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 Processor -- start 2024-05-17T22:00:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 -- start start 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 0x7fb05c0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb05c073560 0x7fb05c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb05c0b4ea0 con 0x7fb05c0aef50 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb05c0b5460 con 0x7fb05c073560 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb060982640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c0b5180 con 0x7fb05c15fa70 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb059d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb05c0aef50 0x7fb05c072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55628/0 (socket says 172.21.15.83:55628) 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb059d9b640 1 -- 172.21.15.83:0/1193764152 learned_addr learned my addr 172.21.15.83:0/1193764152 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05ad9d640 1 --2- 172.21.15.83:0/1193764152 >> v2:172.21.15.83:3300/0 conn(0x7fb05c073560 0x7fb05c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05a59c640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 0x7fb05c0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05ad9d640 1 -- 172.21.15.83:0/1193764152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 msgr2=0x7fb05c0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05ad9d640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 0x7fb05c0b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05ad9d640 1 -- 172.21.15.83:0/1193764152 >> v1:172.21.15.83:6789/0 conn(0x7fb05c0aef50 legacy=0x7fb05c072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05ad9d640 1 -- 172.21.15.83:0/1193764152 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c06efa0 con 0x7fb05c073560 2024-05-17T22:00:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05959a640 1 -- 172.21.15.83:0/1193764152 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985804844 0 0) 0x7fb05c0b4ea0 con 0x7fb05c0aef50 2024-05-17T22:00:51.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.408+0000 7fb05a59c640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 0x7fb05c0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb05ad9d640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c073560 0x7fb05c0af360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa88555f0f4fda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb05959a640 1 -- 172.21.15.83:0/1193764152 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0b2070 con 0x7fb05c073560 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb05959a640 1 -- 172.21.15.83:0/1193764152 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04c0a58a0 con 0x7fb05c073560 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb05959a640 1 -- 172.21.15.83:0/1193764152 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0a5c70 con 0x7fb05c073560 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 -- 172.21.15.83:0/1193764152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c073560 msgr2=0x7fb05c0af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c073560 0x7fb05c0af360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c01c8b0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 -- 172.21.15.83:0/1193764152 shutdown_connections 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c073560 0x7fb05c0af360 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 --2- 172.21.15.83:0/1193764152 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15fa70 0x7fb05c0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 -- 172.21.15.83:0/1193764152 >> 172.21.15.83:0/1193764152 conn(0x7fb05c06cd90 msgr2=0x7fb05c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 -- 172.21.15.83:0/1193764152 shutdown_connections 2024-05-17T22:00:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.412+0000 7fb060982640 1 -- 172.21.15.83:0/1193764152 wait complete. 2024-05-17T22:00:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 Processor -- start 2024-05-17T22:00:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- start start 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0aef50 0x7fb05c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15b620 0x7fb05c1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c0b4b80 con 0x7fb05c15fa70 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c0b4a00 con 0x7fb05c0aef50 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c16c7e0 con 0x7fb05c15b620 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43666/0 (socket says 172.21.15.83:43666) 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 -- 172.21.15.83:0/779022870 learned_addr learned my addr 172.21.15.83:0/779022870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb05ad9d640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15b620 0x7fb05c1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb05a59c640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0aef50 0x7fb05c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 -- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15b620 msgr2=0x7fb05c1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15b620 0x7fb05c1521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 -- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0aef50 msgr2=0x7fb05c14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0aef50 0x7fb05c14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 -- 172.21.15.83:0/779022870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c14f320 con 0x7fb05c15fa70 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb059d9b640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c460 tx=0x7fb05000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050018070 con 0x7fb05c15fa70 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb050007500 con 0x7fb05c15fa70 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c150b40 con 0x7fb05c15fa70 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050004a30 con 0x7fb05c15fa70 2024-05-17T22:00:51.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c151100 con 0x7fb05c15fa70 2024-05-17T22:00:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.416+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c0af360 con 0x7fb05c15fa70 2024-05-17T22:00:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.420+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb05001b070 con 0x7fb05c15fa70 2024-05-17T22:00:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.420+0000 7fb04affd640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 0x7fb03c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.420+0000 7fb05a59c640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 0x7fb03c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.420+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5357+0+0 (secure 0 0 0) 0x7fb05008e3f0 con 0x7fb05c15fa70 2024-05-17T22:00:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.420+0000 7fb05a59c640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 0x7fb03c071940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb054006fd0 tx=0x7fb054008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.424+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb05005b3a0 con 0x7fb05c15fa70 2024-05-17T22:00:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:51.584+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb05c0b47e0 con 0x7fb05c15fa70 2024-05-17T22:00:52.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.220+0000 7fb04affd640 1 -- 172.21.15.83:0/779022870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7fb05005aca0 con 0x7fb05c15fa70 2024-05-17T22:00:52.212 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T22:00:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 msgr2=0x7fb03c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 0x7fb03c071940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb054006fd0 tx=0x7fb054008040 comp rx=0 tx=0).stop 2024-05-17T22:00:52.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 msgr2=0x7fb05c15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c460 tx=0x7fb05000c920 comp rx=0 tx=0).stop 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 shutdown_connections 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb05c15b620 0x7fb05c1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb05c15fa70 0x7fb05c15b0e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb03c06f430 0x7fb03c071940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 --2- 172.21.15.83:0/779022870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb05c0aef50 0x7fb05c14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 >> 172.21.15.83:0/779022870 conn(0x7fb05c06cd90 msgr2=0x7fb05c072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 shutdown_connections 2024-05-17T22:00:52.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.224+0000 7fb060982640 1 -- 172.21.15.83:0/779022870 wait complete. 2024-05-17T22:00:52.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T22:00:52.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 Processor -- start 2024-05-17T22:00:52.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 -- start start 2024-05-17T22:00:52.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0b3d00 0x7f258c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f258c143f20 0x7f258c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f258c0733c0 0x7f258c06eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f258c0b4110 con 0x7f258c143f20 2024-05-17T22:00:52.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f258c070770 con 0x7f258c0733c0 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f259409b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0737d0 con 0x7f258c0b3d00 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f258c143f20 0x7f258c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f258c143f20 0x7f258c072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43680/0 (socket says 172.21.15.83:43680) 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 -- 172.21.15.83:0/167833869 learned_addr learned my addr 172.21.15.83:0/167833869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591e37640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0b3d00 0x7f258c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2592638640 1 --2- 172.21.15.83:0/167833869 >> v2:172.21.15.155:3300/0 conn(0x7f258c0733c0 0x7f258c06eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 -- 172.21.15.83:0/167833869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0b3d00 msgr2=0x7f258c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0b3d00 0x7f258c0d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 -- 172.21.15.83:0/167833869 >> v2:172.21.15.155:3300/0 conn(0x7f258c0733c0 msgr2=0x7f258c06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 --2- 172.21.15.83:0/167833869 >> v2:172.21.15.155:3300/0 conn(0x7f258c0733c0 0x7f258c06eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.404+0000 7f2591636640 1 -- 172.21.15.83:0/167833869 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c0d10d0 con 0x7f258c143f20 2024-05-17T22:00:52.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f2591636640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c143f20 0x7f258c072e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f258c072e10 tx=0x7f257c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac3499a8b1ea972 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f2590e35640 1 -- 172.21.15.83:0/167833869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0aa610 con 0x7f258c143f20 2024-05-17T22:00:52.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f2590e35640 1 -- 172.21.15.83:0/167833869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0af070 con 0x7f258c143f20 2024-05-17T22:00:52.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f2590e35640 1 -- 172.21.15.83:0/167833869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0aa9d0 con 0x7f258c143f20 2024-05-17T22:00:52.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 -- 172.21.15.83:0/167833869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c143f20 msgr2=0x7f258c072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c143f20 0x7f258c072e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f258c072e10 tx=0x7f257c01c4f0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 -- 172.21.15.83:0/167833869 shutdown_connections 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 --2- 172.21.15.83:0/167833869 >> v2:172.21.15.155:3300/0 conn(0x7f258c0733c0 0x7f258c06eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c143f20 0x7f258c072e80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 --2- 172.21.15.83:0/167833869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0b3d00 0x7f258c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 -- 172.21.15.83:0/167833869 >> 172.21.15.83:0/167833869 conn(0x7f258c06cd90 msgr2=0x7f258c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 -- 172.21.15.83:0/167833869 shutdown_connections 2024-05-17T22:00:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.408+0000 7f259409b640 1 -- 172.21.15.83:0/167833869 wait complete. 2024-05-17T22:00:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 Processor -- start 2024-05-17T22:00:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- start start 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0733c0 0x7f258c1425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c143f20 0x7f258c140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f258c070c00 con 0x7f258c0b3d00 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f258c144330 con 0x7f258c143f20 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f258c1458c0 con 0x7f258c0733c0 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43692/0 (socket says 172.21.15.83:43692) 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 -- 172.21.15.83:0/3995470862 learned_addr learned my addr 172.21.15.83:0/3995470862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591e37640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0733c0 0x7f258c1425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 -- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0733c0 msgr2=0x7f258c1425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2592638640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c143f20 0x7f258c140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0733c0 0x7f258c1425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 -- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c143f20 msgr2=0x7f258c140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c143f20 0x7f258c140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 -- 172.21.15.83:0/3995470862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c15cae0 con 0x7f258c0b3d00 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f2591636640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ac040 tx=0x7f257c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b4810 con 0x7f258c0b3d00 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0b49d0 con 0x7f258c0b3d00 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c145b10 con 0x7f258c0b3d00 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b4dc0 con 0x7f258c0b3d00 2024-05-17T22:00:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1401d0 con 0x7f258c0b3d00 2024-05-17T22:00:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.412+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c13ffc0 con 0x7f258c0b3d00 2024-05-17T22:00:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.416+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f257c0bc020 con 0x7f258c0b3d00 2024-05-17T22:00:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.416+0000 7f25827fc640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 0x7f2570071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.416+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f257c129c90 con 0x7f258c0b3d00 2024-05-17T22:00:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.416+0000 7f2591e37640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 0x7f2570071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.416+0000 7f2591e37640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 0x7f2570071910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f258800b740 tx=0x7f2588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.420+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f257c0c3060 con 0x7f258c0b3d00 2024-05-17T22:00:52.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.580+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f258c06fc10 con 0x7f258c0b3d00 2024-05-17T22:00:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f25827fc640 1 -- 172.21.15.83:0/3995470862 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f257c0a7070 con 0x7f258c0b3d00 2024-05-17T22:00:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 msgr2=0x7f2570071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 0x7f2570071910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f258800b740 tx=0x7f2588008040 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 msgr2=0x7f258c1409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f257c0ac040 tx=0x7f257c019290 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 shutdown_connections 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f257006f400 0x7f2570071910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f258c143f20 0x7f258c140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f258c0b3d00 0x7f258c1409d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 --2- 172.21.15.83:0/3995470862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258c0733c0 0x7f258c1425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 >> 172.21.15.83:0/3995470862 conn(0x7f258c06cd90 msgr2=0x7f258c0b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 shutdown_connections 2024-05-17T22:00:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.588+0000 7f259409b640 1 -- 172.21.15.83:0/3995470862 wait complete. 2024-05-17T22:00:52.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-17T22:00:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 Processor -- start 2024-05-17T22:00:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 -- start start 2024-05-17T22:00:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 0x7f3e2c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e2c06cea0 0x7f3e2c167380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e2c09c0d0 con 0x7f3e2c09bcc0 2024-05-17T22:00:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e2c0afc40 con 0x7f3e2c06cea0 2024-05-17T22:00:52.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e34517640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3e2c15e110 con 0x7f3e2c0b4910 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e2c06cea0 0x7f3e2c167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e2c06cea0 0x7f3e2c167380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43696/0 (socket says 172.21.15.83:43696) 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 -- 172.21.15.83:0/2838644691 learned_addr learned my addr 172.21.15.83:0/2838644691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e322b3640 1 --2- 172.21.15.83:0/2838644691 >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 0x7f3e2c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 -- 172.21.15.83:0/2838644691 >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 msgr2=0x7f3e2c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 --2- 172.21.15.83:0/2838644691 >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 0x7f3e2c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 -- 172.21.15.83:0/2838644691 >> v1:172.21.15.155:6789/0 conn(0x7f3e2c09bcc0 legacy=0x7f3e2c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e32ab4640 1 -- 172.21.15.83:0/2838644691 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c0b3a20 con 0x7f3e2c06cea0 2024-05-17T22:00:52.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.764+0000 7f3e322b3640 1 --2- 172.21.15.83:0/2838644691 >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 0x7f3e2c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e32ab4640 1 --2- 172.21.15.83:0/2838644691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c06cea0 0x7f3e2c167380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c073ca0 tx=0x7f3e2401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b3daea4f06b2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e312b1640 1 -- 172.21.15.83:0/2838644691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240a5730 con 0x7f3e2c06cea0 2024-05-17T22:00:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e312b1640 1 -- 172.21.15.83:0/2838644691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e240a58d0 con 0x7f3e2c06cea0 2024-05-17T22:00:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e312b1640 1 -- 172.21.15.83:0/2838644691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240a5ca0 con 0x7f3e2c06cea0 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 -- 172.21.15.83:0/2838644691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c06cea0 msgr2=0x7f3e2c167380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 --2- 172.21.15.83:0/2838644691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c06cea0 0x7f3e2c167380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c073ca0 tx=0x7f3e2401c960 comp rx=0 tx=0).stop 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 -- 172.21.15.83:0/2838644691 shutdown_connections 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 --2- 172.21.15.83:0/2838644691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c06cea0 0x7f3e2c167380 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 --2- 172.21.15.83:0/2838644691 >> v2:172.21.15.155:3300/0 conn(0x7f3e2c0b4910 0x7f3e2c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 -- 172.21.15.83:0/2838644691 >> 172.21.15.83:0/2838644691 conn(0x7f3e2c09b8b0 msgr2=0x7f3e2c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 -- 172.21.15.83:0/2838644691 shutdown_connections 2024-05-17T22:00:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.768+0000 7f3e34517640 1 -- 172.21.15.83:0/2838644691 wait complete. 2024-05-17T22:00:52.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 Processor -- start 2024-05-17T22:00:52.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- start start 2024-05-17T22:00:52.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 0x7f3e2c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e2c0b4910 0x7f3e2c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c16a260 con 0x7f3e2c14c180 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c16a0e0 con 0x7f3e2c09bcc0 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c16a3e0 con 0x7f3e2c0b4910 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43698/0 (socket says 172.21.15.83:43698) 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 -- 172.21.15.83:0/157753236 learned_addr learned my addr 172.21.15.83:0/157753236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e322b3640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 0x7f3e2c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e31ab2640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e2c0b4910 0x7f3e2c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 -- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e2c0b4910 msgr2=0x7f3e2c14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e2c0b4910 0x7f3e2c14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 -- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 msgr2=0x7f3e2c14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 0x7f3e2c14a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 -- 172.21.15.83:0/157753236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c15c510 con 0x7f3e2c14c180 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e322b3640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 0x7f3e2c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e32ab4640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3e240af040 tx=0x7f3e24019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240bc020 con 0x7f3e2c14c180 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e240b86d0 con 0x7f3e2c14c180 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c151730 con 0x7f3e2c14c180 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240b8ac0 con 0x7f3e2c14c180 2024-05-17T22:00:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.772+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c151cf0 con 0x7f3e2c14c180 2024-05-17T22:00:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c06c790 con 0x7f3e2c14c180 2024-05-17T22:00:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3e240bf020 con 0x7f3e2c14c180 2024-05-17T22:00:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e22ffd640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 0x7f3e10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f3e24129d70 con 0x7f3e2c14c180 2024-05-17T22:00:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e322b3640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 0x7f3e10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.776+0000 7f3e322b3640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 0x7f3e10071910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b990 tx=0x7f3e28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:52.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:52.780+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3e240165e0 con 0x7f3e2c14c180 2024-05-17T22:00:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.000+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f3e2c156bd0 con 0x7f3e2c14c180 2024-05-17T22:00:53.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.240+0000 7f3e22ffd640 1 -- 172.21.15.83:0/157753236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v180) ==== 169+0+0 (secure 0 0 0) 0x7f3e240f6f60 con 0x7f3e2c14c180 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 msgr2=0x7f3e10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 0x7f3e10071910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b990 tx=0x7f3e28008040 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 msgr2=0x7f3e2c14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3e240af040 tx=0x7f3e24019700 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 shutdown_connections 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e2c14c180 0x7f3e2c14b160 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e2c0b4910 0x7f3e2c14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3e1006f400 0x7f3e10071910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 --2- 172.21.15.83:0/157753236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e2c09bcc0 0x7f3e2c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 >> 172.21.15.83:0/157753236 conn(0x7f3e2c09b8b0 msgr2=0x7f3e2c0b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:53.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 shutdown_connections 2024-05-17T22:00:53.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.244+0000 7f3e34517640 1 -- 172.21.15.83:0/157753236 wait complete. 2024-05-17T22:00:53.258 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-17T22:00:53.258 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-17T22:00:53.258 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T22:00:53.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:00:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.416+0000 7f1be4d83640 1 Processor -- start 2024-05-17T22:00:53.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.416+0000 7f1be4d83640 1 -- start start 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1be01461c0 0x7f1be015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be015e6b0 0x7f1be006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1be0070510 0x7f1be01605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1be01465d0 con 0x7f1be0070510 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1be0070920 con 0x7f1be01461c0 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1be4d83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1be006ef30 con 0x7f1be015e6b0 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1be0070510 0x7f1be01605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1be0070510 0x7f1be01605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43714/0 (socket says 172.21.15.83:43714) 2024-05-17T22:00:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 -- 172.21.15.83:0/1552805003 learned_addr learned my addr 172.21.15.83:0/1552805003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bd7fff640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be015e6b0 0x7f1be006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 -- 172.21.15.83:0/1552805003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be015e6b0 msgr2=0x7f1be006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be015e6b0 0x7f1be006ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bde59c640 1 --2- 172.21.15.83:0/1552805003 >> v2:172.21.15.155:3300/0 conn(0x7f1be01461c0 0x7f1be015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 -- 172.21.15.83:0/1552805003 >> v2:172.21.15.155:3300/0 conn(0x7f1be01461c0 msgr2=0x7f1be015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 --2- 172.21.15.83:0/1552805003 >> v2:172.21.15.155:3300/0 conn(0x7f1be01461c0 0x7f1be015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.420+0000 7f1bded9d640 1 -- 172.21.15.83:0/1552805003 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be006f210 con 0x7f1be0070510 2024-05-17T22:00:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1bded9d640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be0070510 0x7f1be01605a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1be006d5a0 tx=0x7f1bd801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1205087f48ce077e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1bddd9b640 1 -- 172.21.15.83:0/1552805003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80a5750 con 0x7f1be0070510 2024-05-17T22:00:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1bddd9b640 1 -- 172.21.15.83:0/1552805003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd80a58f0 con 0x7f1be0070510 2024-05-17T22:00:53.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1bddd9b640 1 -- 172.21.15.83:0/1552805003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80a5cc0 con 0x7f1be0070510 2024-05-17T22:00:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- 172.21.15.83:0/1552805003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be0070510 msgr2=0x7f1be01605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be0070510 0x7f1be01605a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1be006d5a0 tx=0x7f1bd801c9c0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- 172.21.15.83:0/1552805003 shutdown_connections 2024-05-17T22:00:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be0070510 0x7f1be01605a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 --2- 172.21.15.83:0/1552805003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be015e6b0 0x7f1be006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 --2- 172.21.15.83:0/1552805003 >> v2:172.21.15.155:3300/0 conn(0x7f1be01461c0 0x7f1be015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- 172.21.15.83:0/1552805003 >> 172.21.15.83:0/1552805003 conn(0x7f1be006cd90 msgr2=0x7f1be00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- 172.21.15.83:0/1552805003 shutdown_connections 2024-05-17T22:00:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- 172.21.15.83:0/1552805003 wait complete. 2024-05-17T22:00:53.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 Processor -- start 2024-05-17T22:00:53.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.424+0000 7f1be4d83640 1 -- start start 2024-05-17T22:00:53.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be0070510 0x7f1be015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 0x7f1be0142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00d1220 con 0x7f1be01461c0 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00d10a0 con 0x7f1be015e6b0 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1be00d13a0 con 0x7f1be0070510 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43728/0 (socket says 172.21.15.83:43728) 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2762127813 learned_addr learned my addr 172.21.15.83:0/2762127813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bde59c640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be0070510 0x7f1be015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bded9d640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 0x7f1be0142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be0070510 msgr2=0x7f1be015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be0070510 0x7f1be015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 msgr2=0x7f1be0142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 0x7f1be0142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2762127813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be0164050 con 0x7f1be01461c0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bded9d640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 0x7f1be0142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd7fff640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c850 tx=0x7f1bc800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8015070 con 0x7f1be01461c0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc80079d0 con 0x7f1be01461c0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be0164330 con 0x7f1be01461c0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8007dc0 con 0x7f1be01461c0 2024-05-17T22:00:53.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01648f0 con 0x7f1be01461c0 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be015e170 con 0x7f1be01461c0 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1bc8018070 con 0x7f1be01461c0 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 0x7f1bc0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bde59c640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 0x7f1bc0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.428+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f1bc808ee20 con 0x7f1be01461c0 2024-05-17T22:00:53.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.432+0000 7f1bde59c640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 0x7f1bc0071940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b11f0 tx=0x7f1bd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.436+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1bc805bf70 con 0x7f1be01461c0 2024-05-17T22:00:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1be00b41a0 con 0x7f1be01461c0 2024-05-17T22:00:53.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1bd6ffd640 1 -- 172.21.15.83:0/2762127813 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12996 (secure 0 0 0) 0x7f1bc805b870 con 0x7f1be01461c0 2024-05-17T22:00:53.592 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:53.593 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:00:52.219722+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/3998902165":"2024-05-18T22:00:49.068210+0000","172.21.15.83:6836/3998902165":"2024-05-18T22:00:49.068210+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 msgr2=0x7f1bc0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 0x7f1bc0071940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1be00b11f0 tx=0x7f1bd0008040 comp rx=0 tx=0).stop 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 msgr2=0x7f1be0142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.600+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c850 tx=0x7f1bc800cd10 comp rx=0 tx=0).stop 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 shutdown_connections 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bc006f430 0x7f1bc0071940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1be015e6b0 0x7f1be0142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1be01461c0 0x7f1be0142740 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 --2- 172.21.15.83:0/2762127813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1be0070510 0x7f1be015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 >> 172.21.15.83:0/2762127813 conn(0x7f1be006cd90 msgr2=0x7f1be01441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:53.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 shutdown_connections 2024-05-17T22:00:53.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.604+0000 7f1be4d83640 1 -- 172.21.15.83:0/2762127813 wait complete. 2024-05-17T22:00:53.615 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T22:00:53.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:00:53.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 Processor -- start 2024-05-17T22:00:53.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 -- start start 2024-05-17T22:00:53.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 0x7fb7d80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d815d700 0x7fb7d81663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7d80b5640 con 0x7fb7d80cdf40 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb7d8070870 con 0x7fb7d815d700 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7df2b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d80b58e0 con 0x7fb7d80cf830 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd852640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d815d700 0x7fb7d81663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dc850640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7d80cdf40 0x7fb7d815cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55632/0 (socket says 172.21.15.83:55632) 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dc850640 1 -- 172.21.15.83:0/4215434214 learned_addr learned my addr 172.21.15.83:0/4215434214 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd051640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 0x7fb7d80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd852640 1 -- 172.21.15.83:0/4215434214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 msgr2=0x7fb7d80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd852640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 0x7fb7d80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7cffff640 1 -- 172.21.15.83:0/4215434214 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594606309 0 0) 0x7fb7d80b5640 con 0x7fb7d80cdf40 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd852640 1 -- 172.21.15.83:0/4215434214 >> v1:172.21.15.83:6789/0 conn(0x7fb7d80cdf40 legacy=0x7fb7d815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd852640 1 -- 172.21.15.83:0/4215434214 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d806f060 con 0x7fb7d815d700 2024-05-17T22:00:53.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.768+0000 7fb7dd051640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 0x7fb7d80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:53.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7dd852640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d815d700 0x7fb7d81663d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009d90 tx=0x7fb7d001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805631703438e98f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7cffff640 1 -- 172.21.15.83:0/4215434214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00b2070 con 0x7fb7d815d700 2024-05-17T22:00:53.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7cffff640 1 -- 172.21.15.83:0/4215434214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d00a58a0 con 0x7fb7d815d700 2024-05-17T22:00:53.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7cffff640 1 -- 172.21.15.83:0/4215434214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00a5c70 con 0x7fb7d815d700 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- 172.21.15.83:0/4215434214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d815d700 msgr2=0x7fb7d81663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d815d700 0x7fb7d81663d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009d90 tx=0x7fb7d001c990 comp rx=0 tx=0).stop 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- 172.21.15.83:0/4215434214 shutdown_connections 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d815d700 0x7fb7d81663d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/4215434214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d80cf830 0x7fb7d80b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- 172.21.15.83:0/4215434214 >> 172.21.15.83:0/4215434214 conn(0x7fb7d806cd90 msgr2=0x7fb7d80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- 172.21.15.83:0/4215434214 shutdown_connections 2024-05-17T22:00:53.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- 172.21.15.83:0/4215434214 wait complete. 2024-05-17T22:00:53.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 Processor -- start 2024-05-17T22:00:53.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.772+0000 7fb7df2b5640 1 -- start start 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d80cf830 0x7fb7d8158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d815d700 0x7fb7d8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81692b0 con 0x7fb7d80cdf40 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8169130 con 0x7fb7d80cf830 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d8169430 con 0x7fb7d815d700 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43746/0 (socket says 172.21.15.83:43746) 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 -- 172.21.15.83:0/894628954 learned_addr learned my addr 172.21.15.83:0/894628954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 -- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d815d700 msgr2=0x7fb7d8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d815d700 0x7fb7d8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 -- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d80cf830 msgr2=0x7fb7d8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d80cf830 0x7fb7d8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 -- 172.21.15.83:0/894628954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d80ab480 con 0x7fb7d80cdf40 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7dd051640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400c820 tx=0x7fb7d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4015070 con 0x7fb7d80cdf40 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d4007a30 con 0x7fb7d80cdf40 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d80ab760 con 0x7fb7d80cdf40 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4007e40 con 0x7fb7d80cdf40 2024-05-17T22:00:53.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d80abd20 con 0x7fb7d80cdf40 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d80b3aa0 con 0x7fb7d80cdf40 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7d4018070 con 0x7fb7d80cdf40 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.776+0000 7fb7cdffb640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 0x7fb7bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.780+0000 7fb7dc850640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 0x7fb7bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.780+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fb7d408eed0 con 0x7fb7d80cdf40 2024-05-17T22:00:53.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.780+0000 7fb7dc850640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 0x7fb7bc071910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80aa7f0 tx=0x7fb7c8007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:53.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.784+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb7d405c150 con 0x7fb7d80cdf40 2024-05-17T22:00:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.928+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7d81593f0 con 0x7fb7d80cdf40 2024-05-17T22:00:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7cdffb640 1 -- 172.21.15.83:0/894628954 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fb7d405ba50 con 0x7fb7d80cdf40 2024-05-17T22:00:53.923 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T22:00:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 msgr2=0x7fb7bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 0x7fb7bc071910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80aa7f0 tx=0x7fb7c8007970 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 msgr2=0x7fb7d8141f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400c820 tx=0x7fb7d400cce0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 shutdown_connections 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7d815d700 0x7fb7d8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb7bc06f400 0x7fb7bc071910 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7d80cf830 0x7fb7d8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 --2- 172.21.15.83:0/894628954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d80cdf40 0x7fb7d8141f00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 >> 172.21.15.83:0/894628954 conn(0x7fb7d806cd90 msgr2=0x7fb7d8159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 shutdown_connections 2024-05-17T22:00:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:53.932+0000 7fb7df2b5640 1 -- 172.21.15.83:0/894628954 wait complete. 2024-05-17T22:00:53.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-17T22:00:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.084+0000 7fddce7ce640 1 Processor -- start 2024-05-17T22:00:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.084+0000 7fddce7ce640 1 -- start start 2024-05-17T22:00:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddce7ce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc806d1a0 0x7fddc80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddce7ce640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fddc80b2df0 0x7fddc80cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddce7ce640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc806f7e0 con 0x7fddc80b3520 2024-05-17T22:00:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddce7ce640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fddc80ce940 con 0x7fddc80b2df0 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddce7ce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fddc80cec20 con 0x7fddc806d1a0 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fddc80b2df0 0x7fddc80cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fddc80b2df0 0x7fddc80cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43748/0 (socket says 172.21.15.83:43748) 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 -- 172.21.15.83:0/2490495779 learned_addr learned my addr 172.21.15.83:0/2490495779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddccd6b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fddc80b3520 0x7fddc80b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55642/0 (socket says 172.21.15.83:55642) 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc7fff640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc806d1a0 0x7fddc80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 -- 172.21.15.83:0/2490495779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc806d1a0 msgr2=0x7fddc80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc806d1a0 0x7fddc80ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 -- 172.21.15.83:0/2490495779 >> v1:172.21.15.83:6789/0 conn(0x7fddc80b3520 legacy=0x7fddc80b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 -- 172.21.15.83:0/2490495779 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc80cef00 con 0x7fddc80b2df0 2024-05-17T22:00:54.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.088+0000 7fddc77fe640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc80b2df0 0x7fddc80cf9e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009870 tx=0x7fddbc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c13350171ad7562 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:54.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddc6ffd640 1 -- 172.21.15.83:0/2490495779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0b0070 con 0x7fddc80b2df0 2024-05-17T22:00:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddc6ffd640 1 -- 172.21.15.83:0/2490495779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc0ab600 con 0x7fddc80b2df0 2024-05-17T22:00:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddc6ffd640 1 -- 172.21.15.83:0/2490495779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0ab9d0 con 0x7fddc80b2df0 2024-05-17T22:00:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- 172.21.15.83:0/2490495779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc80b2df0 msgr2=0x7fddc80cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc80b2df0 0x7fddc80cf9e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009870 tx=0x7fddbc01c560 comp rx=0 tx=0).stop 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- 172.21.15.83:0/2490495779 shutdown_connections 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc80b2df0 0x7fddc80cf9e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- 172.21.15.83:0/2490495779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc806d1a0 0x7fddc80ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- 172.21.15.83:0/2490495779 >> 172.21.15.83:0/2490495779 conn(0x7fddc80b4170 msgr2=0x7fddc80b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- 172.21.15.83:0/2490495779 shutdown_connections 2024-05-17T22:00:54.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- 172.21.15.83:0/2490495779 wait complete. 2024-05-17T22:00:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 Processor -- start 2024-05-17T22:00:54.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- start start 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddc80b2df0 0x7fddc80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc80b3520 0x7fddc81525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8160060 con 0x7fddc806d1a0 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fddc815fee0 con 0x7fddc80b2df0 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.092+0000 7fddce7ce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fddc81601e0 con 0x7fddc80b3520 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43756/0 (socket says 172.21.15.83:43756) 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 -- 172.21.15.83:0/3208864522 learned_addr learned my addr 172.21.15.83:0/3208864522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 -- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc80b3520 msgr2=0x7fddc81525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc80b3520 0x7fddc81525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:54.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc77fe640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddc80b2df0 0x7fddc80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 -- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddc80b2df0 msgr2=0x7fddc80afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddc80b2df0 0x7fddc80afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 -- 172.21.15.83:0/3208864522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc8155340 con 0x7fddc806d1a0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc7fff640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fddb800d620 tx=0x7fddb800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800b620 con 0x7fddc806d1a0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8019070 con 0x7fddc806d1a0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc8155620 con 0x7fddc806d1a0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800b9e0 con 0x7fddc806d1a0 2024-05-17T22:00:54.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc8152ae0 con 0x7fddc806d1a0 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc80ce310 con 0x7fddc806d1a0 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fddb800bb80 con 0x7fddc806d1a0 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc77fe640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7fddb808f420 con 0x7fddc806d1a0 2024-05-17T22:00:54.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.096+0000 7fddc77fe640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 0x7fddb4071910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0ad000 tx=0x7fddbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:54.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.100+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fddb805c610 con 0x7fddc806d1a0 2024-05-17T22:00:54.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:54.312+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fddc80b4830 con 0x7fddc806d1a0 2024-05-17T22:00:55.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.252+0000 7fddc4ff9640 1 -- 172.21.15.83:0/3208864522 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v182) ==== 169+0+0 (secure 0 0 0) 0x7fddb805bf10 con 0x7fddc806d1a0 2024-05-17T22:00:55.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 msgr2=0x7fddb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 0x7fddb4071910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0ad000 tx=0x7fddbc008040 comp rx=0 tx=0).stop 2024-05-17T22:00:55.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 msgr2=0x7fddc8154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fddb800d620 tx=0x7fddb800dae0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 shutdown_connections 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddc80b3520 0x7fddc81525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fddb406f400 0x7fddb4071910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddc80b2df0 0x7fddc80afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 --2- 172.21.15.83:0/3208864522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc806d1a0 0x7fddc8154af0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 >> 172.21.15.83:0/3208864522 conn(0x7fddc80b4170 msgr2=0x7fddc8071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 shutdown_connections 2024-05-17T22:00:55.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.256+0000 7fddce7ce640 1 -- 172.21.15.83:0/3208864522 wait complete. 2024-05-17T22:00:55.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T22:00:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 Processor -- start 2024-05-17T22:00:55.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 -- start start 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 0x7f0ebc0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ebc143f20 0x7f0ebc1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc0d10d0 con 0x7f0ebc0b3d00 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ebc0b4110 con 0x7f0ebc143f20 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec2b9a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc070770 con 0x7f0ebc073590 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec1137640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ebc143f20 0x7f0ebc1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 0x7f0ebc0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec0936640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0ebc0b3d00 0x7f0ebc072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:46974/0 (socket says 172.21.15.83:46974) 2024-05-17T22:00:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ec0936640 1 -- 172.21.15.83:0/275906201 learned_addr learned my addr 172.21.15.83:0/275906201 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ebbfff640 1 -- 172.21.15.83:0/275906201 >> v2:172.21.15.155:3300/0 conn(0x7f0ebc143f20 msgr2=0x7f0ebc1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ebbfff640 1 --2- 172.21.15.83:0/275906201 >> v2:172.21.15.155:3300/0 conn(0x7f0ebc143f20 0x7f0ebc1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ebbfff640 1 -- 172.21.15.83:0/275906201 >> v1:172.21.15.155:6789/0 conn(0x7f0ebc0b3d00 legacy=0x7f0ebc072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.440+0000 7f0ebbfff640 1 -- 172.21.15.83:0/275906201 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc070120 con 0x7f0ebc073590 2024-05-17T22:00:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ebbfff640 1 --2- 172.21.15.83:0/275906201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 0x7f0ebc0d0a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00097c0 tx=0x7f0eb001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210ccbba19f3b6f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ebb7fe640 1 -- 172.21.15.83:0/275906201 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00b0070 con 0x7f0ebc073590 2024-05-17T22:00:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ebb7fe640 1 -- 172.21.15.83:0/275906201 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb00ab600 con 0x7f0ebc073590 2024-05-17T22:00:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ebb7fe640 1 -- 172.21.15.83:0/275906201 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00ab9d0 con 0x7f0ebc073590 2024-05-17T22:00:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/275906201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 msgr2=0x7f0ebc0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/275906201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 0x7f0ebc0d0a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00097c0 tx=0x7f0eb001c4a0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/275906201 shutdown_connections 2024-05-17T22:00:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/275906201 >> v2:172.21.15.155:3300/0 conn(0x7f0ebc143f20 0x7f0ebc1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/275906201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc073590 0x7f0ebc0d0a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/275906201 >> 172.21.15.83:0/275906201 conn(0x7f0ebc06cd90 msgr2=0x7f0ebc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/275906201 shutdown_connections 2024-05-17T22:00:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.444+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/275906201 wait complete. 2024-05-17T22:00:55.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 Processor -- start 2024-05-17T22:00:55.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 -- start start 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ebc0b3d00 0x7f0ebc15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 0x7f0ebc15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc070c00 con 0x7f0ebc073590 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc145540 con 0x7f0ebc0b3d00 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec2b9a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc1386f0 con 0x7f0ebc143f20 2024-05-17T22:00:55.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec0936640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec0936640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43758/0 (socket says 172.21.15.83:43758) 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec0936640 1 -- 172.21.15.83:0/266687393 learned_addr learned my addr 172.21.15.83:0/266687393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ec1137640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 0x7f0ebc15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.448+0000 7f0ebbfff640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ebc0b3d00 0x7f0ebc15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 -- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 msgr2=0x7f0ebc15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 0x7f0ebc15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 -- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ebc0b3d00 msgr2=0x7f0ebc15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ebc0b3d00 0x7f0ebc15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 -- 172.21.15.83:0/266687393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc15b8f0 con 0x7f0ebc073590 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec1137640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 0x7f0ebc15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec0936640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c460 tx=0x7f0eac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac017070 con 0x7f0ebc073590 2024-05-17T22:00:55.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac005d10 con 0x7f0ebc073590 2024-05-17T22:00:55.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0072b0 con 0x7f0ebc073590 2024-05-17T22:00:55.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc13ade0 con 0x7f0ebc073590 2024-05-17T22:00:55.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc13b3a0 con 0x7f0ebc073590 2024-05-17T22:00:55.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc1447f0 con 0x7f0ebc073590 2024-05-17T22:00:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0eac01a070 con 0x7f0ebc073590 2024-05-17T22:00:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 0x7f0ea8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f0eac08e300 con 0x7f0ebc073590 2024-05-17T22:00:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ebbfff640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 0x7f0ea8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.452+0000 7f0ebbfff640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 0x7f0ea8071910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00ad000 tx=0x7f0eb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:55.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.456+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0ebc15b8f0 con 0x7f0ebc073590 2024-05-17T22:00:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.616+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0ebc1406b0 con 0x7f0ebc073590 2024-05-17T22:00:55.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0eb97fa640 1 -- 172.21.15.83:0/266687393 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f0eac05b4f0 con 0x7f0ebc073590 2024-05-17T22:00:55.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 msgr2=0x7f0ea8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 0x7f0ea8071910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00ad000 tx=0x7f0eb0008040 comp rx=0 tx=0).stop 2024-05-17T22:00:55.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 msgr2=0x7f0ebc137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c460 tx=0x7f0eac00c920 comp rx=0 tx=0).stop 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 shutdown_connections 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ebc143f20 0x7f0ebc15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ea806f400 0x7f0ea8071910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ebc0b3d00 0x7f0ebc15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 --2- 172.21.15.83:0/266687393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ebc073590 0x7f0ebc137f60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 >> 172.21.15.83:0/266687393 conn(0x7f0ebc06cd90 msgr2=0x7f0ebc140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 shutdown_connections 2024-05-17T22:00:55.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.624+0000 7f0ec2b9a640 1 -- 172.21.15.83:0/266687393 wait complete. 2024-05-17T22:00:55.637 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:00:55.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:00:55.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:00:55.734 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:00:55.734 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:00:55.734 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:55.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:00:55.740 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:00:55.740 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:55.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:00:55.745 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:00:55.745 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:55.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:00:55.791 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:00:55.791 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:00:55.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:00:55.796 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T22:00:55.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:00:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 Processor -- start 2024-05-17T22:00:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 -- start start 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ae815c9c0 0x7f0ae80b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae815cdd0 con 0x7f0ae80b68b0 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae80b5630 con 0x7f0ae80b0180 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0aecd82640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae80b5910 con 0x7f0ae815c9c0 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ae80b68b0 0x7f0ae80b6cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55650/0 (socket says 172.21.15.83:55650) 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae5d9b640 1 -- 172.21.15.83:0/985217427 learned_addr learned my addr 172.21.15.83:0/985217427 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae659c640 1 --2- 172.21.15.83:0/985217427 >> v2:172.21.15.155:3300/0 conn(0x7f0ae815c9c0 0x7f0ae80b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561031368 0 0) 0x7f0ae815cdd0 con 0x7f0ae80b68b0 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad0003530 con 0x7f0ae80b68b0 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1932954498 0 0) 0x7f0ad0003530 con 0x7f0ae80b68b0 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 >> v2:172.21.15.155:3300/0 conn(0x7f0ae815c9c0 msgr2=0x7f0ae80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 --2- 172.21.15.83:0/985217427 >> v2:172.21.15.155:3300/0 conn(0x7f0ae815c9c0 0x7f0ae80b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 >> v1:172.21.15.155:6789/0 conn(0x7f0ae80b0180 legacy=0x7f0ae80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae80b5bf0 con 0x7f0ae80b68b0 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f0adc001700 con 0x7f0ae80b68b0 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 >> v1:172.21.15.83:6789/0 conn(0x7f0ae80b68b0 legacy=0x7f0ae80b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ad0003a60 0x7f0ad0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad0005360 0x7f0ad000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b5910 con 0x7f0ad0004810 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad000c290 con 0x7f0ad0005360 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad000c490 con 0x7f0ad0003a60 2024-05-17T22:00:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:55.996+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae6d9d640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad0005360 0x7f0ad000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae659c640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ad0003a60 0x7f0ad0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 -- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ad0003a60 msgr2=0x7f0ad0003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ad0003a60 0x7f0ad0003ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 -- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad0005360 msgr2=0x7f0ad000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ad0005360 0x7f0ad000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae6d9d640 1 -- 172.21.15.83:0/985217427 reap_dead start 2024-05-17T22:00:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 -- 172.21.15.83:0/985217427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad0003530 con 0x7f0ad0004810 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001060 tx=0x7f0adc01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1da0785fae2825b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc007d20 con 0x7f0ad0004810 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc0a9020 con 0x7f0ad0004810 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0ae559a640 1 -- 172.21.15.83:0/985217427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc005b20 con 0x7f0ad0004810 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 -- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 msgr2=0x7f0ad0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001060 tx=0x7f0adc01cba0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 -- 172.21.15.83:0/985217427 shutdown_connections 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 --2- 172.21.15.83:0/985217427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 -- 172.21.15.83:0/985217427 >> 172.21.15.83:0/985217427 conn(0x7f0ae809b8b0 msgr2=0x7f0ae80b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 -- 172.21.15.83:0/985217427 shutdown_connections 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.000+0000 7f0aecd82640 1 -- 172.21.15.83:0/985217427 wait complete. 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 Processor -- start 2024-05-17T22:00:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- start start 2024-05-17T22:00:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae8159680 0x7f0ae81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 0x7f0ae81609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae80b0590 con 0x7f0ad0004810 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae809bff0 con 0x7f0ae8159a90 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae80b0710 con 0x7f0ae8159680 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43764/0 (socket says 172.21.15.83:43764) 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 -- 172.21.15.83:0/201177691 learned_addr learned my addr 172.21.15.83:0/201177691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae6d9d640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 0x7f0ae81609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae8159680 0x7f0ae81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 -- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae8159680 msgr2=0x7f0ae81604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae8159680 0x7f0ae81604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 -- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 msgr2=0x7f0ae81609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 0x7f0ae81609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 -- 172.21.15.83:0/201177691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae8156de0 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae6d9d640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 0x7f0ae81609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ae659c640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8010900 tx=0x7f0ad8010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800f040 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad800ec20 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae80b1bf0 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad801c400 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.004+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae80b21b0 con 0x7f0ad0004810 2024-05-17T22:00:56.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae80b0aa0 con 0x7f0ad0004810 2024-05-17T22:00:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ad801c5a0 con 0x7f0ad0004810 2024-05-17T22:00:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0ad6ffd640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 0x7f0ac8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f0ad808f5e0 con 0x7f0ad0004810 2024-05-17T22:00:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 0x7f0ac8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.008+0000 7f0ae5d9b640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 0x7f0ac8071910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa000 tx=0x7f0adc000ff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.012+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0ae8156de0 con 0x7f0ad0004810 2024-05-17T22:00:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae80b44b0 con 0x7f0ad0004810 2024-05-17T22:00:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0ad6ffd640 1 -- 172.21.15.83:0/201177691 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+3872 (secure 0 0 0) 0x7f0ad8018030 con 0x7f0ad0004810 2024-05-17T22:00:56.187 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:56.188 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/798710379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":798710379},{"type":"v1","addr":"172.21.15.155:6833","nonce":798710379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3662325693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3662325693},{"type":"v1","addr":"172.21.15.155:6835","nonce":3662325693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2373156250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2373156250},{"type":"v1","addr":"172.21.15.83:6835","nonce":2373156250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1284798567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1284798567},{"type":"v1","addr":"172.21.15.83:6837","nonce":1284798567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-05-17T22:00:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-05-17T22:00:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 msgr2=0x7f0ac8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 0x7f0ac8071910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa000 tx=0x7f0adc000ff0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 msgr2=0x7f0ae81568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8010900 tx=0x7f0ad8010dc0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 shutdown_connections 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0ac806f400 0x7f0ac8071910 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ae8159a90 0x7f0ae81609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ae8159680 0x7f0ae81604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 --2- 172.21.15.83:0/201177691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ad0004810 0x7f0ae81568a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 >> 172.21.15.83:0/201177691 conn(0x7f0ae809b8b0 msgr2=0x7f0ae8070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 shutdown_connections 2024-05-17T22:00:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.196+0000 7f0aecd82640 1 -- 172.21.15.83:0/201177691 wait complete. 2024-05-17T22:00:56.208 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T22:00:56.208 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T22:00:56.208 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T22:00:56.208 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:00:56.208 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T22:00:56.209 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T22:00:56.209 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T22:00:56.209 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:00:56.209 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T22:00:56.209 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T22:00:56.209 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T22:00:56.209 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:00:56.210 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T22:00:56.210 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T22:00:56.210 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T22:00:56.210 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:00:56.222 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T22:00:56.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T22:00:56.223 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T22:00:56.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T22:00:56.225 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T22:00:56.225 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T22:00:56.228 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T22:00:56.228 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T22:00:56.264 INFO:tasks.ceph.mds.c:Started 2024-05-17T22:00:56.264 DEBUG:teuthology.parallel:result is None 2024-05-17T22:00:56.264 INFO:tasks.ceph.mds.a:Started 2024-05-17T22:00:56.265 DEBUG:teuthology.parallel:result is None 2024-05-17T22:00:56.269 INFO:tasks.ceph.mds.b:Started 2024-05-17T22:00:56.269 DEBUG:teuthology.parallel:result is None 2024-05-17T22:00:56.270 INFO:tasks.ceph.mds.d:Started 2024-05-17T22:00:56.270 DEBUG:teuthology.parallel:result is None 2024-05-17T22:00:56.271 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T22:00:56.350 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:00:56.351 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:00:56.368 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T22:00:56.369 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T22:00:56.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.416+0000 7f2a45ffd640 1 Processor -- start 2024-05-17T22:00:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 -- start start 2024-05-17T22:00:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2a400d5210 0x7f2a400d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a400d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400d4170 con 0x7f2a400b4810 2024-05-17T22:00:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2a400d5620 con 0x7f2a400d5210 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a45ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400d4450 con 0x7f2a400d2000 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2a400b4810 0x7f2a4015fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55676/0 (socket says 172.21.15.83:55676) 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3f7fe640 1 -- 172.21.15.83:0/608897019 learned_addr learned my addr 172.21.15.83:0/608897019 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 --2- 172.21.15.83:0/608897019 >> v2:172.21.15.155:3300/0 conn(0x7f2a400d5210 0x7f2a400d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2150797160 0 0) 0x7f2a400d4170 con 0x7f2a400b4810 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3ffff640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a400d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a20003530 con 0x7f2a400b4810 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1859653095 0 0) 0x7f2a20003530 con 0x7f2a400b4810 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 msgr2=0x7f2a400d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a400d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 >> v2:172.21.15.155:3300/0 conn(0x7f2a400d5210 msgr2=0x7f2a400d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 --2- 172.21.15.83:0/608897019 >> v2:172.21.15.155:3300/0 conn(0x7f2a400d5210 0x7f2a400d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400d4730 con 0x7f2a400b4810 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2a38001a80 con 0x7f2a400b4810 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 >> v1:172.21.15.83:6789/0 conn(0x7f2a400b4810 legacy=0x7f2a4015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a20003a60 0x7f2a20003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a20004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a20005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d5620 con 0x7f2a20004810 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d4450 con 0x7f2a20003a60 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2000bab0 con 0x7f2a400d2000 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a20004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3ffff640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a20005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3f7fe640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a20003a60 0x7f2a20003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 -- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 msgr2=0x7f2a20005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a400d2000 0x7f2a20005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 -- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a20003a60 msgr2=0x7f2a20003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a20003a60 0x7f2a20003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.420+0000 7f2a3effd640 1 -- 172.21.15.83:0/608897019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a20003530 con 0x7f2a20004810 2024-05-17T22:00:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a3effd640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a20004c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009ae0 tx=0x7f2a2801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2422e8d54f3cf4ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280b2070 con 0x7f2a20004810 2024-05-17T22:00:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a280a5590 con 0x7f2a20004810 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a3e7fc640 1 -- 172.21.15.83:0/608897019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280096b0 con 0x7f2a20004810 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- 172.21.15.83:0/608897019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 msgr2=0x7f2a20004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/608897019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a20004c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009ae0 tx=0x7f2a2801cab0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a3ffff640 1 -- 172.21.15.83:0/608897019 reap_dead start 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- 172.21.15.83:0/608897019 shutdown_connections 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- 172.21.15.83:0/608897019 >> 172.21.15.83:0/608897019 conn(0x7f2a4006e010 msgr2=0x7f2a4006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- 172.21.15.83:0/608897019 shutdown_connections 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- 172.21.15.83:0/608897019 wait complete. 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 Processor -- start 2024-05-17T22:00:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.424+0000 7f2a45ffd640 1 -- start start 2024-05-17T22:00:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a401ea2a0 0x7f2a401ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a401e8740 0x7f2a401e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d2410 con 0x7f2a20004810 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d4cf0 con 0x7f2a401ea2a0 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400d2590 con 0x7f2a401e8740 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43830/0 (socket says 172.21.15.83:43830) 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 -- 172.21.15.83:0/1388596768 learned_addr learned my addr 172.21.15.83:0/1388596768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3ffff640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a401e8740 0x7f2a401e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3effd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a401ea2a0 0x7f2a401ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 -- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a401e8740 msgr2=0x7f2a401e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a401e8740 0x7f2a401e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 -- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a401ea2a0 msgr2=0x7f2a401ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a401ea2a0 0x7f2a401ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 -- 172.21.15.83:0/1388596768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4015e170 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a3f7fe640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011820 tx=0x7f2a38011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38010040 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3801a070 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a40158750 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800fd60 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a40158d10 con 0x7f2a20004810 2024-05-17T22:00:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.428+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a400d29c0 con 0x7f2a20004810 2024-05-17T22:00:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2a3801f460 con 0x7f2a20004810 2024-05-17T22:00:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a37fff640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 0x7f2a24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4943+0+0 (secure 0 0 0) 0x7f2a38092a60 con 0x7f2a20004810 2024-05-17T22:00:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a3effd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 0x7f2a24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a3effd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 0x7f2a24071910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a280af000 tx=0x7f2a280046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.432+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2a3805fdc0 con 0x7f2a20004810 2024-05-17T22:00:56.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:56.596+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2a4006e6d0 con 0x7f2a20004810 2024-05-17T22:00:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.276+0000 7f2a37fff640 1 -- 172.21.15.83:0/1388596768 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) ==== 97+0+0 (secure 0 0 0) 0x7f2a3805f6c0 con 0x7f2a20004810 2024-05-17T22:00:57.269 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T22:00:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 msgr2=0x7f2a24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 0x7f2a24071910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a280af000 tx=0x7f2a280046d0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 msgr2=0x7f2a401eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011820 tx=0x7f2a38011c20 comp rx=0 tx=0).stop 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 shutdown_connections 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2a401e8740 0x7f2a401e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2a2406f400 0x7f2a24071910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a401ea2a0 0x7f2a401ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 --2- 172.21.15.83:0/1388596768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a20004810 0x7f2a401eb860 secure :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011820 tx=0x7f2a38011c20 comp rx=0 tx=0).stop 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 >> 172.21.15.83:0/1388596768 conn(0x7f2a4006e010 msgr2=0x7f2a400d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 shutdown_connections 2024-05-17T22:00:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.280+0000 7f2a45ffd640 1 -- 172.21.15.83:0/1388596768 wait complete. 2024-05-17T22:00:57.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T22:00:57.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 Processor -- start 2024-05-17T22:00:57.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 -- start start 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6dc06ffd0 0x7fb6dc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb6dc1605a0 0x7fb6dc1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc1465d0 con 0x7fb6dc1461c0 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc160ef0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6e08b0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc06ef30 con 0x7fb6dc1605a0 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d9d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb6dc06ffd0 0x7fb6dc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6da59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb6dc1461c0 0x7fb6dc15e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40272/0 (socket says 172.21.15.83:40272) 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6da59c640 1 -- 172.21.15.83:0/4066692059 learned_addr learned my addr 172.21.15.83:0/4066692059 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d9d9b640 1 -- 172.21.15.83:0/4066692059 >> v2:172.21.15.155:3300/0 conn(0x7fb6dc1605a0 msgr2=0x7fb6dc1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6dad9d640 1 --2- 172.21.15.83:0/4066692059 >> v2:172.21.15.155:3300/0 conn(0x7fb6dc1605a0 0x7fb6dc1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d9d9b640 1 --2- 172.21.15.83:0/4066692059 >> v2:172.21.15.155:3300/0 conn(0x7fb6dc1605a0 0x7fb6dc1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d9d9b640 1 -- 172.21.15.83:0/4066692059 >> v1:172.21.15.83:6789/0 conn(0x7fb6dc1461c0 legacy=0x7fb6dc15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d959a640 1 -- 172.21.15.83:0/4066692059 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154948511 0 0) 0x7fb6dc1465d0 con 0x7fb6dc1461c0 2024-05-17T22:00:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.456+0000 7fb6d9d9b640 1 -- 172.21.15.83:0/4066692059 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc070920 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6d9d9b640 1 --2- 172.21.15.83:0/4066692059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc0703e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc072c00 tx=0x7fb6d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3ee19aa19fe775 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6d959a640 1 -- 172.21.15.83:0/4066692059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00ab650 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6d959a640 1 -- 172.21.15.83:0/4066692059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d00b0070 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6d959a640 1 -- 172.21.15.83:0/4066692059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00aba10 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 -- 172.21.15.83:0/4066692059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 msgr2=0x7fb6dc0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/4066692059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc0703e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc072c00 tx=0x7fb6d001c450 comp rx=0 tx=0).stop 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 -- 172.21.15.83:0/4066692059 shutdown_connections 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/4066692059 >> v2:172.21.15.155:3300/0 conn(0x7fb6dc1605a0 0x7fb6dc1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/4066692059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc0703e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 -- 172.21.15.83:0/4066692059 >> 172.21.15.83:0/4066692059 conn(0x7fb6dc06cd90 msgr2=0x7fb6dc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 -- 172.21.15.83:0/4066692059 shutdown_connections 2024-05-17T22:00:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.460+0000 7fb6e08b0640 1 -- 172.21.15.83:0/4066692059 wait complete. 2024-05-17T22:00:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 Processor -- start 2024-05-17T22:00:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- start start 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6dc1461c0 0x7fb6dc12d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6dc1605a0 0x7fb6dc12dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc15e880 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc06fab0 con 0x7fb6dc1461c0 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d10a0 con 0x7fb6dc1605a0 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49912/0 (socket says 172.21.15.83:49912) 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 -- 172.21.15.83:0/1418762116 learned_addr learned my addr 172.21.15.83:0/1418762116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6d9d9b640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6dc1461c0 0x7fb6dc12d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6dad9d640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6dc1605a0 0x7fb6dc12dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 -- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6dc1605a0 msgr2=0x7fb6dc12dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6dc1605a0 0x7fb6dc12dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 -- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6dc1461c0 msgr2=0x7fb6dc12d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6dc1461c0 0x7fb6dc12d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 -- 172.21.15.83:0/1418762116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0b1460 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6da59c640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c400 tx=0x7fb6d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40074e0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d40047f0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4007680 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc166fb0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc167570 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.464+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc0703e0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.468+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb6d4011070 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.468+0000 7fb6caffd640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 0x7fb6bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.468+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7fb6d408ea50 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.468+0000 7fb6d9d9b640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 0x7fb6bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.468+0000 7fb6d9d9b640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 0x7fb6bc071940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00af000 tx=0x7fb6d00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.472+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb6dc0b1460 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.684+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb6dc0b41a0 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.684+0000 7fb6caffd640 1 -- 172.21.15.83:0/1418762116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fb6d405bd20 con 0x7fb6dc06ffd0 2024-05-17T22:00:57.678 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:57.678 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:00:57.678 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T22:00:57.678 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T22:00:57.679 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:00:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.682 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.684 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.685 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.687 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.689 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T22:00:57.691 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:00:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 msgr2=0x7fb6bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 0x7fb6bc071940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00af000 tx=0x7fb6d00a7000 comp rx=0 tx=0).stop 2024-05-17T22:00:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 msgr2=0x7fb6dc166510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c400 tx=0x7fb6d400c8c0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 shutdown_connections 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6dc1605a0 0x7fb6dc12dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb6bc06f430 0x7fb6bc071940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6dc1461c0 0x7fb6dc12d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 --2- 172.21.15.83:0/1418762116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6dc06ffd0 0x7fb6dc166510 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 >> 172.21.15.83:0/1418762116 conn(0x7fb6dc06cd90 msgr2=0x7fb6dc0b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 shutdown_connections 2024-05-17T22:00:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.688+0000 7fb6e08b0640 1 -- 172.21.15.83:0/1418762116 wait complete. 2024-05-17T22:00:57.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T22:00:57.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.852+0000 7fe73ed5e640 1 Processor -- start 2024-05-17T22:00:57.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.852+0000 7fe73ed5e640 1 -- start start 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7380d1040 0x7fe738144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 0x7fe738072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e490 0x7fe738073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe73815ea40 con 0x7fe7380d1040 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe73815ed20 con 0x7fe73815e080 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73ed5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe73815f000 con 0x7fe73815e490 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7380d1040 0x7fe738144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7380d1040 0x7fe738144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49926/0 (socket says 172.21.15.83:49926) 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 -- 172.21.15.83:0/3466329863 learned_addr learned my addr 172.21.15.83:0/3466329863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe737fff640 1 --2- 172.21.15.83:0/3466329863 >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 0x7fe738072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73d2fb640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e490 0x7fe738073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 -- 172.21.15.83:0/3466329863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e490 msgr2=0x7fe738073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:57.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e490 0x7fe738073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 -- 172.21.15.83:0/3466329863 >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 msgr2=0x7fe738072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:57.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3466329863 >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 0x7fe738072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe73cafa640 1 -- 172.21.15.83:0/3466329863 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73806ef30 con 0x7fe7380d1040 2024-05-17T22:00:57.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.856+0000 7fe737fff640 1 --2- 172.21.15.83:0/3466329863 >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 0x7fe738072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:57.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738144080 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe73815ef30 tx=0x7fe73001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0f2cf20a642998 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe7377fe640 1 -- 172.21.15.83:0/3466329863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300a5780 con 0x7fe7380d1040 2024-05-17T22:00:57.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe7377fe640 1 -- 172.21.15.83:0/3466329863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7300a5920 con 0x7fe7380d1040 2024-05-17T22:00:57.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe7377fe640 1 -- 172.21.15.83:0/3466329863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300a5cf0 con 0x7fe7380d1040 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3466329863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 msgr2=0x7fe738144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738144080 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe73815ef30 tx=0x7fe73001c9f0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3466329863 shutdown_connections 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e490 0x7fe738073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3466329863 >> v2:172.21.15.155:3300/0 conn(0x7fe73815e080 0x7fe738072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3466329863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738144080 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3466329863 >> 172.21.15.83:0/3466329863 conn(0x7fe73806cd90 msgr2=0x7fe7380b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3466329863 shutdown_connections 2024-05-17T22:00:57.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3466329863 wait complete. 2024-05-17T22:00:57.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 Processor -- start 2024-05-17T22:00:57.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.860+0000 7fe73ed5e640 1 -- start start 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 0x7fe73813f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe73815e490 0x7fe73815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe738162260 con 0x7fe7380d1040 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe7381620e0 con 0x7fe73815e490 2024-05-17T22:00:57.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe7381623e0 con 0x7fe73815e080 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49932/0 (socket says 172.21.15.83:49932) 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 -- 172.21.15.83:0/3124953225 learned_addr learned my addr 172.21.15.83:0/3124953225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73d2fb640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe73815e490 0x7fe73815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe737fff640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 0x7fe73813f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 -- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 msgr2=0x7fe73813f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 0x7fe73813f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 -- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe73815e490 msgr2=0x7fe73815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe73815e490 0x7fe73815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 -- 172.21.15.83:0/3124953225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73813f540 con 0x7fe7380d1040 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe737fff640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 0x7fe73813f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73cafa640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7300af040 tx=0x7fe730019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300bc020 con 0x7fe7380d1040 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7300b88a0 con 0x7fe7380d1040 2024-05-17T22:00:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73813fa50 con 0x7fe7380d1040 2024-05-17T22:00:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300b8c90 con 0x7fe7380d1040 2024-05-17T22:00:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73813d0b0 con 0x7fe7380d1040 2024-05-17T22:00:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe738144080 con 0x7fe7380d1040 2024-05-17T22:00:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7300bf020 con 0x7fe7380d1040 2024-05-17T22:00:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.864+0000 7fe7357fa640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.868+0000 7fe737fff640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.868+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7fe73012a790 con 0x7fe7380d1040 2024-05-17T22:00:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.868+0000 7fe737fff640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 0x7fe724071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe738070320 tx=0x7fe72c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:57.872+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe7300c6030 con 0x7fe7380d1040 2024-05-17T22:00:58.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.032+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe738070070 con 0x7fe7380d1040 2024-05-17T22:00:58.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.032+0000 7fe7357fa640 1 -- 172.21.15.83:0/3124953225 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7fe7300165e0 con 0x7fe7380d1040 2024-05-17T22:00:58.026 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:00:58.026 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:00:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:00:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T22:00:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T22:00:58.027 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:00:58.027 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:00:58.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 msgr2=0x7fe724071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 0x7fe724071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe738070320 tx=0x7fe72c008040 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 msgr2=0x7fe738159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7300af040 tx=0x7fe730019790 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 shutdown_connections 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe73815e490 0x7fe73815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe73815e080 0x7fe73813f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 --2- 172.21.15.83:0/3124953225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7380d1040 0x7fe738159e40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 >> 172.21.15.83:0/3124953225 conn(0x7fe73806cd90 msgr2=0x7fe738160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 shutdown_connections 2024-05-17T22:00:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.036+0000 7fe73ed5e640 1 -- 172.21.15.83:0/3124953225 wait complete. 2024-05-17T22:00:58.049 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T22:00:58.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T22:00:58.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.212+0000 7f11211a8640 1 Processor -- start 2024-05-17T22:00:58.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.212+0000 7f11211a8640 1 -- start start 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f11211a8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f111c0d03b0 0x7f111c145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f11211a8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 0x7f111c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f11211a8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111c0d1210 con 0x7f111c0cde20 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f11211a8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f111c0d0970 con 0x7f111c0d03b0 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f11211a8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f111c0d0c50 con 0x7f111c160b00 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f111c0d03b0 0x7f111c145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 0x7f111c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f111c0d03b0 0x7f111c145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54864/0 (socket says 172.21.15.83:54864) 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 -- 172.21.15.83:0/3962612595 learned_addr learned my addr 172.21.15.83:0/3962612595 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111a59c640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f111c0cde20 0x7f111c1603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:38166/0 (socket says 172.21.15.83:38166) 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 -- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 msgr2=0x7f111c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 0x7f111c06f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 -- 172.21.15.83:0/3962612595 >> v1:172.21.15.155:6789/0 conn(0x7f111c0cde20 legacy=0x7f111c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111ad9d640 1 -- 172.21.15.83:0/3962612595 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c0d0f30 con 0x7f111c0d03b0 2024-05-17T22:00:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.216+0000 7f111b59e640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 0x7f111c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:00:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f111ad9d640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c0d03b0 0x7f111c145fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d150 tx=0x7f110c01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f2acae14834755 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f1119d9b640 1 -- 172.21.15.83:0/3962612595 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c00b2d0 con 0x7f111c0d03b0 2024-05-17T22:00:58.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f1119d9b640 1 -- 172.21.15.83:0/3962612595 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c00b470 con 0x7f111c0d03b0 2024-05-17T22:00:58.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f1119d9b640 1 -- 172.21.15.83:0/3962612595 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0a7b20 con 0x7f111c0d03b0 2024-05-17T22:00:58.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c0d03b0 msgr2=0x7f111c145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c0d03b0 0x7f111c145fd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d150 tx=0x7f110c01ec30 comp rx=0 tx=0).stop 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- 172.21.15.83:0/3962612595 shutdown_connections 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c160b00 0x7f111c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 --2- 172.21.15.83:0/3962612595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c0d03b0 0x7f111c145fd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- 172.21.15.83:0/3962612595 >> 172.21.15.83:0/3962612595 conn(0x7f111c06cd90 msgr2=0x7f111c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- 172.21.15.83:0/3962612595 shutdown_connections 2024-05-17T22:00:58.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- 172.21.15.83:0/3962612595 wait complete. 2024-05-17T22:00:58.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 Processor -- start 2024-05-17T22:00:58.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.220+0000 7f11211a8640 1 -- start start 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c0d03b0 0x7f111c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c145010 0x7f111c145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f111c070580 con 0x7f111c0cde20 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f111c070400 con 0x7f111c145010 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f111c070700 con 0x7f111c0d03b0 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49942/0 (socket says 172.21.15.83:49942) 2024-05-17T22:00:58.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 -- 172.21.15.83:0/1195397697 learned_addr learned my addr 172.21.15.83:0/1195397697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111a59c640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c0d03b0 0x7f111c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111b59e640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c145010 0x7f111c145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 -- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c0d03b0 msgr2=0x7f111c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c0d03b0 0x7f111c159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 -- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c145010 msgr2=0x7f111c145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c145010 0x7f111c145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 -- 172.21.15.83:0/1195397697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c13cde0 con 0x7f111c0cde20 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f111ad9d640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f110c01c860 tx=0x7f110c01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c009800 con 0x7f111c0cde20 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c0c0070 con 0x7f111c0cde20 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c13d0c0 con 0x7f111c0cde20 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c009ba0 con 0x7f111c0cde20 2024-05-17T22:00:58.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1403d0 con 0x7f111c0cde20 2024-05-17T22:00:58.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.224+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c145fd0 con 0x7f111c0cde20 2024-05-17T22:00:58.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.228+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f110c00b2e0 con 0x7f111c0cde20 2024-05-17T22:00:58.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.228+0000 7f11137fe640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 0x7f10fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:58.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.228+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 4857+0+0 (secure 0 0 0) 0x7f110c12c0c0 con 0x7f111c0cde20 2024-05-17T22:00:58.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.228+0000 7f111a59c640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 0x7f10fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:58.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.228+0000 7f111a59c640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 0x7f10fc071940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1104000f70 tx=0x7f1104007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:58.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.232+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f110c018820 con 0x7f111c0cde20 2024-05-17T22:00:58.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:58.396+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f111c0b3f40 con 0x7f111c0cde20 2024-05-17T22:00:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.280+0000 7f11137fe640 1 -- 172.21.15.83:0/1195397697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7f110c0f9390 con 0x7f111c0cde20 2024-05-17T22:00:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.284+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 msgr2=0x7f10fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.284+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 0x7f10fc071940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1104000f70 tx=0x7f1104007cb0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.284+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 msgr2=0x7f111c13fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.284+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f110c01c860 tx=0x7f110c01b9d0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 shutdown_connections 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f430 0x7f10fc071940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c145010 0x7f111c145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c0d03b0 0x7f111c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 --2- 172.21.15.83:0/1195397697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c0cde20 0x7f111c13fe00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 >> 172.21.15.83:0/1195397697 conn(0x7f111c06cd90 msgr2=0x7f111c15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 shutdown_connections 2024-05-17T22:00:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.288+0000 7f11211a8640 1 -- 172.21.15.83:0/1195397697 wait complete. 2024-05-17T22:00:59.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T22:00:59.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 Processor -- start 2024-05-17T22:00:59.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 -- start start 2024-05-17T22:00:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca8c0b4cf0 0x7fca8c0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8c0cf9a0 con 0x7fca8c161730 2024-05-17T22:00:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fca8c0d2bb0 con 0x7fca8c0b4cf0 2024-05-17T22:00:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca920da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c0d05b0 con 0x7fca8c161b40 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca8b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca8c0b4cf0 0x7fca8c0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca8bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:34042/0 (socket says 172.21.15.83:34042) 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.476+0000 7fca8bfff640 1 -- 172.21.15.83:0/2373308106 learned_addr learned my addr 172.21.15.83:0/2373308106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8bfff640 1 -- 172.21.15.83:0/2373308106 >> v2:172.21.15.155:3300/0 conn(0x7fca8c0b4cf0 msgr2=0x7fca8c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8bfff640 1 --2- 172.21.15.83:0/2373308106 >> v2:172.21.15.155:3300/0 conn(0x7fca8c0b4cf0 0x7fca8c0d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8bfff640 1 -- 172.21.15.83:0/2373308106 >> v1:172.21.15.155:6789/0 conn(0x7fca8c161730 legacy=0x7fca8c06c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8bfff640 1 -- 172.21.15.83:0/2373308106 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c0cfc80 con 0x7fca8c161b40 2024-05-17T22:00:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8bfff640 1 --2- 172.21.15.83:0/2373308106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009870 tx=0x7fca7c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269b463c1dd3ac3e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8a7fc640 1 -- 172.21.15.83:0/2373308106 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0b0070 con 0x7fca8c161b40 2024-05-17T22:00:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8a7fc640 1 -- 172.21.15.83:0/2373308106 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca7c0ab600 con 0x7fca8c161b40 2024-05-17T22:00:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca8a7fc640 1 -- 172.21.15.83:0/2373308106 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0ab9d0 con 0x7fca8c161b40 2024-05-17T22:00:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 -- 172.21.15.83:0/2373308106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 msgr2=0x7fca8c06cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 --2- 172.21.15.83:0/2373308106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009870 tx=0x7fca7c01c4d0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 -- 172.21.15.83:0/2373308106 shutdown_connections 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 --2- 172.21.15.83:0/2373308106 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c161b40 0x7fca8c06cea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 --2- 172.21.15.83:0/2373308106 >> v2:172.21.15.155:3300/0 conn(0x7fca8c0b4cf0 0x7fca8c0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.480+0000 7fca920da640 1 -- 172.21.15.83:0/2373308106 >> 172.21.15.83:0/2373308106 conn(0x7fca8c0d1ca0 msgr2=0x7fca8c0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- 172.21.15.83:0/2373308106 shutdown_connections 2024-05-17T22:00:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- 172.21.15.83:0/2373308106 wait complete. 2024-05-17T22:00:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 Processor -- start 2024-05-17T22:00:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- start start 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c0b4cf0 0x7fca8c09f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c161b40 0x7fca8c15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c16aa70 con 0x7fca8c161730 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c16a8f0 con 0x7fca8c161b40 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.484+0000 7fca920da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c16abf0 con 0x7fca8c0b4cf0 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49952/0 (socket says 172.21.15.83:49952) 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 -- 172.21.15.83:0/1269170199 learned_addr learned my addr 172.21.15.83:0/1269170199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:00:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8bfff640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c161b40 0x7fca8c15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8b7fe640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c0b4cf0 0x7fca8c09f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 -- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c0b4cf0 msgr2=0x7fca8c09f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c0b4cf0 0x7fca8c09f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 -- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c161b40 msgr2=0x7fca8c15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c161b40 0x7fca8c15e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 -- 172.21.15.83:0/1269170199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c156d00 con 0x7fca8c161730 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8affd640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c3f0 tx=0x7fca7400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74017070 con 0x7fca8c161730 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca740052d0 con 0x7fca8c161730 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c156fe0 con 0x7fca8c161730 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74003820 con 0x7fca8c161730 2024-05-17T22:00:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c1579f0 con 0x7fca8c161730 2024-05-17T22:00:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c156a70 con 0x7fca8c161730 2024-05-17T22:00:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca7401a070 con 0x7fca8c161730 2024-05-17T22:00:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 0x7fca6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:00:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5227+0+0 (secure 0 0 0) 0x7fca7408e370 con 0x7fca8c161730 2024-05-17T22:00:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.488+0000 7fca8b7fe640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 0x7fca6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:00:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.492+0000 7fca8b7fe640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 0x7fca6c071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca84011340 tx=0x7fca8400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:00:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.496+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fca74005480 con 0x7fca8c161730 2024-05-17T22:00:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:00:59.656+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fca8c0d1680 con 0x7fca8c161730 2024-05-17T22:01:00.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:01:00.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:01:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.284+0000 7fca83fff640 1 -- 172.21.15.83:0/1269170199 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7fca7405adf0 con 0x7fca8c161730 2024-05-17T22:01:00.277 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 msgr2=0x7fca6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 0x7fca6c071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca84011340 tx=0x7fca8400b040 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 msgr2=0x7fca8c154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c3f0 tx=0x7fca7400c8b0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 shutdown_connections 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fca6c06f400 0x7fca6c071910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c161b40 0x7fca8c15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8c161730 0x7fca8c154b70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 --2- 172.21.15.83:0/1269170199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c0b4cf0 0x7fca8c09f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 >> 172.21.15.83:0/1269170199 conn(0x7fca8c0d1ca0 msgr2=0x7fca8c0b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 shutdown_connections 2024-05-17T22:01:00.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.288+0000 7fca920da640 1 -- 172.21.15.83:0/1269170199 wait complete. 2024-05-17T22:01:00.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T22:01:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 Processor -- start 2024-05-17T22:01:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 -- start start 2024-05-17T22:01:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc42c0b5f30 0x7fc42c068730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b1260 0x7fc42c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc42c0b1670 con 0x7fc42c15cc90 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc42c0cf4c0 con 0x7fc42c0b5f30 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc4314f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0cf7a0 con 0x7fc42c0b1260 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc42c0b5f30 0x7fc42c068730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc42c15cc90 0x7fc42c0b5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40274/0 (socket says 172.21.15.83:40274) 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42affd640 1 -- 172.21.15.83:0/2274117281 learned_addr learned my addr 172.21.15.83:0/2274117281 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42b7fe640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b1260 0x7fc42c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42a7fc640 1 -- 172.21.15.83:0/2274117281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b1260 msgr2=0x7fc42c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42a7fc640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b1260 0x7fc42c0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42a7fc640 1 -- 172.21.15.83:0/2274117281 >> v1:172.21.15.83:6789/0 conn(0x7fc42c15cc90 legacy=0x7fc42c0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.512+0000 7fc42a7fc640 1 -- 172.21.15.83:0/2274117281 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c0b64f0 con 0x7fc42c0b5f30 2024-05-17T22:01:00.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc42a7fc640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b5f30 0x7fc42c068730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0097c0 tx=0x7fc41c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17efb7b1964b4803 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc429ffb640 1 -- 172.21.15.83:0/2274117281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0b0070 con 0x7fc42c0b5f30 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc429ffb640 1 -- 172.21.15.83:0/2274117281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c0ab600 con 0x7fc42c0b5f30 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc429ffb640 1 -- 172.21.15.83:0/2274117281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0ab9d0 con 0x7fc42c0b5f30 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- 172.21.15.83:0/2274117281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b5f30 msgr2=0x7fc42c068730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b5f30 0x7fc42c068730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0097c0 tx=0x7fc41c01c410 comp rx=0 tx=0).stop 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- 172.21.15.83:0/2274117281 shutdown_connections 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b1260 0x7fc42c0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 --2- 172.21.15.83:0/2274117281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b5f30 0x7fc42c068730 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- 172.21.15.83:0/2274117281 >> 172.21.15.83:0/2274117281 conn(0x7fc42c0ce6e0 msgr2=0x7fc42c0ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- 172.21.15.83:0/2274117281 shutdown_connections 2024-05-17T22:01:00.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- 172.21.15.83:0/2274117281 wait complete. 2024-05-17T22:01:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 Processor -- start 2024-05-17T22:01:00.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.516+0000 7fc4314f7640 1 -- start start 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b5f30 0x7fc42c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c15cc90 0x7fc42c14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0d0a10 con 0x7fc42c0b1260 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0d0890 con 0x7fc42c15cc90 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c167720 con 0x7fc42c0b5f30 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49966/0 (socket says 172.21.15.83:49966) 2024-05-17T22:01:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 -- 172.21.15.83:0/3762464471 learned_addr learned my addr 172.21.15.83:0/3762464471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42a7fc640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b5f30 0x7fc42c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42b7fe640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c15cc90 0x7fc42c14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 -- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b5f30 msgr2=0x7fc42c14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b5f30 0x7fc42c14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 -- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c15cc90 msgr2=0x7fc42c14c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c15cc90 0x7fc42c14c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 -- 172.21.15.83:0/3762464471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c14fe10 con 0x7fc42c0b1260 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc42affd640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c430 tx=0x7fc41400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414017070 con 0x7fc42c0b1260 2024-05-17T22:01:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c14f050 con 0x7fc42c0b1260 2024-05-17T22:01:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414005d10 con 0x7fc42c0b1260 2024-05-17T22:01:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414007310 con 0x7fc42c0b1260 2024-05-17T22:01:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c14f610 con 0x7fc42c0b1260 2024-05-17T22:01:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.520+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c0cfb40 con 0x7fc42c0b1260 2024-05-17T22:01:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.524+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc41401a070 con 0x7fc42c0b1260 2024-05-17T22:01:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.524+0000 7fc4237fe640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 0x7fc40c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.524+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5609+0+0 (secure 0 0 0) 0x7fc41408e8a0 con 0x7fc42c0b1260 2024-05-17T22:01:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.524+0000 7fc42a7fc640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 0x7fc40c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.524+0000 7fc42a7fc640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 0x7fc40c071910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ad000 tx=0x7fc41c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.528+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc414021020 con 0x7fc42c0b1260 2024-05-17T22:01:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:00.720+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc42c15c2c0 con 0x7fc42c0b1260 2024-05-17T22:01:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.312+0000 7fc4237fe640 1 -- 172.21.15.83:0/3762464471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) ==== 174+0+0 (secure 0 0 0) 0x7fc41405b910 con 0x7fc42c0b1260 2024-05-17T22:01:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 10 and data pool 11 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 msgr2=0x7fc40c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 0x7fc40c071910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ad000 tx=0x7fc41c008040 comp rx=0 tx=0).stop 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 msgr2=0x7fc42c145c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c430 tx=0x7fc41400c8f0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 shutdown_connections 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc40c06f400 0x7fc40c071910 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ad000 tx=0x7fc41c008040 comp rx=0 tx=0).stop 2024-05-17T22:01:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc42c15cc90 0x7fc42c14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc42c0b5f30 0x7fc42c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 --2- 172.21.15.83:0/3762464471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c0b1260 0x7fc42c145c40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 >> 172.21.15.83:0/3762464471 conn(0x7fc42c0ce6e0 msgr2=0x7fc42c06c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 shutdown_connections 2024-05-17T22:01:01.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.320+0000 7fc4314f7640 1 -- 172.21.15.83:0/3762464471 wait complete. 2024-05-17T22:01:01.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:01:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 Processor -- start 2024-05-17T22:01:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 -- start start 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc25415ffa0 0x7fc2540d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2540d1150 0x7fc25406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 0x7fc254163a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc25406e610 con 0x7fc2540d1150 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc2540d34e0 con 0x7fc25415ffa0 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc2591bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2540d3780 con 0x7fc25406e200 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2540d1150 0x7fc25406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2540d1150 0x7fc25406dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49968/0 (socket says 172.21.15.83:49968) 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 -- 172.21.15.83:0/2499407916 learned_addr learned my addr 172.21.15.83:0/2499407916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25359e640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 0x7fc254163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc252d9d640 1 --2- 172.21.15.83:0/2499407916 >> v2:172.21.15.155:3300/0 conn(0x7fc25415ffa0 0x7fc2540d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 -- 172.21.15.83:0/2499407916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 msgr2=0x7fc254163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 0x7fc254163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 -- 172.21.15.83:0/2499407916 >> v2:172.21.15.155:3300/0 conn(0x7fc25415ffa0 msgr2=0x7fc2540d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 --2- 172.21.15.83:0/2499407916 >> v2:172.21.15.155:3300/0 conn(0x7fc25415ffa0 0x7fc2540d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25259c640 1 -- 172.21.15.83:0/2499407916 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540d3a60 con 0x7fc2540d1150 2024-05-17T22:01:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.504+0000 7fc25359e640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 0x7fc254163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc25259c640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc25406dcc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc25408e380 tx=0x7fc23c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef7a8bc2ef1690 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc251d9b640 1 -- 172.21.15.83:0/2499407916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b0070 con 0x7fc2540d1150 2024-05-17T22:01:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc251d9b640 1 -- 172.21.15.83:0/2499407916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0ab600 con 0x7fc2540d1150 2024-05-17T22:01:01.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc251d9b640 1 -- 172.21.15.83:0/2499407916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0ab9d0 con 0x7fc2540d1150 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- 172.21.15.83:0/2499407916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 msgr2=0x7fc25406dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc25406dcc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc25408e380 tx=0x7fc23c01c260 comp rx=0 tx=0).stop 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- 172.21.15.83:0/2499407916 shutdown_connections 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25406e200 0x7fc254163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 --2- 172.21.15.83:0/2499407916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc25406dcc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 --2- 172.21.15.83:0/2499407916 >> v2:172.21.15.155:3300/0 conn(0x7fc25415ffa0 0x7fc2540d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- 172.21.15.83:0/2499407916 >> 172.21.15.83:0/2499407916 conn(0x7fc2540d28f0 msgr2=0x7fc2540d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- 172.21.15.83:0/2499407916 shutdown_connections 2024-05-17T22:01:01.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- 172.21.15.83:0/2499407916 wait complete. 2024-05-17T22:01:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 Processor -- start 2024-05-17T22:01:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.508+0000 7fc2591bf640 1 -- start start 2024-05-17T22:01:01.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25406e200 0x7fc2541f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25415ffa0 0x7fc2540b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2540d43e0 con 0x7fc2540d1150 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc2540d4260 con 0x7fc25406e200 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc2540d4560 con 0x7fc25415ffa0 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49976/0 (socket says 172.21.15.83:49976) 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 -- 172.21.15.83:0/1090691127 learned_addr learned my addr 172.21.15.83:0/1090691127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25359e640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25415ffa0 0x7fc2540b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc252d9d640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25406e200 0x7fc2541f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 -- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25415ffa0 msgr2=0x7fc2540b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25415ffa0 0x7fc2540b9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 -- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25406e200 msgr2=0x7fc2541f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25406e200 0x7fc2541f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 -- 172.21.15.83:0/1090691127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25415a410 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc25259c640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01a910 tx=0x7fc23c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b9070 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c0b4600 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25415a8a0 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0b0040 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25415ae60 con 0x7fc2540d1150 2024-05-17T22:01:01.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25415a5e0 con 0x7fc2540d1150 2024-05-17T22:01:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc23c0bc070 con 0x7fc2540d1150 2024-05-17T22:01:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.512+0000 7fc24b7fe640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 0x7fc234071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.516+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fc23c1283b0 con 0x7fc2540d1150 2024-05-17T22:01:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.516+0000 7fc252d9d640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 0x7fc234071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.516+0000 7fc252d9d640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 0x7fc234071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc24400b740 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.516+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc23c016110 con 0x7fc2540d1150 2024-05-17T22:01:01.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.672+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc25406dcc0 con 0x7fc2540d1150 2024-05-17T22:01:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.672+0000 7fc24b7fe640 1 -- 172.21.15.83:0/1090691127 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16657 (secure 0 0 0) 0x7fc23c0f54a0 con 0x7fc2540d1150 2024-05-17T22:01:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:01.666 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:01:00.724524+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:00:58.412570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-17T22:00:59.672732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 msgr2=0x7fc234071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 0x7fc234071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc24400b740 tx=0x7fc244008040 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 msgr2=0x7fc254159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01a910 tx=0x7fc23c019000 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 shutdown_connections 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc25415ffa0 0x7fc2540b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2540d1150 0x7fc254159ed0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc23406f430 0x7fc234071940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 --2- 172.21.15.83:0/1090691127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc25406e200 0x7fc2541f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 >> 172.21.15.83:0/1090691127 conn(0x7fc2540d28f0 msgr2=0x7fc2540ba950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 shutdown_connections 2024-05-17T22:01:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.676+0000 7fc2591bf640 1 -- 172.21.15.83:0/1090691127 wait complete. 2024-05-17T22:01:01.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:01:01.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 Processor -- start 2024-05-17T22:01:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 -- start start 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 0x7f11f4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f40700e0 con 0x7f11f415d8d0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f40aff10 con 0x7f11f406d7a0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f8e10640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11f40709f0 con 0x7f11f40af7c0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f2d9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f11f415d8d0 0x7f11f415dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40286/0 (socket says 172.21.15.83:40286) 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f2d9d640 1 -- 172.21.15.83:0/2858444134 learned_addr learned my addr 172.21.15.83:0/2858444134 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f259c640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 0x7f11f4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3257339548 0 0) 0x7f11f40700e0 con 0x7f11f415d8d0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11e0003530 con 0x7f11f415d8d0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2245876361 0 0) 0x7f11e0003530 con 0x7f11f415d8d0 2024-05-17T22:01:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 msgr2=0x7f11f4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 0x7f11f4070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 >> v1:172.21.15.155:6789/0 conn(0x7f11f406d7a0 legacy=0x7f11f406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f40703a0 con 0x7f11f415d8d0 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f11ec001700 con 0x7f11f415d8d0 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 >> v1:172.21.15.83:6789/0 conn(0x7f11f415d8d0 legacy=0x7f11f415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0003a40 0x7f11e0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0004810 0x7f11e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 0x7f11e00052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11f40709f0 con 0x7f11e0003a40 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11e000bb90 con 0x7f11e0004810 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11e000bd90 con 0x7f11f40af7c0 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f259c640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0003a40 0x7f11e0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.844+0000 7f11f1d9b640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0004810 0x7f11e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 -- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 msgr2=0x7f11e00052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:01.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40af7c0 0x7f11e00052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 -- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0004810 msgr2=0x7f11e0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0004810 0x7f11e0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 -- 172.21.15.83:0/2858444134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e0003530 con 0x7f11e0003a40 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f259c640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0003a40 0x7f11e0003eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fc0 tx=0x7f11e401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbba5b4d20b402d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e40b2070 con 0x7f11e0003a40 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e40a5c60 con 0x7f11e0003a40 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f159a640 1 -- 172.21.15.83:0/2858444134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4009c10 con 0x7f11e0003a40 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 -- 172.21.15.83:0/2858444134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0003a40 msgr2=0x7f11e0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 --2- 172.21.15.83:0/2858444134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0003a40 0x7f11e0003eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fc0 tx=0x7f11e401cc90 comp rx=0 tx=0).stop 2024-05-17T22:01:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f2d9d640 1 -- 172.21.15.83:0/2858444134 reap_dead start 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 -- 172.21.15.83:0/2858444134 shutdown_connections 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 -- 172.21.15.83:0/2858444134 >> 172.21.15.83:0/2858444134 conn(0x7f11f4073670 msgr2=0x7f11f4073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 -- 172.21.15.83:0/2858444134 shutdown_connections 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.848+0000 7f11f8e10640 1 -- 172.21.15.83:0/2858444134 wait complete. 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 Processor -- start 2024-05-17T22:01:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- start start 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0003a40 0x7f11f4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40b2890 0x7f11f40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11f415e980 con 0x7f11f4156dd0 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f11f415e800 con 0x7f11e0003a40 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f11f4068b30 con 0x7f11f40b2890 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49990/0 (socket says 172.21.15.83:49990) 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 -- 172.21.15.83:0/1102717610 learned_addr learned my addr 172.21.15.83:0/1102717610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:01.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f2d9d640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40b2890 0x7f11f40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f259c640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0003a40 0x7f11f4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 -- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40b2890 msgr2=0x7f11f40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40b2890 0x7f11f40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 -- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0003a40 msgr2=0x7f11f4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0003a40 0x7f11f4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 -- 172.21.15.83:0/1102717610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f40b31e0 con 0x7f11f4156dd0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f1d9b640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e510 tx=0x7f11dc00e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0052e0 con 0x7f11f4156dd0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11dc018070 con 0x7f11f4156dd0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41560c0 con 0x7f11f4156dd0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc003820 con 0x7f11f4156dd0 2024-05-17T22:01:01.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.852+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f4156680 con 0x7f11f4156dd0 2024-05-17T22:01:01.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f4155e10 con 0x7f11f4156dd0 2024-05-17T22:01:01.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11dc0039c0 con 0x7f11f4156dd0 2024-05-17T22:01:01.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11eaffd640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 0x7f11d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:01.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f11dc08f010 con 0x7f11f4156dd0 2024-05-17T22:01:01.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11f259c640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 0x7f11d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:01.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.856+0000 7f11f259c640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 0x7f11d4071910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11e40af000 tx=0x7f11e40a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:01.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:01.860+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f11f40b31e0 con 0x7f11f4156dd0 2024-05-17T22:01:02.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.020+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11f415dce0 con 0x7f11f4156dd0 2024-05-17T22:01:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11eaffd640 1 -- 172.21.15.83:0/1102717610 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16657 (secure 0 0 0) 0x7f11dc05bf00 con 0x7f11f4156dd0 2024-05-17T22:01:02.016 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:02.016 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:01:00.724524+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:00:58.412570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-17T22:00:59.672732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 msgr2=0x7f11d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 0x7f11d4071910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f11e40af000 tx=0x7f11e40a7040 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 msgr2=0x7f11f4155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00e510 tx=0x7f11dc00e9d0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 shutdown_connections 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11f40b2890 0x7f11f40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11f4156dd0 0x7f11f4155250 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f11d406f400 0x7f11d4071910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 --2- 172.21.15.83:0/1102717610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11e0003a40 0x7f11f4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 >> 172.21.15.83:0/1102717610 conn(0x7f11f4073670 msgr2=0x7f11f4161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:02.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 shutdown_connections 2024-05-17T22:01:02.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.024+0000 7f11f8e10640 1 -- 172.21.15.83:0/1102717610 wait complete. 2024-05-17T22:01:02.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T22:01:02.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 Processor -- start 2024-05-17T22:01:02.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 -- start start 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 0x7f26c80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c80d4170 con 0x7f26c80b4810 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80d5620 con 0x7f26c80d5210 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cf5ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80d4450 con 0x7f26c80d2000 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26ccb87640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 0x7f26c80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cdb89640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cdb89640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:34148/0 (socket says 172.21.15.83:34148) 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26ccb87640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 0x7f26c80d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54966/0 (socket says 172.21.15.83:54966) 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cdb89640 1 -- 172.21.15.83:0/1027635714 learned_addr learned my addr 172.21.15.83:0/1027635714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.188+0000 7f26cd388640 1 --1- 172.21.15.83:0/1027635714 >> v1:172.21.15.155:6789/0 conn(0x7f26c80b4810 0x7f26c815fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:38200/0 (socket says 172.21.15.83:38200) 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26cdb89640 1 -- 172.21.15.83:0/1027635714 >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 msgr2=0x7f26c80d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26cdb89640 1 --2- 172.21.15.83:0/1027635714 >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 0x7f26c80d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26cdb89640 1 -- 172.21.15.83:0/1027635714 >> v1:172.21.15.155:6789/0 conn(0x7f26c80b4810 legacy=0x7f26c815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26cdb89640 1 -- 172.21.15.83:0/1027635714 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80d4730 con 0x7f26c80d2000 2024-05-17T22:01:02.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26cdb89640 1 --2- 172.21.15.83:0/1027635714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009870 tx=0x7f26c001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6337efaccdd23b61 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26c7fff640 1 -- 172.21.15.83:0/1027635714 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00a9640 con 0x7f26c80d2000 2024-05-17T22:01:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26c7fff640 1 -- 172.21.15.83:0/1027635714 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c00a97e0 con 0x7f26c80d2000 2024-05-17T22:01:02.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.192+0000 7f26c7fff640 1 -- 172.21.15.83:0/1027635714 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00a9bb0 con 0x7f26c80d2000 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- 172.21.15.83:0/1027635714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 msgr2=0x7f26c80d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/1027635714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009870 tx=0x7f26c001c560 comp rx=0 tx=0).stop 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- 172.21.15.83:0/1027635714 shutdown_connections 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/1027635714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d2000 0x7f26c80d2410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/1027635714 >> v2:172.21.15.155:3300/0 conn(0x7f26c80d5210 0x7f26c80d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- 172.21.15.83:0/1027635714 >> 172.21.15.83:0/1027635714 conn(0x7f26c806e010 msgr2=0x7f26c806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- 172.21.15.83:0/1027635714 shutdown_connections 2024-05-17T22:01:02.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- 172.21.15.83:0/1027635714 wait complete. 2024-05-17T22:01:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 Processor -- start 2024-05-17T22:01:02.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.196+0000 7f26cf5ec640 1 -- start start 2024-05-17T22:01:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 0x7f26c8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 0x7f26c8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b9670 con 0x7f26c80b4810 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b94f0 con 0x7f26c80d2000 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b97f0 con 0x7f26c80d5210 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49994/0 (socket says 172.21.15.83:49994) 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 -- 172.21.15.83:0/3241147802 learned_addr learned my addr 172.21.15.83:0/3241147802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cdb89640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 0x7f26c8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26ccb87640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 0x7f26c8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 -- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 msgr2=0x7f26c8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 0x7f26c8159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 -- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 msgr2=0x7f26c8154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 0x7f26c8154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 -- 172.21.15.83:0/3241147802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80b4f90 con 0x7f26c80b4810 2024-05-17T22:01:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26ccb87640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 0x7f26c8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cdb89640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 0x7f26c8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cd388640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c430 tx=0x7f26b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80074e0 con 0x7f26c80b4810 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8007680 con 0x7f26c80b4810 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c80b5270 con 0x7f26c80b4810 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8004a10 con 0x7f26c80b4810 2024-05-17T22:01:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c80b5830 con 0x7f26c80b4810 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c80d2410 con 0x7f26c80b4810 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26b801a020 con 0x7f26c80b4810 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 0x7f26b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f26b808eb00 con 0x7f26c80b4810 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.200+0000 7f26ccb87640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 0x7f26b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.204+0000 7f26ccb87640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 0x7f26b4071910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8157b00 tx=0x7f26bc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:02.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.204+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f26b805b9f0 con 0x7f26c80b4810 2024-05-17T22:01:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:02.392+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f26c80d0c00 con 0x7f26c80b4810 2024-05-17T22:01:03.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.352+0000 7f26c5ffb640 1 -- 172.21.15.83:0/3241147802 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7f26b805b2f0 con 0x7f26c80b4810 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 msgr2=0x7f26b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 0x7f26b4071910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8157b00 tx=0x7f26bc00b040 comp rx=0 tx=0).stop 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 msgr2=0x7f26c81540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c430 tx=0x7f26b800c8f0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 shutdown_connections 2024-05-17T22:01:03.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f26c80d5210 0x7f26c8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f26b406f400 0x7f26b4071910 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26c80d2000 0x7f26c8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 --2- 172.21.15.83:0/3241147802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26c80b4810 0x7f26c81540d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 >> 172.21.15.83:0/3241147802 conn(0x7f26c806e010 msgr2=0x7f26c8157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 shutdown_connections 2024-05-17T22:01:03.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.356+0000 7f26cf5ec640 1 -- 172.21.15.83:0/3241147802 wait complete. 2024-05-17T22:01:03.371 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T22:01:03.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T22:01:03.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 Processor -- start 2024-05-17T22:01:03.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 -- start start 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4580b3d00 0x7fd4580d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 0x7fd4581447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4580d10d0 con 0x7fd458143f20 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd4580b4110 con 0x7fd4580b3d00 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45f526640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd458070770 con 0x7fd458073590 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4580b3d00 0x7fd4580d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4580b3d00 0x7fd4580d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50008/0 (socket says 172.21.15.83:50008) 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 -- 172.21.15.83:0/4294947387 learned_addr learned my addr 172.21.15.83:0/4294947387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:03.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45dac3640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 0x7fd4581447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 -- 172.21.15.83:0/4294947387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 msgr2=0x7fd4581447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:03.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 0x7fd4581447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 -- 172.21.15.83:0/4294947387 >> v1:172.21.15.155:6789/0 conn(0x7fd458143f20 legacy=0x7fd458072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:03.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45d2c2640 1 -- 172.21.15.83:0/4294947387 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd458070120 con 0x7fd4580b3d00 2024-05-17T22:01:03.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.536+0000 7fd45dac3640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 0x7fd4581447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:03.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45d2c2640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4580b3d00 0x7fd4580d0a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd454009870 tx=0x7fd45401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f9415a748093e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:03.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd44ffff640 1 -- 172.21.15.83:0/4294947387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540b0070 con 0x7fd4580b3d00 2024-05-17T22:01:03.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd44ffff640 1 -- 172.21.15.83:0/4294947387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4540ab600 con 0x7fd4580b3d00 2024-05-17T22:01:03.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd44ffff640 1 -- 172.21.15.83:0/4294947387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540ab9d0 con 0x7fd4580b3d00 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 -- 172.21.15.83:0/4294947387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4580b3d00 msgr2=0x7fd4580d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4580b3d00 0x7fd4580d0a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd454009870 tx=0x7fd45401c560 comp rx=0 tx=0).stop 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 -- 172.21.15.83:0/4294947387 shutdown_connections 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458073590 0x7fd4581447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 --2- 172.21.15.83:0/4294947387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4580b3d00 0x7fd4580d0a10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 -- 172.21.15.83:0/4294947387 >> 172.21.15.83:0/4294947387 conn(0x7fd45806cd90 msgr2=0x7fd4580b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 -- 172.21.15.83:0/4294947387 shutdown_connections 2024-05-17T22:01:03.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.540+0000 7fd45f526640 1 -- 172.21.15.83:0/4294947387 wait complete. 2024-05-17T22:01:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 Processor -- start 2024-05-17T22:01:03.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- start start 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4580b3d00 0x7fd4580af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458143f20 0x7fd4580afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd458070c00 con 0x7fd458073590 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd458145540 con 0x7fd4580b3d00 2024-05-17T22:01:03.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd45815fa00 con 0x7fd458143f20 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50018/0 (socket says 172.21.15.83:50018) 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 -- 172.21.15.83:0/305865622 learned_addr learned my addr 172.21.15.83:0/305865622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45dac3640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458143f20 0x7fd4580afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45cac1640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4580b3d00 0x7fd4580af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 -- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458143f20 msgr2=0x7fd4580afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458143f20 0x7fd4580afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 -- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4580b3d00 msgr2=0x7fd4580af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4580b3d00 0x7fd4580af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 -- 172.21.15.83:0/305865622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4580b4b60 con 0x7fd458073590 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45d2c2640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd45401ce00 tx=0x7fd454019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540b0040 con 0x7fd458073590 2024-05-17T22:01:03.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4540b4440 con 0x7fd458073590 2024-05-17T22:01:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4580b4e40 con 0x7fd458073590 2024-05-17T22:01:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540b4830 con 0x7fd458073590 2024-05-17T22:01:03.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.544+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4580b5400 con 0x7fd458073590 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4581447f0 con 0x7fd458073590 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4540bb020 con 0x7fd458073590 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd44dffb640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 0x7fd43c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fd4541286e0 con 0x7fd458073590 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd45cac1640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 0x7fd43c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:03.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.548+0000 7fd45cac1640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 0x7fd43c071940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd458137350 tx=0x7fd448008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:03.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.552+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd4540c2020 con 0x7fd458073590 2024-05-17T22:01:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:03.748+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd4580b09f0 con 0x7fd458073590 2024-05-17T22:01:04.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.356+0000 7fd44dffb640 1 -- 172.21.15.83:0/305865622 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fd454016150 con 0x7fd458073590 2024-05-17T22:01:04.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 msgr2=0x7fd43c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 0x7fd43c071940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd458137350 tx=0x7fd448008040 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 msgr2=0x7fd45815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd45401ce00 tx=0x7fd454019300 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 shutdown_connections 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd458143f20 0x7fd4580afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd43c06f430 0x7fd43c071940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4580b3d00 0x7fd4580af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 --2- 172.21.15.83:0/305865622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd458073590 0x7fd45815d480 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 >> 172.21.15.83:0/305865622 conn(0x7fd45806cd90 msgr2=0x7fd4580b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 shutdown_connections 2024-05-17T22:01:04.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.360+0000 7fd45f526640 1 -- 172.21.15.83:0/305865622 wait complete. 2024-05-17T22:01:04.378 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T22:01:04.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:04.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 Processor -- start 2024-05-17T22:01:04.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 -- start start 2024-05-17T22:01:04.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58c0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c015c830 con 0x7f58c0072e30 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c01608a0 con 0x7f58c015d2f0 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58c5f0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f58c0160b80 con 0x7f58c015fe20 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f58c0072e30 0x7f58c0073240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40292/0 (socket says 172.21.15.83:40292) 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bffff640 1 -- 172.21.15.83:0/3831352650 learned_addr learned my addr 172.21.15.83:0/3831352650 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58beffd640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58c0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346348643 0 0) 0x7f58c015c830 con 0x7f58c0072e30 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58a8004640 con 0x7f58c0072e30 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 283940712 0 0) 0x7f58a8004640 con 0x7f58c0072e30 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 msgr2=0x7f58c0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58c0160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 >> v1:172.21.15.155:6789/0 conn(0x7f58c015d2f0 legacy=0x7f58c0159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c015caf0 con 0x7f58c0072e30 2024-05-17T22:01:04.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f58b8001a80 con 0x7f58c0072e30 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 >> v1:172.21.15.83:6789/0 conn(0x7f58c0072e30 legacy=0x7f58c0073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58a8004b50 0x7f58a8004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8005920 0x7f58a8005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58a800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0160b80 con 0x7f58a8004b50 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58a80066b0 con 0x7f58a8005920 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f58a800d020 con 0x7f58c015fe20 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58a8004b50 0x7f58a8004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58beffd640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8005920 0x7f58a8005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bffff640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58a800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 -- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 msgr2=0x7f58a800cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58a800cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 -- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8005920 msgr2=0x7f58a8005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8005920 0x7f58a8005d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bf7fe640 1 -- 172.21.15.83:0/3831352650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a8004640 con 0x7f58a8004b50 2024-05-17T22:01:04.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.540+0000 7f58bffff640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c015fe20 0x7f58a800cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58a8004b50 0x7f58a8004fc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58b0001790 tx=0x7f58b001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fb83bea31f2c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b00013e0 con 0x7f58a8004b50 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b00052d0 con 0x7f58a8004b50 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58be7fc640 1 -- 172.21.15.83:0/3831352650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0003840 con 0x7f58a8004b50 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3831352650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58a8004b50 msgr2=0x7f58a8004fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3831352650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58a8004b50 0x7f58a8004fc0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58b0001790 tx=0x7f58b001c3e0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58bffff640 1 -- 172.21.15.83:0/3831352650 reap_dead start 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3831352650 shutdown_connections 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3831352650 >> 172.21.15.83:0/3831352650 conn(0x7f58c006cd90 msgr2=0x7f58c00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3831352650 shutdown_connections 2024-05-17T22:01:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.544+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3831352650 wait complete. 2024-05-17T22:01:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 Processor -- start 2024-05-17T22:01:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- start start 2024-05-17T22:01:04.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 0x7f58c014b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 0x7f58c0141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b0700 con 0x7f58c0073240 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b0580 con 0x7f58a8004b50 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00b0880 con 0x7f58c0163990 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50042/0 (socket says 172.21.15.83:50042) 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 -- 172.21.15.83:0/3223806319 learned_addr learned my addr 172.21.15.83:0/3223806319 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58beffd640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 0x7f58c0141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 0x7f58c014b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 -- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 msgr2=0x7f58c0141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 0x7f58c0141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 -- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 msgr2=0x7f58c014b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 0x7f58c014b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 -- 172.21.15.83:0/3223806319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c0073b90 con 0x7f58c0073240 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58beffd640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 0x7f58c0141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 0x7f58c014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58bffff640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58b8001340 tx=0x7f58b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8004d70 con 0x7f58c0073240 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b800bb80 con 0x7f58c0073240 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8007070 con 0x7f58c0073240 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c014c1f0 con 0x7f58c0073240 2024-05-17T22:01:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c0161ae0 con 0x7f58c0073240 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.548+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c014bf60 con 0x7f58c0073240 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.552+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f58b801b060 con 0x7f58c0073240 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.552+0000 7f58a7fff640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 0x7f58ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.552+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f58b808f240 con 0x7f58c0073240 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.552+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 0x7f58ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.552+0000 7f58bf7fe640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 0x7f58ac071910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58b00ac000 tx=0x7f58b00011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.556+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f58c0073b90 con 0x7f58c0073240 2024-05-17T22:01:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.748+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c01596a0 con 0x7f58c0073240 2024-05-17T22:01:04.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.748+0000 7f58a7fff640 1 -- 172.21.15.83:0/3223806319 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5186 (secure 0 0 0) 0x7f58b805c130 con 0x7f58c0073240 2024-05-17T22:01:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:04.743 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:03.754919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1850779527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1850779527},{"type":"v1","addr":"172.21.15.83:6837","nonce":1850779527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":5}]} 2024-05-17T22:01:04.743 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 43 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 msgr2=0x7f58ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 0x7f58ac071910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58b00ac000 tx=0x7f58b00011a0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 msgr2=0x7f58c0073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58b8001340 tx=0x7f58b800b4a0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 shutdown_connections 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0073240 0x7f58c0073650 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c0163990 0x7f58c0141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f58ac06f400 0x7f58ac071910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 --2- 172.21.15.83:0/3223806319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a8004b50 0x7f58c014b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 >> 172.21.15.83:0/3223806319 conn(0x7f58c006cd90 msgr2=0x7f58c00b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 shutdown_connections 2024-05-17T22:01:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.752+0000 7f58c5f0c640 1 -- 172.21.15.83:0/3223806319 wait complete. 2024-05-17T22:01:04.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:01:04.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 Processor -- start 2024-05-17T22:01:04.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 -- start start 2024-05-17T22:01:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f94c40d1040 0x7f94c4144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 0x7f94c406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c415ea40 con 0x7f94c415e080 2024-05-17T22:01:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f94c415ed20 con 0x7f94c40d1040 2024-05-17T22:01:04.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94cb87a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94c415f000 con 0x7f94c4073540 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c8e15640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f94c415e080 0x7f94c4072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40308/0 (socket says 172.21.15.83:40308) 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c8e15640 1 -- 172.21.15.83:0/1249529530 learned_addr learned my addr 172.21.15.83:0/1249529530 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9e17640 1 --2- 172.21.15.83:0/1249529530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 0x7f94c406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9616640 1 --2- 172.21.15.83:0/1249529530 >> v2:172.21.15.155:3300/0 conn(0x7f94c40d1040 0x7f94c4144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967268009 0 0) 0x7f94c415ea40 con 0x7f94c415e080 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94b0003530 con 0x7f94c415e080 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2667497901 0 0) 0x7f94b0003530 con 0x7f94c415e080 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9e17640 1 -- 172.21.15.83:0/1249529530 >> v2:172.21.15.155:3300/0 conn(0x7f94c40d1040 msgr2=0x7f94c4144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9e17640 1 --2- 172.21.15.83:0/1249529530 >> v2:172.21.15.155:3300/0 conn(0x7f94c40d1040 0x7f94c4144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9e17640 1 -- 172.21.15.83:0/1249529530 >> v1:172.21.15.83:6789/0 conn(0x7f94c415e080 legacy=0x7f94c4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.916+0000 7f94c9e17640 1 -- 172.21.15.83:0/1249529530 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c406ef30 con 0x7f94c4073540 2024-05-17T22:01:04.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94c9e17640 1 --2- 172.21.15.83:0/1249529530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 0x7f94c406f730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009d90 tx=0x7f94bc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16c5da39d40f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0b2070 con 0x7f94c4073540 2024-05-17T22:01:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc0a55a0 con 0x7f94c4073540 2024-05-17T22:01:04.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94bbfff640 1 -- 172.21.15.83:0/1249529530 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4001700 con 0x7f94c4073540 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 -- 172.21.15.83:0/1249529530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 msgr2=0x7f94c406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 --2- 172.21.15.83:0/1249529530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 0x7f94c406f730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009d90 tx=0x7f94bc01c890 comp rx=0 tx=0).stop 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 -- 172.21.15.83:0/1249529530 shutdown_connections 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 --2- 172.21.15.83:0/1249529530 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c4073540 0x7f94c406f730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 --2- 172.21.15.83:0/1249529530 >> v2:172.21.15.155:3300/0 conn(0x7f94c40d1040 0x7f94c4144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 -- 172.21.15.83:0/1249529530 >> 172.21.15.83:0/1249529530 conn(0x7f94c406cd90 msgr2=0x7f94c40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 -- 172.21.15.83:0/1249529530 shutdown_connections 2024-05-17T22:01:04.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.920+0000 7f94cb87a640 1 -- 172.21.15.83:0/1249529530 wait complete. 2024-05-17T22:01:04.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 Processor -- start 2024-05-17T22:01:04.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- start start 2024-05-17T22:01:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c415e080 0x7f94c41613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 0x7f94c4161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4070810 con 0x7f94c40d1040 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f94c4070690 con 0x7f94c415d7c0 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f94c4070990 con 0x7f94c415e080 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50052/0 (socket says 172.21.15.83:50052) 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 -- 172.21.15.83:0/3380625675 learned_addr learned my addr 172.21.15.83:0/3380625675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c8e15640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c415e080 0x7f94c41613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9e17640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 0x7f94c4161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 -- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c415e080 msgr2=0x7f94c41613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c415e080 0x7f94c41613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 -- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 msgr2=0x7f94c4161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 0x7f94c4161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 -- 172.21.15.83:0/3380625675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c415c050 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9e17640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 0x7f94c4161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94c9616640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f94c000e8a0 tx=0x7f94c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0017070 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c000c940 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c415c840 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c000ccf0 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.924+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c415ce00 con 0x7f94c40d1040 2024-05-17T22:01:04.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c406f730 con 0x7f94c40d1040 2024-05-17T22:01:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f94c001a070 con 0x7f94c40d1040 2024-05-17T22:01:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94b9ffb640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 0x7f94a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:04.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94c8e15640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 0x7f94a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:04.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f94c008f120 con 0x7f94c40d1040 2024-05-17T22:01:04.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.928+0000 7f94c8e15640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 0x7f94a8071940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94b4001ec0 tx=0x7f94b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:04.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:04.932+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f94c005c0a0 con 0x7f94c40d1040 2024-05-17T22:01:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.088+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94c4147490 con 0x7f94c40d1040 2024-05-17T22:01:05.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.088+0000 7f94b9ffb640 1 -- 172.21.15.83:0/3380625675 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16644 (secure 0 0 0) 0x7f94c005b9a0 con 0x7f94c40d1040 2024-05-17T22:01:05.082 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:05.082 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":67,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:01:02.334806+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:00:58.412570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-17T22:00:59.672732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:01:05.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 msgr2=0x7f94a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 0x7f94a8071940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94b4001ec0 tx=0x7f94b4009040 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 msgr2=0x7f94c415bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f94c000e8a0 tx=0x7f94c000eca0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 shutdown_connections 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f94a806f430 0x7f94a8071940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94c415d7c0 0x7f94c4161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94c415e080 0x7f94c41613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 --2- 172.21.15.83:0/3380625675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94c40d1040 0x7f94c415bb10 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 >> 172.21.15.83:0/3380625675 conn(0x7f94c406cd90 msgr2=0x7f94c4143260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 shutdown_connections 2024-05-17T22:01:05.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.092+0000 7f94cb87a640 1 -- 172.21.15.83:0/3380625675 wait complete. 2024-05-17T22:01:05.104 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:01:05.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:01:05.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 Processor -- start 2024-05-17T22:01:05.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 -- start start 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7facec1461c0 0x7facec15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec0706e0 0x7facec1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facec1465d0 con 0x7facec15e6b0 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7facec06ef30 con 0x7facec1461c0 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf22d1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facec06f210 con 0x7facec0706e0 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7facec1461c0 0x7facec15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7facec1461c0 0x7facec15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50066/0 (socket says 172.21.15.83:50066) 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 -- 172.21.15.83:0/2229388422 learned_addr learned my addr 172.21.15.83:0/2229388422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facf086e640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec0706e0 0x7facec1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 -- 172.21.15.83:0/2229388422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec0706e0 msgr2=0x7facec1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec0706e0 0x7facec1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 -- 172.21.15.83:0/2229388422 >> v1:172.21.15.155:6789/0 conn(0x7facec15e6b0 legacy=0x7facec06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:05.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.252+0000 7facebfff640 1 -- 172.21.15.83:0/2229388422 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec06f4f0 con 0x7facec1461c0 2024-05-17T22:01:05.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facebfff640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1461c0 0x7facec15e170 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7facdc009870 tx=0x7facdc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c619ebb88720e886 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7faceb7fe640 1 -- 172.21.15.83:0/2229388422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0b1070 con 0x7facec1461c0 2024-05-17T22:01:05.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7faceb7fe640 1 -- 172.21.15.83:0/2229388422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc0ac620 con 0x7facec1461c0 2024-05-17T22:01:05.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7faceb7fe640 1 -- 172.21.15.83:0/2229388422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0ac9f0 con 0x7facec1461c0 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 -- 172.21.15.83:0/2229388422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1461c0 msgr2=0x7facec15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1461c0 0x7facec15e170 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7facdc009870 tx=0x7facdc01c530 comp rx=0 tx=0).stop 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 -- 172.21.15.83:0/2229388422 shutdown_connections 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec0706e0 0x7facec1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 --2- 172.21.15.83:0/2229388422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1461c0 0x7facec15e170 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 -- 172.21.15.83:0/2229388422 >> 172.21.15.83:0/2229388422 conn(0x7facec06cd90 msgr2=0x7facec0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 -- 172.21.15.83:0/2229388422 shutdown_connections 2024-05-17T22:01:05.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.256+0000 7facf22d1640 1 -- 172.21.15.83:0/2229388422 wait complete. 2024-05-17T22:01:05.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 Processor -- start 2024-05-17T22:01:05.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- start start 2024-05-17T22:01:05.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec15e6b0 0x7facec138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 0x7facec13a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facec169940 con 0x7facec0706e0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7facec0d12b0 con 0x7facec13a3a0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7facec169ac0 con 0x7facec15e6b0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50076/0 (socket says 172.21.15.83:50076) 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 -- 172.21.15.83:0/404839452 learned_addr learned my addr 172.21.15.83:0/404839452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7face3fff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec15e6b0 0x7facec138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf086e640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 0x7facec13a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 -- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec15e6b0 msgr2=0x7facec138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec15e6b0 0x7facec138850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 -- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 msgr2=0x7facec13a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 0x7facec13a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 -- 172.21.15.83:0/404839452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec138d90 con 0x7facec0706e0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf086e640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 0x7facec13a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facebfff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af040 tx=0x7facdc0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0b1040 con 0x7facec0706e0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc0ac420 con 0x7facec0706e0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec13acf0 con 0x7facec0706e0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0b56a0 con 0x7facec0706e0 2024-05-17T22:01:05.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec140ae0 con 0x7facec0706e0 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.260+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facec15e170 con 0x7facec0706e0 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.264+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7facdc0bc020 con 0x7facec0706e0 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.264+0000 7face97fa640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 0x7faccc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.264+0000 7face3fff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 0x7faccc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.264+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7facdc1287f0 con 0x7facec0706e0 2024-05-17T22:01:05.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.264+0000 7face3fff640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 0x7faccc071910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facec13eeb0 tx=0x7facd4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.268+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7facdc0c3020 con 0x7facec0706e0 2024-05-17T22:01:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.424+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facec15c560 con 0x7faccc06f400 2024-05-17T22:01:05.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.436+0000 7face97fa640 1 -- 172.21.15.83:0/404839452 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7facd400bd80 con 0x7faccc06f400 2024-05-17T22:01:05.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:05.431 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:01:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 msgr2=0x7faccc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 0x7faccc071910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facec13eeb0 tx=0x7facd4008040 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 msgr2=0x7facec15c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af040 tx=0x7facdc0192d0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 shutdown_connections 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faccc06f400 0x7faccc071910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facec13a3a0 0x7facec13a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facec15e6b0 0x7facec138850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 --2- 172.21.15.83:0/404839452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec0706e0 0x7facec15c020 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 >> 172.21.15.83:0/404839452 conn(0x7facec06cd90 msgr2=0x7facec15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 shutdown_connections 2024-05-17T22:01:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.440+0000 7facf22d1640 1 -- 172.21.15.83:0/404839452 wait complete. 2024-05-17T22:01:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-05-17T22:01:05.043019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252880,"kb_used_data":7580,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721840,"statfs":{"total":773094113280,"available":772835164160,"internally_reserved":0,"allocated":7761920,"data_stored":3908720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.755496"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408887+0000","last_change":"2024-05-17T22:01:01.383630+0000","last_active":"2024-05-17T22:01:02.408887+0000","last_peered":"2024-05-17T22:01:02.408887+0000","last_clean":"2024-05-17T22:01:02.408887+0000","last_became_active":"2024-05-17T22:01:01.380878+0000","last_became_peered":"2024-05-17T22:01:01.380878+0000","last_unstale":"2024-05-17T22:01:02.408887+0000","last_undegraded":"2024-05-17T22:01:02.408887+0000","last_fullsized":"2024-05-17T22:01:02.408887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:01:58.690181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461994+0000","last_change":"2024-05-17T22:01:01.396593+0000","last_active":"2024-05-17T22:01:03.461994+0000","last_peered":"2024-05-17T22:01:03.461994+0000","last_clean":"2024-05-17T22:01:03.461994+0000","last_became_active":"2024-05-17T22:01:01.395937+0000","last_became_peered":"2024-05-17T22:01:01.395937+0000","last_unstale":"2024-05-17T22:01:03.461994+0000","last_undegraded":"2024-05-17T22:01:03.461994+0000","last_fullsized":"2024-05-17T22:01:03.461994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:51.193518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458853+0000","last_change":"2024-05-17T22:01:01.341074+0000","last_active":"2024-05-17T22:01:03.458853+0000","last_peered":"2024-05-17T22:01:03.458853+0000","last_clean":"2024-05-17T22:01:03.458853+0000","last_became_active":"2024-05-17T22:01:01.340248+0000","last_became_peered":"2024-05-17T22:01:01.340248+0000","last_unstale":"2024-05-17T22:01:03.458853+0000","last_undegraded":"2024-05-17T22:01:03.458853+0000","last_fullsized":"2024-05-17T22:01:03.458853+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:35.273265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459196+0000","last_change":"2024-05-17T22:01:01.397884+0000","last_active":"2024-05-17T22:01:03.459196+0000","last_peered":"2024-05-17T22:01:03.459196+0000","last_clean":"2024-05-17T22:01:03.459196+0000","last_became_active":"2024-05-17T22:01:01.397503+0000","last_became_peered":"2024-05-17T22:01:01.397503+0000","last_unstale":"2024-05-17T22:01:03.459196+0000","last_undegraded":"2024-05-17T22:01:03.459196+0000","last_fullsized":"2024-05-17T22:01:03.459196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:49.836526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347738+0000","last_change":"2024-05-17T22:01:01.365695+0000","last_active":"2024-05-17T22:01:02.347738+0000","last_peered":"2024-05-17T22:01:02.347738+0000","last_clean":"2024-05-17T22:01:02.347738+0000","last_became_active":"2024-05-17T22:01:01.365286+0000","last_became_peered":"2024-05-17T22:01:01.365286+0000","last_unstale":"2024-05-17T22:01:02.347738+0000","last_undegraded":"2024-05-17T22:01:02.347738+0000","last_fullsized":"2024-05-17T22:01:02.347738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:14:52.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461444+0000","last_change":"2024-05-17T22:01:01.395615+0000","last_active":"2024-05-17T22:01:03.461444+0000","last_peered":"2024-05-17T22:01:03.461444+0000","last_clean":"2024-05-17T22:01:03.461444+0000","last_became_active":"2024-05-17T22:01:01.392160+0000","last_became_peered":"2024-05-17T22:01:01.392160+0000","last_unstale":"2024-05-17T22:01:03.461444+0000","last_undegraded":"2024-05-17T22:01:03.461444+0000","last_fullsized":"2024-05-17T22:01:03.461444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:45:02.348773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349113+0000","last_change":"2024-05-17T22:01:01.338148+0000","last_active":"2024-05-17T22:01:02.349113+0000","last_peered":"2024-05-17T22:01:02.349113+0000","last_clean":"2024-05-17T22:01:02.349113+0000","last_became_active":"2024-05-17T22:01:01.337277+0000","last_became_peered":"2024-05-17T22:01:01.337277+0000","last_unstale":"2024-05-17T22:01:02.349113+0000","last_undegraded":"2024-05-17T22:01:02.349113+0000","last_fullsized":"2024-05-17T22:01:02.349113+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:45.219441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462145+0000","last_change":"2024-05-17T22:01:01.395528+0000","last_active":"2024-05-17T22:01:03.462145+0000","last_peered":"2024-05-17T22:01:03.462145+0000","last_clean":"2024-05-17T22:01:03.462145+0000","last_became_active":"2024-05-17T22:01:01.394048+0000","last_became_peered":"2024-05-17T22:01:01.394048+0000","last_unstale":"2024-05-17T22:01:03.462145+0000","last_undegraded":"2024-05-17T22:01:03.462145+0000","last_fullsized":"2024-05-17T22:01:03.462145+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:21.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348097+0000","last_change":"2024-05-17T22:01:01.346394+0000","last_active":"2024-05-17T22:01:02.348097+0000","last_peered":"2024-05-17T22:01:02.348097+0000","last_clean":"2024-05-17T22:01:02.348097+0000","last_became_active":"2024-05-17T22:01:01.345640+0000","last_became_peered":"2024-05-17T22:01:01.345640+0000","last_unstale":"2024-05-17T22:01:02.348097+0000","last_undegraded":"2024-05-17T22:01:02.348097+0000","last_fullsized":"2024-05-17T22:01:02.348097+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:59:14.633720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348805+0000","last_change":"2024-05-17T22:01:01.352419+0000","last_active":"2024-05-17T22:01:02.348805+0000","last_peered":"2024-05-17T22:01:02.348805+0000","last_clean":"2024-05-17T22:01:02.348805+0000","last_became_active":"2024-05-17T22:01:01.351601+0000","last_became_peered":"2024-05-17T22:01:01.351601+0000","last_unstale":"2024-05-17T22:01:02.348805+0000","last_undegraded":"2024-05-17T22:01:02.348805+0000","last_fullsized":"2024-05-17T22:01:02.348805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:21.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407965+0000","last_change":"2024-05-17T22:01:01.378591+0000","last_active":"2024-05-17T22:01:02.407965+0000","last_peered":"2024-05-17T22:01:02.407965+0000","last_clean":"2024-05-17T22:01:02.407965+0000","last_became_active":"2024-05-17T22:01:01.378002+0000","last_became_peered":"2024-05-17T22:01:01.378002+0000","last_unstale":"2024-05-17T22:01:02.407965+0000","last_undegraded":"2024-05-17T22:01:02.407965+0000","last_fullsized":"2024-05-17T22:01:02.407965+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:02:23.028552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459397+0000","last_change":"2024-05-17T22:01:01.335434+0000","last_active":"2024-05-17T22:01:03.459397+0000","last_peered":"2024-05-17T22:01:03.459397+0000","last_clean":"2024-05-17T22:01:03.459397+0000","last_became_active":"2024-05-17T22:01:01.334998+0000","last_became_peered":"2024-05-17T22:01:01.334998+0000","last_unstale":"2024-05-17T22:01:03.459397+0000","last_undegraded":"2024-05-17T22:01:03.459397+0000","last_fullsized":"2024-05-17T22:01:03.459397+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:34.124011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332329+0000","last_change":"2024-05-17T22:01:01.393015+0000","last_active":"2024-05-17T22:01:03.332329+0000","last_peered":"2024-05-17T22:01:03.332329+0000","last_clean":"2024-05-17T22:01:03.332329+0000","last_became_active":"2024-05-17T22:01:01.392624+0000","last_became_peered":"2024-05-17T22:01:01.392624+0000","last_unstale":"2024-05-17T22:01:03.332329+0000","last_undegraded":"2024-05-17T22:01:03.332329+0000","last_fullsized":"2024-05-17T22:01:03.332329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:35:36.659116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347699+0000","last_change":"2024-05-17T22:01:01.354909+0000","last_active":"2024-05-17T22:01:02.347699+0000","last_peered":"2024-05-17T22:01:02.347699+0000","last_clean":"2024-05-17T22:01:02.347699+0000","last_became_active":"2024-05-17T22:01:01.354549+0000","last_became_peered":"2024-05-17T22:01:01.354549+0000","last_unstale":"2024-05-17T22:01:02.347699+0000","last_undegraded":"2024-05-17T22:01:02.347699+0000","last_fullsized":"2024-05-17T22:01:02.347699+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:01:00.201810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331844+0000","last_change":"2024-05-17T22:01:01.338289+0000","last_active":"2024-05-17T22:01:03.331844+0000","last_peered":"2024-05-17T22:01:03.331844+0000","last_clean":"2024-05-17T22:01:03.331844+0000","last_became_active":"2024-05-17T22:01:01.337978+0000","last_became_peered":"2024-05-17T22:01:01.337978+0000","last_unstale":"2024-05-17T22:01:03.331844+0000","last_undegraded":"2024-05-17T22:01:03.331844+0000","last_fullsized":"2024-05-17T22:01:03.331844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:18:31.323279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349879+0000","last_change":"2024-05-17T22:01:01.404748+0000","last_active":"2024-05-17T22:01:02.349879+0000","last_peered":"2024-05-17T22:01:02.349879+0000","last_clean":"2024-05-17T22:01:02.349879+0000","last_became_active":"2024-05-17T22:01:01.403624+0000","last_became_peered":"2024-05-17T22:01:01.403624+0000","last_unstale":"2024-05-17T22:01:02.349879+0000","last_undegraded":"2024-05-17T22:01:02.349879+0000","last_fullsized":"2024-05-17T22:01:02.349879+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:24:30.624743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348122+0000","last_change":"2024-05-17T22:01:01.346211+0000","last_active":"2024-05-17T22:01:02.348122+0000","last_peered":"2024-05-17T22:01:02.348122+0000","last_clean":"2024-05-17T22:01:02.348122+0000","last_became_active":"2024-05-17T22:01:01.345141+0000","last_became_peered":"2024-05-17T22:01:01.345141+0000","last_unstale":"2024-05-17T22:01:02.348122+0000","last_undegraded":"2024-05-17T22:01:02.348122+0000","last_fullsized":"2024-05-17T22:01:02.348122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:05:33.996730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458889+0000","last_change":"2024-05-17T22:01:01.394881+0000","last_active":"2024-05-17T22:01:03.458889+0000","last_peered":"2024-05-17T22:01:03.458889+0000","last_clean":"2024-05-17T22:01:03.458889+0000","last_became_active":"2024-05-17T22:01:01.394513+0000","last_became_peered":"2024-05-17T22:01:01.394513+0000","last_unstale":"2024-05-17T22:01:03.458889+0000","last_undegraded":"2024-05-17T22:01:03.458889+0000","last_fullsized":"2024-05-17T22:01:03.458889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:42:09.741553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462154+0000","last_change":"2024-05-17T22:01:01.401188+0000","last_active":"2024-05-17T22:01:03.462154+0000","last_peered":"2024-05-17T22:01:03.462154+0000","last_clean":"2024-05-17T22:01:03.462154+0000","last_became_active":"2024-05-17T22:01:01.400487+0000","last_became_peered":"2024-05-17T22:01:01.400487+0000","last_unstale":"2024-05-17T22:01:03.462154+0000","last_undegraded":"2024-05-17T22:01:03.462154+0000","last_fullsized":"2024-05-17T22:01:03.462154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:24.904719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332437+0000","last_change":"2024-05-17T22:01:01.343123+0000","last_active":"2024-05-17T22:01:03.332437+0000","last_peered":"2024-05-17T22:01:03.332437+0000","last_clean":"2024-05-17T22:01:03.332437+0000","last_became_active":"2024-05-17T22:01:01.342760+0000","last_became_peered":"2024-05-17T22:01:01.342760+0000","last_unstale":"2024-05-17T22:01:03.332437+0000","last_undegraded":"2024-05-17T22:01:03.332437+0000","last_fullsized":"2024-05-17T22:01:03.332437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:32.788833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347675+0000","last_change":"2024-05-17T22:01:01.350168+0000","last_active":"2024-05-17T22:01:02.347675+0000","last_peered":"2024-05-17T22:01:02.347675+0000","last_clean":"2024-05-17T22:01:02.347675+0000","last_became_active":"2024-05-17T22:01:01.348753+0000","last_became_peered":"2024-05-17T22:01:01.348753+0000","last_unstale":"2024-05-17T22:01:02.347675+0000","last_undegraded":"2024-05-17T22:01:02.347675+0000","last_fullsized":"2024-05-17T22:01:02.347675+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:51.193518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347753+0000","last_change":"2024-05-17T22:01:01.349951+0000","last_active":"2024-05-17T22:01:02.347753+0000","last_peered":"2024-05-17T22:01:02.347753+0000","last_clean":"2024-05-17T22:01:02.347753+0000","last_became_active":"2024-05-17T22:01:01.349091+0000","last_became_peered":"2024-05-17T22:01:01.349091+0000","last_unstale":"2024-05-17T22:01:02.347753+0000","last_undegraded":"2024-05-17T22:01:02.347753+0000","last_fullsized":"2024-05-17T22:01:02.347753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:42.859600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407887+0000","last_change":"2024-05-17T22:01:01.373818+0000","last_active":"2024-05-17T22:01:02.407887+0000","last_peered":"2024-05-17T22:01:02.407887+0000","last_clean":"2024-05-17T22:01:02.407887+0000","last_became_active":"2024-05-17T22:01:01.373064+0000","last_became_peered":"2024-05-17T22:01:01.373064+0000","last_unstale":"2024-05-17T22:01:02.407887+0000","last_undegraded":"2024-05-17T22:01:02.407887+0000","last_fullsized":"2024-05-17T22:01:02.407887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:08:52.775863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407849+0000","last_change":"2024-05-17T22:01:01.402527+0000","last_active":"2024-05-17T22:01:02.407849+0000","last_peered":"2024-05-17T22:01:02.407849+0000","last_clean":"2024-05-17T22:01:02.407849+0000","last_became_active":"2024-05-17T22:01:01.401605+0000","last_became_peered":"2024-05-17T22:01:01.401605+0000","last_unstale":"2024-05-17T22:01:02.407849+0000","last_undegraded":"2024-05-17T22:01:02.407849+0000","last_fullsized":"2024-05-17T22:01:02.407849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:27.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348930+0000","last_change":"2024-05-17T22:01:01.383379+0000","last_active":"2024-05-17T22:01:02.348930+0000","last_peered":"2024-05-17T22:01:02.348930+0000","last_clean":"2024-05-17T22:01:02.348930+0000","last_became_active":"2024-05-17T22:01:01.382541+0000","last_became_peered":"2024-05-17T22:01:01.382541+0000","last_unstale":"2024-05-17T22:01:02.348930+0000","last_undegraded":"2024-05-17T22:01:02.348930+0000","last_fullsized":"2024-05-17T22:01:02.348930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.351006+0000","last_change":"2024-05-17T22:01:01.396802+0000","last_active":"2024-05-17T22:01:02.351006+0000","last_peered":"2024-05-17T22:01:02.351006+0000","last_clean":"2024-05-17T22:01:02.351006+0000","last_became_active":"2024-05-17T22:01:01.394908+0000","last_became_peered":"2024-05-17T22:01:01.394908+0000","last_unstale":"2024-05-17T22:01:02.351006+0000","last_undegraded":"2024-05-17T22:01:02.351006+0000","last_fullsized":"2024-05-17T22:01:02.351006+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459076+0000","last_change":"2024-05-17T22:01:01.375228+0000","last_active":"2024-05-17T22:01:03.459076+0000","last_peered":"2024-05-17T22:01:03.459076+0000","last_clean":"2024-05-17T22:01:03.459076+0000","last_became_active":"2024-05-17T22:01:01.374888+0000","last_became_peered":"2024-05-17T22:01:01.374888+0000","last_unstale":"2024-05-17T22:01:03.459076+0000","last_undegraded":"2024-05-17T22:01:03.459076+0000","last_fullsized":"2024-05-17T22:01:03.459076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:41.822295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347962+0000","last_change":"2024-05-17T22:01:01.349859+0000","last_active":"2024-05-17T22:01:02.347962+0000","last_peered":"2024-05-17T22:01:02.347962+0000","last_clean":"2024-05-17T22:01:02.347962+0000","last_became_active":"2024-05-17T22:01:01.348977+0000","last_became_peered":"2024-05-17T22:01:01.348977+0000","last_unstale":"2024-05-17T22:01:02.347962+0000","last_undegraded":"2024-05-17T22:01:02.347962+0000","last_fullsized":"2024-05-17T22:01:02.347962+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:16.696964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458562+0000","last_change":"2024-05-17T22:01:01.356422+0000","last_active":"2024-05-17T22:01:03.458562+0000","last_peered":"2024-05-17T22:01:03.458562+0000","last_clean":"2024-05-17T22:01:03.458562+0000","last_became_active":"2024-05-17T22:01:01.356076+0000","last_became_peered":"2024-05-17T22:01:01.356076+0000","last_unstale":"2024-05-17T22:01:03.458562+0000","last_undegraded":"2024-05-17T22:01:03.458562+0000","last_fullsized":"2024-05-17T22:01:03.458562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:07:13.745478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348658+0000","last_change":"2024-05-17T22:01:01.389882+0000","last_active":"2024-05-17T22:01:02.348658+0000","last_peered":"2024-05-17T22:01:02.348658+0000","last_clean":"2024-05-17T22:01:02.348658+0000","last_became_active":"2024-05-17T22:01:01.389013+0000","last_became_peered":"2024-05-17T22:01:01.389013+0000","last_unstale":"2024-05-17T22:01:02.348658+0000","last_undegraded":"2024-05-17T22:01:02.348658+0000","last_fullsized":"2024-05-17T22:01:02.348658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:07.400732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349450+0000","last_change":"2024-05-17T22:01:01.336758+0000","last_active":"2024-05-17T22:01:02.349450+0000","last_peered":"2024-05-17T22:01:02.349450+0000","last_clean":"2024-05-17T22:01:02.349450+0000","last_became_active":"2024-05-17T22:01:01.335920+0000","last_became_peered":"2024-05-17T22:01:01.335920+0000","last_unstale":"2024-05-17T22:01:02.349450+0000","last_undegraded":"2024-05-17T22:01:02.349450+0000","last_fullsized":"2024-05-17T22:01:02.349450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:18:31.323279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461066+0000","last_change":"2024-05-17T22:01:01.393309+0000","last_active":"2024-05-17T22:01:03.461066+0000","last_peered":"2024-05-17T22:01:03.461066+0000","last_clean":"2024-05-17T22:01:03.461066+0000","last_became_active":"2024-05-17T22:01:01.391635+0000","last_became_peered":"2024-05-17T22:01:01.391635+0000","last_unstale":"2024-05-17T22:01:03.461066+0000","last_undegraded":"2024-05-17T22:01:03.461066+0000","last_fullsized":"2024-05-17T22:01:03.461066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:58:13.902831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408744+0000","last_change":"2024-05-17T22:01:01.402351+0000","last_active":"2024-05-17T22:01:02.408744+0000","last_peered":"2024-05-17T22:01:02.408744+0000","last_clean":"2024-05-17T22:01:02.408744+0000","last_became_active":"2024-05-17T22:01:01.401831+0000","last_became_peered":"2024-05-17T22:01:01.401831+0000","last_unstale":"2024-05-17T22:01:02.408744+0000","last_undegraded":"2024-05-17T22:01:02.408744+0000","last_fullsized":"2024-05-17T22:01:02.408744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:03.853003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.409117+0000","last_change":"2024-05-17T22:01:00.316047+0000","last_active":"2024-05-17T22:01:02.409117+0000","last_peered":"2024-05-17T22:01:02.409117+0000","last_clean":"2024-05-17T22:01:02.409117+0000","last_became_active":"2024-05-17T22:01:00.315692+0000","last_became_peered":"2024-05-17T22:01:00.315692+0000","last_unstale":"2024-05-17T22:01:02.409117+0000","last_undegraded":"2024-05-17T22:01:02.409117+0000","last_fullsized":"2024-05-17T22:01:02.409117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:23.893723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348194+0000","last_change":"2024-05-17T22:01:01.346565+0000","last_active":"2024-05-17T22:01:02.348194+0000","last_peered":"2024-05-17T22:01:02.348194+0000","last_clean":"2024-05-17T22:01:02.348194+0000","last_became_active":"2024-05-17T22:01:01.346039+0000","last_became_peered":"2024-05-17T22:01:01.346039+0000","last_unstale":"2024-05-17T22:01:02.348194+0000","last_undegraded":"2024-05-17T22:01:02.348194+0000","last_fullsized":"2024-05-17T22:01:02.348194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:40:39.239245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332766+0000","last_change":"2024-05-17T22:01:00.309435+0000","last_active":"2024-05-17T22:01:03.332766+0000","last_peered":"2024-05-17T22:01:03.332766+0000","last_clean":"2024-05-17T22:01:03.332766+0000","last_became_active":"2024-05-17T22:01:00.309112+0000","last_became_peered":"2024-05-17T22:01:00.309112+0000","last_unstale":"2024-05-17T22:01:03.332766+0000","last_undegraded":"2024-05-17T22:01:03.332766+0000","last_fullsized":"2024-05-17T22:01:03.332766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:20.802320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462011+0000","last_change":"2024-05-17T22:01:01.394528+0000","last_active":"2024-05-17T22:01:03.462011+0000","last_peered":"2024-05-17T22:01:03.462011+0000","last_clean":"2024-05-17T22:01:03.462011+0000","last_became_active":"2024-05-17T22:01:01.391743+0000","last_became_peered":"2024-05-17T22:01:01.391743+0000","last_unstale":"2024-05-17T22:01:03.462011+0000","last_undegraded":"2024-05-17T22:01:03.462011+0000","last_fullsized":"2024-05-17T22:01:03.462011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:42.859600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349585+0000","last_change":"2024-05-17T22:01:00.328114+0000","last_active":"2024-05-17T22:01:02.349585+0000","last_peered":"2024-05-17T22:01:02.349585+0000","last_clean":"2024-05-17T22:01:02.349585+0000","last_became_active":"2024-05-17T22:01:00.327381+0000","last_became_peered":"2024-05-17T22:01:00.327381+0000","last_unstale":"2024-05-17T22:01:02.349585+0000","last_undegraded":"2024-05-17T22:01:02.349585+0000","last_fullsized":"2024-05-17T22:01:02.349585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:42:08.730557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408600+0000","last_change":"2024-05-17T22:01:01.377880+0000","last_active":"2024-05-17T22:01:02.408600+0000","last_peered":"2024-05-17T22:01:02.408600+0000","last_clean":"2024-05-17T22:01:02.408600+0000","last_became_active":"2024-05-17T22:01:01.372339+0000","last_became_peered":"2024-05-17T22:01:01.372339+0000","last_unstale":"2024-05-17T22:01:02.408600+0000","last_undegraded":"2024-05-17T22:01:02.408600+0000","last_fullsized":"2024-05-17T22:01:02.408600+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:43:26.614346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462272+0000","last_change":"2024-05-17T22:01:00.310855+0000","last_active":"2024-05-17T22:01:03.462272+0000","last_peered":"2024-05-17T22:01:03.462272+0000","last_clean":"2024-05-17T22:01:03.462272+0000","last_became_active":"2024-05-17T22:01:00.310392+0000","last_became_peered":"2024-05-17T22:01:00.310392+0000","last_unstale":"2024-05-17T22:01:03.462272+0000","last_undegraded":"2024-05-17T22:01:03.462272+0000","last_fullsized":"2024-05-17T22:01:03.462272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:33:03.787565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350222+0000","last_change":"2024-05-17T22:01:01.345139+0000","last_active":"2024-05-17T22:01:02.350222+0000","last_peered":"2024-05-17T22:01:02.350222+0000","last_clean":"2024-05-17T22:01:02.350222+0000","last_became_active":"2024-05-17T22:01:01.344212+0000","last_became_peered":"2024-05-17T22:01:01.344212+0000","last_unstale":"2024-05-17T22:01:02.350222+0000","last_undegraded":"2024-05-17T22:01:02.350222+0000","last_fullsized":"2024-05-17T22:01:02.350222+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:18:11.913763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458973+0000","last_change":"2024-05-17T22:01:00.325464+0000","last_active":"2024-05-17T22:01:03.458973+0000","last_peered":"2024-05-17T22:01:03.458973+0000","last_clean":"2024-05-17T22:01:03.458973+0000","last_became_active":"2024-05-17T22:01:00.325104+0000","last_became_peered":"2024-05-17T22:01:00.325104+0000","last_unstale":"2024-05-17T22:01:03.458973+0000","last_undegraded":"2024-05-17T22:01:03.458973+0000","last_fullsized":"2024-05-17T22:01:03.458973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:23:01.026719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348119+0000","last_change":"2024-05-17T22:01:01.374133+0000","last_active":"2024-05-17T22:01:02.348119+0000","last_peered":"2024-05-17T22:01:02.348119+0000","last_clean":"2024-05-17T22:01:02.348119+0000","last_became_active":"2024-05-17T22:01:01.373225+0000","last_became_peered":"2024-05-17T22:01:01.373225+0000","last_unstale":"2024-05-17T22:01:02.348119+0000","last_undegraded":"2024-05-17T22:01:02.348119+0000","last_fullsized":"2024-05-17T22:01:02.348119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:24.904719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461223+0000","last_change":"2024-05-17T22:01:00.329515+0000","last_active":"2024-05-17T22:01:03.461223+0000","last_peered":"2024-05-17T22:01:03.461223+0000","last_clean":"2024-05-17T22:01:03.461223+0000","last_became_active":"2024-05-17T22:01:00.328555+0000","last_became_peered":"2024-05-17T22:01:00.328555+0000","last_unstale":"2024-05-17T22:01:03.461223+0000","last_undegraded":"2024-05-17T22:01:03.461223+0000","last_fullsized":"2024-05-17T22:01:03.461223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:33.676626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347401+0000","last_change":"2024-05-17T22:01:01.361007+0000","last_active":"2024-05-17T22:01:02.347401+0000","last_peered":"2024-05-17T22:01:02.347401+0000","last_clean":"2024-05-17T22:01:02.347401+0000","last_became_active":"2024-05-17T22:01:01.360759+0000","last_became_peered":"2024-05-17T22:01:01.360759+0000","last_unstale":"2024-05-17T22:01:02.347401+0000","last_undegraded":"2024-05-17T22:01:02.347401+0000","last_fullsized":"2024-05-17T22:01:02.347401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:27.118303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459346+0000","last_change":"2024-05-17T22:01:00.328555+0000","last_active":"2024-05-17T22:01:03.459346+0000","last_peered":"2024-05-17T22:01:03.459346+0000","last_clean":"2024-05-17T22:01:03.459346+0000","last_became_active":"2024-05-17T22:01:00.328236+0000","last_became_peered":"2024-05-17T22:01:00.328236+0000","last_unstale":"2024-05-17T22:01:03.459346+0000","last_undegraded":"2024-05-17T22:01:03.459346+0000","last_fullsized":"2024-05-17T22:01:03.459346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:22:36.050481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332175+0000","last_change":"2024-05-17T22:01:01.362035+0000","last_active":"2024-05-17T22:01:03.332175+0000","last_peered":"2024-05-17T22:01:03.332175+0000","last_clean":"2024-05-17T22:01:03.332175+0000","last_became_active":"2024-05-17T22:01:01.361753+0000","last_became_peered":"2024-05-17T22:01:01.361753+0000","last_unstale":"2024-05-17T22:01:03.332175+0000","last_undegraded":"2024-05-17T22:01:03.332175+0000","last_fullsized":"2024-05-17T22:01:03.332175+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:18:11.913763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347357+0000","last_change":"2024-05-17T22:01:00.306375+0000","last_active":"2024-05-17T22:01:02.347357+0000","last_peered":"2024-05-17T22:01:02.347357+0000","last_clean":"2024-05-17T22:01:02.347357+0000","last_became_active":"2024-05-17T22:01:00.305886+0000","last_became_peered":"2024-05-17T22:01:00.305886+0000","last_unstale":"2024-05-17T22:01:02.347357+0000","last_undegraded":"2024-05-17T22:01:02.347357+0000","last_fullsized":"2024-05-17T22:01:02.347357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:11:27.011576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.351165+0000","last_change":"2024-05-17T22:01:01.366828+0000","last_active":"2024-05-17T22:01:02.351165+0000","last_peered":"2024-05-17T22:01:02.351165+0000","last_clean":"2024-05-17T22:01:02.351165+0000","last_became_active":"2024-05-17T22:01:01.365965+0000","last_became_peered":"2024-05-17T22:01:01.365965+0000","last_unstale":"2024-05-17T22:01:02.351165+0000","last_undegraded":"2024-05-17T22:01:02.351165+0000","last_fullsized":"2024-05-17T22:01:02.351165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:27.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349200+0000","last_change":"2024-05-17T22:01:00.327739+0000","last_active":"2024-05-17T22:01:02.349200+0000","last_peered":"2024-05-17T22:01:02.349200+0000","last_clean":"2024-05-17T22:01:02.349200+0000","last_became_active":"2024-05-17T22:01:00.327381+0000","last_became_peered":"2024-05-17T22:01:00.327381+0000","last_unstale":"2024-05-17T22:01:02.349200+0000","last_undegraded":"2024-05-17T22:01:02.349200+0000","last_fullsized":"2024-05-17T22:01:02.349200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:48.825530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408944+0000","last_change":"2024-05-17T22:01:01.403914+0000","last_active":"2024-05-17T22:01:02.408944+0000","last_peered":"2024-05-17T22:01:02.408944+0000","last_clean":"2024-05-17T22:01:02.408944+0000","last_became_active":"2024-05-17T22:01:01.403248+0000","last_became_peered":"2024-05-17T22:01:01.403248+0000","last_unstale":"2024-05-17T22:01:02.408944+0000","last_undegraded":"2024-05-17T22:01:02.408944+0000","last_fullsized":"2024-05-17T22:01:02.408944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:40:54.523318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461873+0000","last_change":"2024-05-17T22:01:00.331079+0000","last_active":"2024-05-17T22:01:03.461873+0000","last_peered":"2024-05-17T22:01:03.461873+0000","last_clean":"2024-05-17T22:01:03.461873+0000","last_became_active":"2024-05-17T22:01:00.330509+0000","last_became_peered":"2024-05-17T22:01:00.330509+0000","last_unstale":"2024-05-17T22:01:03.461873+0000","last_undegraded":"2024-05-17T22:01:03.461873+0000","last_fullsized":"2024-05-17T22:01:03.461873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:00.740087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462098+0000","last_change":"2024-05-17T22:01:01.397260+0000","last_active":"2024-05-17T22:01:03.462098+0000","last_peered":"2024-05-17T22:01:03.462098+0000","last_clean":"2024-05-17T22:01:03.462098+0000","last_became_active":"2024-05-17T22:01:01.393027+0000","last_became_peered":"2024-05-17T22:01:01.393027+0000","last_unstale":"2024-05-17T22:01:03.462098+0000","last_undegraded":"2024-05-17T22:01:03.462098+0000","last_fullsized":"2024-05-17T22:01:03.462098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:16.696964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459173+0000","last_change":"2024-05-17T22:01:00.314472+0000","last_active":"2024-05-17T22:01:03.459173+0000","last_peered":"2024-05-17T22:01:03.459173+0000","last_clean":"2024-05-17T22:01:03.459173+0000","last_became_active":"2024-05-17T22:01:00.314073+0000","last_became_peered":"2024-05-17T22:01:00.314073+0000","last_unstale":"2024-05-17T22:01:03.459173+0000","last_undegraded":"2024-05-17T22:01:03.459173+0000","last_fullsized":"2024-05-17T22:01:03.459173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:02.842007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347708+0000","last_change":"2024-05-17T22:01:01.343893+0000","last_active":"2024-05-17T22:01:02.347708+0000","last_peered":"2024-05-17T22:01:02.347708+0000","last_clean":"2024-05-17T22:01:02.347708+0000","last_became_active":"2024-05-17T22:01:01.343477+0000","last_became_peered":"2024-05-17T22:01:01.343477+0000","last_unstale":"2024-05-17T22:01:02.347708+0000","last_undegraded":"2024-05-17T22:01:02.347708+0000","last_fullsized":"2024-05-17T22:01:02.347708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:42.607139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459517+0000","last_change":"2024-05-17T22:01:00.309580+0000","last_active":"2024-05-17T22:01:03.459517+0000","last_peered":"2024-05-17T22:01:03.459517+0000","last_clean":"2024-05-17T22:01:03.459517+0000","last_became_active":"2024-05-17T22:01:00.309263+0000","last_became_peered":"2024-05-17T22:01:00.309263+0000","last_unstale":"2024-05-17T22:01:03.459517+0000","last_undegraded":"2024-05-17T22:01:03.459517+0000","last_fullsized":"2024-05-17T22:01:03.459517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:29.966233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407644+0000","last_change":"2024-05-17T22:01:01.368634+0000","last_active":"2024-05-17T22:01:02.407644+0000","last_peered":"2024-05-17T22:01:02.407644+0000","last_clean":"2024-05-17T22:01:02.407644+0000","last_became_active":"2024-05-17T22:01:01.367805+0000","last_became_peered":"2024-05-17T22:01:01.367805+0000","last_unstale":"2024-05-17T22:01:02.407644+0000","last_undegraded":"2024-05-17T22:01:02.407644+0000","last_fullsized":"2024-05-17T22:01:02.407644+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458548+0000","last_change":"2024-05-17T22:01:00.307058+0000","last_active":"2024-05-17T22:01:03.458548+0000","last_peered":"2024-05-17T22:01:03.458548+0000","last_clean":"2024-05-17T22:01:03.458548+0000","last_became_active":"2024-05-17T22:01:00.306788+0000","last_became_peered":"2024-05-17T22:01:00.306788+0000","last_unstale":"2024-05-17T22:01:03.458548+0000","last_undegraded":"2024-05-17T22:01:03.458548+0000","last_fullsized":"2024-05-17T22:01:03.458548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:33.676626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350583+0000","last_change":"2024-05-17T22:01:01.348019+0000","last_active":"2024-05-17T22:01:02.350583+0000","last_peered":"2024-05-17T22:01:02.350583+0000","last_clean":"2024-05-17T22:01:02.350583+0000","last_became_active":"2024-05-17T22:01:01.347176+0000","last_became_peered":"2024-05-17T22:01:01.347176+0000","last_unstale":"2024-05-17T22:01:02.350583+0000","last_undegraded":"2024-05-17T22:01:02.350583+0000","last_fullsized":"2024-05-17T22:01:02.350583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:35:36.659116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461569+0000","last_change":"2024-05-17T22:01:00.323505+0000","last_active":"2024-05-17T22:01:03.461569+0000","last_peered":"2024-05-17T22:01:03.461569+0000","last_clean":"2024-05-17T22:01:03.461569+0000","last_became_active":"2024-05-17T22:01:00.322503+0000","last_became_peered":"2024-05-17T22:01:00.322503+0000","last_unstale":"2024-05-17T22:01:03.461569+0000","last_undegraded":"2024-05-17T22:01:03.461569+0000","last_fullsized":"2024-05-17T22:01:03.461569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:04:00.933298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331844+0000","last_change":"2024-05-17T22:01:01.401267+0000","last_active":"2024-05-17T22:01:03.331844+0000","last_peered":"2024-05-17T22:01:03.331844+0000","last_clean":"2024-05-17T22:01:03.331844+0000","last_became_active":"2024-05-17T22:01:01.400933+0000","last_became_peered":"2024-05-17T22:01:01.400933+0000","last_unstale":"2024-05-17T22:01:03.331844+0000","last_undegraded":"2024-05-17T22:01:03.331844+0000","last_fullsized":"2024-05-17T22:01:03.331844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:45.219441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349132+0000","last_change":"2024-05-17T22:01:00.315139+0000","last_active":"2024-05-17T22:01:02.349132+0000","last_peered":"2024-05-17T22:01:02.349132+0000","last_clean":"2024-05-17T22:01:02.349132+0000","last_became_active":"2024-05-17T22:01:00.314569+0000","last_became_peered":"2024-05-17T22:01:00.314569+0000","last_unstale":"2024-05-17T22:01:02.349132+0000","last_undegraded":"2024-05-17T22:01:02.349132+0000","last_fullsized":"2024-05-17T22:01:02.349132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:45:01.337777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347535+0000","last_change":"2024-05-17T22:01:01.346073+0000","last_active":"2024-05-17T22:01:02.347535+0000","last_peered":"2024-05-17T22:01:02.347535+0000","last_clean":"2024-05-17T22:01:02.347535+0000","last_became_active":"2024-05-17T22:01:01.345062+0000","last_became_peered":"2024-05-17T22:01:01.345062+0000","last_unstale":"2024-05-17T22:01:02.347535+0000","last_undegraded":"2024-05-17T22:01:02.347535+0000","last_fullsized":"2024-05-17T22:01:02.347535+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:37.799390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347445+0000","last_change":"2024-05-17T22:01:00.328366+0000","last_active":"2024-05-17T22:01:02.347445+0000","last_peered":"2024-05-17T22:01:02.347445+0000","last_clean":"2024-05-17T22:01:02.347445+0000","last_became_active":"2024-05-17T22:01:00.327870+0000","last_became_peered":"2024-05-17T22:01:00.327870+0000","last_unstale":"2024-05-17T22:01:02.347445+0000","last_undegraded":"2024-05-17T22:01:02.347445+0000","last_fullsized":"2024-05-17T22:01:02.347445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:19.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349297+0000","last_change":"2024-05-17T22:01:01.350254+0000","last_active":"2024-05-17T22:01:02.349297+0000","last_peered":"2024-05-17T22:01:02.349297+0000","last_clean":"2024-05-17T22:01:02.349297+0000","last_became_active":"2024-05-17T22:01:01.349361+0000","last_became_peered":"2024-05-17T22:01:01.349361+0000","last_unstale":"2024-05-17T22:01:02.349297+0000","last_undegraded":"2024-05-17T22:01:02.349297+0000","last_fullsized":"2024-05-17T22:01:02.349297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:13:52.122119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461816+0000","last_change":"2024-05-17T22:01:00.321403+0000","last_active":"2024-05-17T22:01:03.461816+0000","last_peered":"2024-05-17T22:01:03.461816+0000","last_clean":"2024-05-17T22:01:03.461816+0000","last_became_active":"2024-05-17T22:01:00.320624+0000","last_became_peered":"2024-05-17T22:01:00.320624+0000","last_unstale":"2024-05-17T22:01:03.461816+0000","last_undegraded":"2024-05-17T22:01:03.461816+0000","last_fullsized":"2024-05-17T22:01:03.461816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:22:36.050481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.409112+0000","last_change":"2024-05-17T22:01:01.385066+0000","last_active":"2024-05-17T22:01:02.409112+0000","last_peered":"2024-05-17T22:01:02.409112+0000","last_clean":"2024-05-17T22:01:02.409112+0000","last_became_active":"2024-05-17T22:01:01.382925+0000","last_became_peered":"2024-05-17T22:01:01.382925+0000","last_unstale":"2024-05-17T22:01:02.409112+0000","last_undegraded":"2024-05-17T22:01:02.409112+0000","last_fullsized":"2024-05-17T22:01:02.409112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:17.554044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349469+0000","last_change":"2024-05-17T22:01:00.329592+0000","last_active":"2024-05-17T22:01:02.349469+0000","last_peered":"2024-05-17T22:01:02.349469+0000","last_clean":"2024-05-17T22:01:02.349469+0000","last_became_active":"2024-05-17T22:01:00.329099+0000","last_became_peered":"2024-05-17T22:01:00.329099+0000","last_unstale":"2024-05-17T22:01:02.349469+0000","last_undegraded":"2024-05-17T22:01:02.349469+0000","last_fullsized":"2024-05-17T22:01:02.349469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:33.113015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459455+0000","last_change":"2024-05-17T22:01:01.335326+0000","last_active":"2024-05-17T22:01:03.459455+0000","last_peered":"2024-05-17T22:01:03.459455+0000","last_clean":"2024-05-17T22:01:03.459455+0000","last_became_active":"2024-05-17T22:01:01.334588+0000","last_became_peered":"2024-05-17T22:01:01.334588+0000","last_unstale":"2024-05-17T22:01:03.459455+0000","last_undegraded":"2024-05-17T22:01:03.459455+0000","last_fullsized":"2024-05-17T22:01:03.459455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:21:39.294671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462099+0000","last_change":"2024-05-17T22:01:00.332474+0000","last_active":"2024-05-17T22:01:03.462099+0000","last_peered":"2024-05-17T22:01:03.462099+0000","last_clean":"2024-05-17T22:01:03.462099+0000","last_became_active":"2024-05-17T22:01:00.331949+0000","last_became_peered":"2024-05-17T22:01:00.331949+0000","last_unstale":"2024-05-17T22:01:03.462099+0000","last_undegraded":"2024-05-17T22:01:03.462099+0000","last_fullsized":"2024-05-17T22:01:03.462099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:50.714168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347754+0000","last_change":"2024-05-17T22:01:01.347350+0000","last_active":"2024-05-17T22:01:02.347754+0000","last_peered":"2024-05-17T22:01:02.347754+0000","last_clean":"2024-05-17T22:01:02.347754+0000","last_became_active":"2024-05-17T22:01:01.347032+0000","last_became_peered":"2024-05-17T22:01:01.347032+0000","last_unstale":"2024-05-17T22:01:02.347754+0000","last_undegraded":"2024-05-17T22:01:02.347754+0000","last_fullsized":"2024-05-17T22:01:02.347754+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:11:16.386285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408556+0000","last_change":"2024-05-17T22:01:00.327376+0000","last_active":"2024-05-17T22:01:02.408556+0000","last_peered":"2024-05-17T22:01:02.408556+0000","last_clean":"2024-05-17T22:01:02.408556+0000","last_became_active":"2024-05-17T22:01:00.326887+0000","last_became_peered":"2024-05-17T22:01:00.326887+0000","last_unstale":"2024-05-17T22:01:02.408556+0000","last_undegraded":"2024-05-17T22:01:02.408556+0000","last_fullsized":"2024-05-17T22:01:02.408556+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:15.685968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458806+0000","last_change":"2024-05-17T22:01:01.347594+0000","last_active":"2024-05-17T22:01:03.458806+0000","last_peered":"2024-05-17T22:01:03.458806+0000","last_clean":"2024-05-17T22:01:03.458806+0000","last_became_active":"2024-05-17T22:01:01.347271+0000","last_became_peered":"2024-05-17T22:01:01.347271+0000","last_unstale":"2024-05-17T22:01:03.458806+0000","last_undegraded":"2024-05-17T22:01:03.458806+0000","last_fullsized":"2024-05-17T22:01:03.458806+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:44.857791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461763+0000","last_change":"2024-05-17T22:01:00.323760+0000","last_active":"2024-05-17T22:01:03.461763+0000","last_peered":"2024-05-17T22:01:03.461763+0000","last_clean":"2024-05-17T22:01:03.461763+0000","last_became_active":"2024-05-17T22:01:00.319194+0000","last_became_peered":"2024-05-17T22:01:00.319194+0000","last_unstale":"2024-05-17T22:01:03.461763+0000","last_undegraded":"2024-05-17T22:01:03.461763+0000","last_fullsized":"2024-05-17T22:01:03.461763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:29.966233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458661+0000","last_change":"2024-05-17T22:01:01.392920+0000","last_active":"2024-05-17T22:01:03.458661+0000","last_peered":"2024-05-17T22:01:03.458661+0000","last_clean":"2024-05-17T22:01:03.458661+0000","last_became_active":"2024-05-17T22:01:01.392494+0000","last_became_peered":"2024-05-17T22:01:01.392494+0000","last_unstale":"2024-05-17T22:01:03.458661+0000","last_undegraded":"2024-05-17T22:01:03.458661+0000","last_fullsized":"2024-05-17T22:01:03.458661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:26.754616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349603+0000","last_change":"2024-05-17T22:01:00.313840+0000","last_active":"2024-05-17T22:01:02.349603+0000","last_peered":"2024-05-17T22:01:02.349603+0000","last_clean":"2024-05-17T22:01:02.349603+0000","last_became_active":"2024-05-17T22:01:00.313278+0000","last_became_peered":"2024-05-17T22:01:00.313278+0000","last_unstale":"2024-05-17T22:01:02.349603+0000","last_undegraded":"2024-05-17T22:01:02.349603+0000","last_fullsized":"2024-05-17T22:01:02.349603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:50.182522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458491+0000","last_change":"2024-05-17T22:01:01.392990+0000","last_active":"2024-05-17T22:01:03.458491+0000","last_peered":"2024-05-17T22:01:03.458491+0000","last_clean":"2024-05-17T22:01:03.458491+0000","last_became_active":"2024-05-17T22:01:01.392200+0000","last_became_peered":"2024-05-17T22:01:01.392200+0000","last_unstale":"2024-05-17T22:01:03.458491+0000","last_undegraded":"2024-05-17T22:01:03.458491+0000","last_fullsized":"2024-05-17T22:01:03.458491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:20.238581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331808+0000","last_change":"2024-05-17T22:01:00.309668+0000","last_active":"2024-05-17T22:01:03.331808+0000","last_peered":"2024-05-17T22:01:03.331808+0000","last_clean":"2024-05-17T22:01:03.331808+0000","last_became_active":"2024-05-17T22:01:00.309344+0000","last_became_peered":"2024-05-17T22:01:00.309344+0000","last_unstale":"2024-05-17T22:01:03.331808+0000","last_undegraded":"2024-05-17T22:01:03.331808+0000","last_fullsized":"2024-05-17T22:01:03.331808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:15.685968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332494+0000","last_change":"2024-05-17T22:01:01.339126+0000","last_active":"2024-05-17T22:01:03.332494+0000","last_peered":"2024-05-17T22:01:03.332494+0000","last_clean":"2024-05-17T22:01:03.332494+0000","last_became_active":"2024-05-17T22:01:01.338816+0000","last_became_peered":"2024-05-17T22:01:01.338816+0000","last_unstale":"2024-05-17T22:01:03.332494+0000","last_undegraded":"2024-05-17T22:01:03.332494+0000","last_fullsized":"2024-05-17T22:01:03.332494+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461527+0000","last_change":"2024-05-17T22:01:00.318933+0000","last_active":"2024-05-17T22:01:03.461527+0000","last_peered":"2024-05-17T22:01:03.461527+0000","last_clean":"2024-05-17T22:01:03.461527+0000","last_became_active":"2024-05-17T22:01:00.311013+0000","last_became_peered":"2024-05-17T22:01:00.311013+0000","last_unstale":"2024-05-17T22:01:03.461527+0000","last_undegraded":"2024-05-17T22:01:03.461527+0000","last_fullsized":"2024-05-17T22:01:03.461527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:40:07.783952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350940+0000","last_change":"2024-05-17T22:01:01.335713+0000","last_active":"2024-05-17T22:01:02.350940+0000","last_peered":"2024-05-17T22:01:02.350940+0000","last_clean":"2024-05-17T22:01:02.350940+0000","last_became_active":"2024-05-17T22:01:01.334908+0000","last_became_peered":"2024-05-17T22:01:01.334908+0000","last_unstale":"2024-05-17T22:01:02.350940+0000","last_undegraded":"2024-05-17T22:01:02.350940+0000","last_fullsized":"2024-05-17T22:01:02.350940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:32.788833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.330561+0000","last_change":"2024-05-17T22:01:00.319773+0000","last_active":"2024-05-17T22:01:03.330561+0000","last_peered":"2024-05-17T22:01:03.330561+0000","last_clean":"2024-05-17T22:01:03.330561+0000","last_became_active":"2024-05-17T22:01:00.313371+0000","last_became_peered":"2024-05-17T22:01:00.313371+0000","last_unstale":"2024-05-17T22:01:03.330561+0000","last_undegraded":"2024-05-17T22:01:03.330561+0000","last_fullsized":"2024-05-17T22:01:03.330561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:41.848604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408578+0000","last_change":"2024-05-17T22:01:01.397051+0000","last_active":"2024-05-17T22:01:02.408578+0000","last_peered":"2024-05-17T22:01:02.408578+0000","last_clean":"2024-05-17T22:01:02.408578+0000","last_became_active":"2024-05-17T22:01:01.396030+0000","last_became_peered":"2024-05-17T22:01:01.396030+0000","last_unstale":"2024-05-17T22:01:02.408578+0000","last_undegraded":"2024-05-17T22:01:02.408578+0000","last_fullsized":"2024-05-17T22:01:02.408578+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:46:16.984756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347944+0000","last_change":"2024-05-17T22:01:00.309343+0000","last_active":"2024-05-17T22:01:02.347944+0000","last_peered":"2024-05-17T22:01:02.347944+0000","last_clean":"2024-05-17T22:01:02.347944+0000","last_became_active":"2024-05-17T22:01:00.309062+0000","last_became_peered":"2024-05-17T22:01:00.309062+0000","last_unstale":"2024-05-17T22:01:02.347944+0000","last_undegraded":"2024-05-17T22:01:02.347944+0000","last_fullsized":"2024-05-17T22:01:02.347944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:19:16.312055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408667+0000","last_change":"2024-05-17T22:01:01.378631+0000","last_active":"2024-05-17T22:01:02.408667+0000","last_peered":"2024-05-17T22:01:02.408667+0000","last_clean":"2024-05-17T22:01:02.408667+0000","last_became_active":"2024-05-17T22:01:01.373285+0000","last_became_peered":"2024-05-17T22:01:01.373285+0000","last_unstale":"2024-05-17T22:01:02.408667+0000","last_undegraded":"2024-05-17T22:01:02.408667+0000","last_fullsized":"2024-05-17T22:01:02.408667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:23:02.037715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332164+0000","last_change":"2024-05-17T22:01:00.325477+0000","last_active":"2024-05-17T22:01:03.332164+0000","last_peered":"2024-05-17T22:01:03.332164+0000","last_clean":"2024-05-17T22:01:03.332164+0000","last_became_active":"2024-05-17T22:01:00.325208+0000","last_became_peered":"2024-05-17T22:01:00.325208+0000","last_unstale":"2024-05-17T22:01:03.332164+0000","last_undegraded":"2024-05-17T22:01:03.332164+0000","last_fullsized":"2024-05-17T22:01:03.332164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:23.893723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347802+0000","last_change":"2024-05-17T22:01:01.347590+0000","last_active":"2024-05-17T22:01:02.347802+0000","last_peered":"2024-05-17T22:01:02.347802+0000","last_clean":"2024-05-17T22:01:02.347802+0000","last_became_active":"2024-05-17T22:01:01.347239+0000","last_became_peered":"2024-05-17T22:01:01.347239+0000","last_unstale":"2024-05-17T22:01:02.347802+0000","last_undegraded":"2024-05-17T22:01:02.347802+0000","last_fullsized":"2024-05-17T22:01:02.347802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:50:29.242277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347564+0000","last_change":"2024-05-17T22:01:00.314570+0000","last_active":"2024-05-17T22:01:02.347564+0000","last_peered":"2024-05-17T22:01:02.347564+0000","last_clean":"2024-05-17T22:01:02.347564+0000","last_became_active":"2024-05-17T22:01:00.314145+0000","last_became_peered":"2024-05-17T22:01:00.314145+0000","last_unstale":"2024-05-17T22:01:02.347564+0000","last_undegraded":"2024-05-17T22:01:02.347564+0000","last_fullsized":"2024-05-17T22:01:02.347564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:47:21.456793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461025+0000","last_change":"2024-05-17T22:01:01.394727+0000","last_active":"2024-05-17T22:01:03.461025+0000","last_peered":"2024-05-17T22:01:03.461025+0000","last_clean":"2024-05-17T22:01:03.461025+0000","last_became_active":"2024-05-17T22:01:01.391950+0000","last_became_peered":"2024-05-17T22:01:01.391950+0000","last_unstale":"2024-05-17T22:01:03.461025+0000","last_undegraded":"2024-05-17T22:01:03.461025+0000","last_fullsized":"2024-05-17T22:01:03.461025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:31:35.092742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458168+0000","last_change":"2024-05-17T22:01:00.327489+0000","last_active":"2024-05-17T22:01:03.458168+0000","last_peered":"2024-05-17T22:01:03.458168+0000","last_clean":"2024-05-17T22:01:03.458168+0000","last_became_active":"2024-05-17T22:01:00.327226+0000","last_became_peered":"2024-05-17T22:01:00.327226+0000","last_unstale":"2024-05-17T22:01:03.458168+0000","last_undegraded":"2024-05-17T22:01:03.458168+0000","last_fullsized":"2024-05-17T22:01:03.458168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:40:07.783952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458191+0000","last_change":"2024-05-17T22:01:01.341387+0000","last_active":"2024-05-17T22:01:03.458191+0000","last_peered":"2024-05-17T22:01:03.458191+0000","last_clean":"2024-05-17T22:01:03.458191+0000","last_became_active":"2024-05-17T22:01:01.341027+0000","last_became_peered":"2024-05-17T22:01:01.341027+0000","last_unstale":"2024-05-17T22:01:03.458191+0000","last_undegraded":"2024-05-17T22:01:03.458191+0000","last_fullsized":"2024-05-17T22:01:03.458191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:20.272519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348240+0000","last_change":"2024-05-17T22:01:00.325512+0000","last_active":"2024-05-17T22:01:02.348240+0000","last_peered":"2024-05-17T22:01:02.348240+0000","last_clean":"2024-05-17T22:01:02.348240+0000","last_became_active":"2024-05-17T22:01:00.325096+0000","last_became_peered":"2024-05-17T22:01:00.325096+0000","last_unstale":"2024-05-17T22:01:02.348240+0000","last_undegraded":"2024-05-17T22:01:02.348240+0000","last_fullsized":"2024-05-17T22:01:02.348240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:29:50.049155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.460973+0000","last_change":"2024-05-17T22:01:01.391883+0000","last_active":"2024-05-17T22:01:03.460973+0000","last_peered":"2024-05-17T22:01:03.460973+0000","last_clean":"2024-05-17T22:01:03.460973+0000","last_became_active":"2024-05-17T22:01:01.388362+0000","last_became_peered":"2024-05-17T22:01:01.388362+0000","last_unstale":"2024-05-17T22:01:03.460973+0000","last_undegraded":"2024-05-17T22:01:03.460973+0000","last_fullsized":"2024-05-17T22:01:03.460973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:31:09.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461369+0000","last_change":"2024-05-17T22:01:00.319014+0000","last_active":"2024-05-17T22:01:03.461369+0000","last_peered":"2024-05-17T22:01:03.461369+0000","last_clean":"2024-05-17T22:01:03.461369+0000","last_became_active":"2024-05-17T22:01:00.311177+0000","last_became_peered":"2024-05-17T22:01:00.311177+0000","last_unstale":"2024-05-17T22:01:03.461369+0000","last_undegraded":"2024-05-17T22:01:03.461369+0000","last_fullsized":"2024-05-17T22:01:03.461369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:29:50.049155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459270+0000","last_change":"2024-05-17T22:01:01.388434+0000","last_active":"2024-05-17T22:01:03.459270+0000","last_peered":"2024-05-17T22:01:03.459270+0000","last_clean":"2024-05-17T22:01:03.459270+0000","last_became_active":"2024-05-17T22:01:01.388045+0000","last_became_peered":"2024-05-17T22:01:01.388045+0000","last_unstale":"2024-05-17T22:01:03.459270+0000","last_undegraded":"2024-05-17T22:01:03.459270+0000","last_fullsized":"2024-05-17T22:01:03.459270+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:36.694401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461079+0000","last_change":"2024-05-17T22:01:00.319680+0000","last_active":"2024-05-17T22:01:03.461079+0000","last_peered":"2024-05-17T22:01:03.461079+0000","last_clean":"2024-05-17T22:01:03.461079+0000","last_became_active":"2024-05-17T22:01:00.311337+0000","last_became_peered":"2024-05-17T22:01:00.311337+0000","last_unstale":"2024-05-17T22:01:03.461079+0000","last_undegraded":"2024-05-17T22:01:03.461079+0000","last_fullsized":"2024-05-17T22:01:03.461079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:07:54.738653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":194,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407834+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:01:02.407834+0000","last_peered":"2024-05-17T22:01:02.407834+0000","last_clean":"2024-05-17T22:01:02.407834+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:01:02.407834+0000","last_undegraded":"2024-05-17T22:01:02.407834+0000","last_fullsized":"2024-05-17T22:01:02.407834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1417216,"data_stored":932167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":341275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":334995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":827392,"data_stored":342795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":835584,"data_stored":343411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":931551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:01:05.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:01:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.644+0000 7fb5216b2640 1 Processor -- start 2024-05-17T22:01:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.644+0000 7fb5216b2640 1 -- start start 2024-05-17T22:01:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb5216b2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb51c1461c0 0x7fb51c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb5216b2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb5216b2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb51c1465d0 con 0x7fb51c070510 2024-05-17T22:01:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb5216b2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb51c06ef30 con 0x7fb51c1461c0 2024-05-17T22:01:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb5216b2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c06f210 con 0x7fb51c15e6b0 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:34258/0 (socket says 172.21.15.83:34258) 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 -- 172.21.15.83:0/304030544 learned_addr learned my addr 172.21.15.83:0/304030544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51affd640 1 --2- 172.21.15.83:0/304030544 >> v2:172.21.15.155:3300/0 conn(0x7fb51c1461c0 0x7fb51c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 -- 172.21.15.83:0/304030544 >> v2:172.21.15.155:3300/0 conn(0x7fb51c1461c0 msgr2=0x7fb51c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 --2- 172.21.15.83:0/304030544 >> v2:172.21.15.155:3300/0 conn(0x7fb51c1461c0 0x7fb51c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 -- 172.21.15.83:0/304030544 >> v1:172.21.15.155:6789/0 conn(0x7fb51c070510 legacy=0x7fb51c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.648+0000 7fb51a7fc640 1 -- 172.21.15.83:0/304030544 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c070920 con 0x7fb51c15e6b0 2024-05-17T22:01:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb51a7fc640 1 --2- 172.21.15.83:0/304030544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb510009870 tx=0x7fb51001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57765a66d8a4c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb519ffb640 1 -- 172.21.15.83:0/304030544 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100a95f0 con 0x7fb51c15e6b0 2024-05-17T22:01:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb519ffb640 1 -- 172.21.15.83:0/304030544 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5100a9790 con 0x7fb51c15e6b0 2024-05-17T22:01:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb519ffb640 1 -- 172.21.15.83:0/304030544 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100a9b60 con 0x7fb51c15e6b0 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 -- 172.21.15.83:0/304030544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 msgr2=0x7fb51c06ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 --2- 172.21.15.83:0/304030544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb510009870 tx=0x7fb51001c530 comp rx=0 tx=0).stop 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 -- 172.21.15.83:0/304030544 shutdown_connections 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 --2- 172.21.15.83:0/304030544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c06ffd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 --2- 172.21.15.83:0/304030544 >> v2:172.21.15.155:3300/0 conn(0x7fb51c1461c0 0x7fb51c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 -- 172.21.15.83:0/304030544 >> 172.21.15.83:0/304030544 conn(0x7fb51c06cd90 msgr2=0x7fb51c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 -- 172.21.15.83:0/304030544 shutdown_connections 2024-05-17T22:01:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.652+0000 7fb5216b2640 1 -- 172.21.15.83:0/304030544 wait complete. 2024-05-17T22:01:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 Processor -- start 2024-05-17T22:01:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- start start 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb51c1461c0 0x7fb51c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c160e30 con 0x7fb51c070510 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c160cb0 con 0x7fb51c1461c0 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c0d10a0 con 0x7fb51c15e6b0 2024-05-17T22:01:05.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50082/0 (socket says 172.21.15.83:50082) 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 -- 172.21.15.83:0/3532697751 learned_addr learned my addr 172.21.15.83:0/3532697751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51a7fc640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb51c1461c0 0x7fb51c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51b7fe640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 -- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 msgr2=0x7fb51c0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c0d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 -- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb51c1461c0 msgr2=0x7fb51c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb51c1461c0 0x7fb51c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 -- 172.21.15.83:0/3532697751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c0b4b10 con 0x7fb51c070510 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb51affd640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e840 tx=0x7fb50c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c017070 con 0x7fb51c070510 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c00ee90 con 0x7fb51c070510 2024-05-17T22:01:05.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c0b5020 con 0x7fb51c070510 2024-05-17T22:01:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00cb60 con 0x7fb51c070510 2024-05-17T22:01:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c0b55e0 con 0x7fb51c070510 2024-05-17T22:01:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.656+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51c15e170 con 0x7fb51c070510 2024-05-17T22:01:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.660+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb50c01a070 con 0x7fb51c070510 2024-05-17T22:01:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.660+0000 7fb5037fe640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 0x7fb508071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.660+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fb50c08f0d0 con 0x7fb51c070510 2024-05-17T22:01:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.660+0000 7fb51a7fc640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 0x7fb508071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.660+0000 7fb51a7fc640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 0x7fb508071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb5100ae000 tx=0x7fb510009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.664+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb50c05c050 con 0x7fb51c070510 2024-05-17T22:01:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.808+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb51c0b41a0 con 0x7fb50806f400 2024-05-17T22:01:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.812+0000 7fb5037fe640 1 -- 172.21.15.83:0/3532697751 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fb51c0b41a0 con 0x7fb50806f400 2024-05-17T22:01:05.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:05.809 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:01:05.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 msgr2=0x7fb508071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 0x7fb508071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb5100ae000 tx=0x7fb510009040 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 msgr2=0x7fb51c0b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e840 tx=0x7fb50c00ec40 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 shutdown_connections 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb51c15e6b0 0x7fb51c0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb50806f400 0x7fb508071910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb51c1461c0 0x7fb51c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 --2- 172.21.15.83:0/3532697751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb51c070510 0x7fb51c0b45d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 >> 172.21.15.83:0/3532697751 conn(0x7fb51c06cd90 msgr2=0x7fb51c075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 shutdown_connections 2024-05-17T22:01:05.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:05.816+0000 7fb5216b2640 1 -- 172.21.15.83:0/3532697751 wait complete. 2024-05-17T22:01:05.827 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-05-17T22:01:05.043019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252880,"kb_used_data":7580,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721840,"statfs":{"total":773094113280,"available":772835164160,"internally_reserved":0,"allocated":7761920,"data_stored":3908720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.755496"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408887+0000","last_change":"2024-05-17T22:01:01.383630+0000","last_active":"2024-05-17T22:01:02.408887+0000","last_peered":"2024-05-17T22:01:02.408887+0000","last_clean":"2024-05-17T22:01:02.408887+0000","last_became_active":"2024-05-17T22:01:01.380878+0000","last_became_peered":"2024-05-17T22:01:01.380878+0000","last_unstale":"2024-05-17T22:01:02.408887+0000","last_undegraded":"2024-05-17T22:01:02.408887+0000","last_fullsized":"2024-05-17T22:01:02.408887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:01:58.690181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461994+0000","last_change":"2024-05-17T22:01:01.396593+0000","last_active":"2024-05-17T22:01:03.461994+0000","last_peered":"2024-05-17T22:01:03.461994+0000","last_clean":"2024-05-17T22:01:03.461994+0000","last_became_active":"2024-05-17T22:01:01.395937+0000","last_became_peered":"2024-05-17T22:01:01.395937+0000","last_unstale":"2024-05-17T22:01:03.461994+0000","last_undegraded":"2024-05-17T22:01:03.461994+0000","last_fullsized":"2024-05-17T22:01:03.461994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:51.193518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458853+0000","last_change":"2024-05-17T22:01:01.341074+0000","last_active":"2024-05-17T22:01:03.458853+0000","last_peered":"2024-05-17T22:01:03.458853+0000","last_clean":"2024-05-17T22:01:03.458853+0000","last_became_active":"2024-05-17T22:01:01.340248+0000","last_became_peered":"2024-05-17T22:01:01.340248+0000","last_unstale":"2024-05-17T22:01:03.458853+0000","last_undegraded":"2024-05-17T22:01:03.458853+0000","last_fullsized":"2024-05-17T22:01:03.458853+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:35.273265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459196+0000","last_change":"2024-05-17T22:01:01.397884+0000","last_active":"2024-05-17T22:01:03.459196+0000","last_peered":"2024-05-17T22:01:03.459196+0000","last_clean":"2024-05-17T22:01:03.459196+0000","last_became_active":"2024-05-17T22:01:01.397503+0000","last_became_peered":"2024-05-17T22:01:01.397503+0000","last_unstale":"2024-05-17T22:01:03.459196+0000","last_undegraded":"2024-05-17T22:01:03.459196+0000","last_fullsized":"2024-05-17T22:01:03.459196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:49.836526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347738+0000","last_change":"2024-05-17T22:01:01.365695+0000","last_active":"2024-05-17T22:01:02.347738+0000","last_peered":"2024-05-17T22:01:02.347738+0000","last_clean":"2024-05-17T22:01:02.347738+0000","last_became_active":"2024-05-17T22:01:01.365286+0000","last_became_peered":"2024-05-17T22:01:01.365286+0000","last_unstale":"2024-05-17T22:01:02.347738+0000","last_undegraded":"2024-05-17T22:01:02.347738+0000","last_fullsized":"2024-05-17T22:01:02.347738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:14:52.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461444+0000","last_change":"2024-05-17T22:01:01.395615+0000","last_active":"2024-05-17T22:01:03.461444+0000","last_peered":"2024-05-17T22:01:03.461444+0000","last_clean":"2024-05-17T22:01:03.461444+0000","last_became_active":"2024-05-17T22:01:01.392160+0000","last_became_peered":"2024-05-17T22:01:01.392160+0000","last_unstale":"2024-05-17T22:01:03.461444+0000","last_undegraded":"2024-05-17T22:01:03.461444+0000","last_fullsized":"2024-05-17T22:01:03.461444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:45:02.348773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349113+0000","last_change":"2024-05-17T22:01:01.338148+0000","last_active":"2024-05-17T22:01:02.349113+0000","last_peered":"2024-05-17T22:01:02.349113+0000","last_clean":"2024-05-17T22:01:02.349113+0000","last_became_active":"2024-05-17T22:01:01.337277+0000","last_became_peered":"2024-05-17T22:01:01.337277+0000","last_unstale":"2024-05-17T22:01:02.349113+0000","last_undegraded":"2024-05-17T22:01:02.349113+0000","last_fullsized":"2024-05-17T22:01:02.349113+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:45.219441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462145+0000","last_change":"2024-05-17T22:01:01.395528+0000","last_active":"2024-05-17T22:01:03.462145+0000","last_peered":"2024-05-17T22:01:03.462145+0000","last_clean":"2024-05-17T22:01:03.462145+0000","last_became_active":"2024-05-17T22:01:01.394048+0000","last_became_peered":"2024-05-17T22:01:01.394048+0000","last_unstale":"2024-05-17T22:01:03.462145+0000","last_undegraded":"2024-05-17T22:01:03.462145+0000","last_fullsized":"2024-05-17T22:01:03.462145+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:21.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348097+0000","last_change":"2024-05-17T22:01:01.346394+0000","last_active":"2024-05-17T22:01:02.348097+0000","last_peered":"2024-05-17T22:01:02.348097+0000","last_clean":"2024-05-17T22:01:02.348097+0000","last_became_active":"2024-05-17T22:01:01.345640+0000","last_became_peered":"2024-05-17T22:01:01.345640+0000","last_unstale":"2024-05-17T22:01:02.348097+0000","last_undegraded":"2024-05-17T22:01:02.348097+0000","last_fullsized":"2024-05-17T22:01:02.348097+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:59:14.633720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348805+0000","last_change":"2024-05-17T22:01:01.352419+0000","last_active":"2024-05-17T22:01:02.348805+0000","last_peered":"2024-05-17T22:01:02.348805+0000","last_clean":"2024-05-17T22:01:02.348805+0000","last_became_active":"2024-05-17T22:01:01.351601+0000","last_became_peered":"2024-05-17T22:01:01.351601+0000","last_unstale":"2024-05-17T22:01:02.348805+0000","last_undegraded":"2024-05-17T22:01:02.348805+0000","last_fullsized":"2024-05-17T22:01:02.348805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:21.813316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407965+0000","last_change":"2024-05-17T22:01:01.378591+0000","last_active":"2024-05-17T22:01:02.407965+0000","last_peered":"2024-05-17T22:01:02.407965+0000","last_clean":"2024-05-17T22:01:02.407965+0000","last_became_active":"2024-05-17T22:01:01.378002+0000","last_became_peered":"2024-05-17T22:01:01.378002+0000","last_unstale":"2024-05-17T22:01:02.407965+0000","last_undegraded":"2024-05-17T22:01:02.407965+0000","last_fullsized":"2024-05-17T22:01:02.407965+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:02:23.028552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459397+0000","last_change":"2024-05-17T22:01:01.335434+0000","last_active":"2024-05-17T22:01:03.459397+0000","last_peered":"2024-05-17T22:01:03.459397+0000","last_clean":"2024-05-17T22:01:03.459397+0000","last_became_active":"2024-05-17T22:01:01.334998+0000","last_became_peered":"2024-05-17T22:01:01.334998+0000","last_unstale":"2024-05-17T22:01:03.459397+0000","last_undegraded":"2024-05-17T22:01:03.459397+0000","last_fullsized":"2024-05-17T22:01:03.459397+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:34.124011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332329+0000","last_change":"2024-05-17T22:01:01.393015+0000","last_active":"2024-05-17T22:01:03.332329+0000","last_peered":"2024-05-17T22:01:03.332329+0000","last_clean":"2024-05-17T22:01:03.332329+0000","last_became_active":"2024-05-17T22:01:01.392624+0000","last_became_peered":"2024-05-17T22:01:01.392624+0000","last_unstale":"2024-05-17T22:01:03.332329+0000","last_undegraded":"2024-05-17T22:01:03.332329+0000","last_fullsized":"2024-05-17T22:01:03.332329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:35:36.659116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347699+0000","last_change":"2024-05-17T22:01:01.354909+0000","last_active":"2024-05-17T22:01:02.347699+0000","last_peered":"2024-05-17T22:01:02.347699+0000","last_clean":"2024-05-17T22:01:02.347699+0000","last_became_active":"2024-05-17T22:01:01.354549+0000","last_became_peered":"2024-05-17T22:01:01.354549+0000","last_unstale":"2024-05-17T22:01:02.347699+0000","last_undegraded":"2024-05-17T22:01:02.347699+0000","last_fullsized":"2024-05-17T22:01:02.347699+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:01:00.201810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331844+0000","last_change":"2024-05-17T22:01:01.338289+0000","last_active":"2024-05-17T22:01:03.331844+0000","last_peered":"2024-05-17T22:01:03.331844+0000","last_clean":"2024-05-17T22:01:03.331844+0000","last_became_active":"2024-05-17T22:01:01.337978+0000","last_became_peered":"2024-05-17T22:01:01.337978+0000","last_unstale":"2024-05-17T22:01:03.331844+0000","last_undegraded":"2024-05-17T22:01:03.331844+0000","last_fullsized":"2024-05-17T22:01:03.331844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:18:31.323279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349879+0000","last_change":"2024-05-17T22:01:01.404748+0000","last_active":"2024-05-17T22:01:02.349879+0000","last_peered":"2024-05-17T22:01:02.349879+0000","last_clean":"2024-05-17T22:01:02.349879+0000","last_became_active":"2024-05-17T22:01:01.403624+0000","last_became_peered":"2024-05-17T22:01:01.403624+0000","last_unstale":"2024-05-17T22:01:02.349879+0000","last_undegraded":"2024-05-17T22:01:02.349879+0000","last_fullsized":"2024-05-17T22:01:02.349879+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:24:30.624743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348122+0000","last_change":"2024-05-17T22:01:01.346211+0000","last_active":"2024-05-17T22:01:02.348122+0000","last_peered":"2024-05-17T22:01:02.348122+0000","last_clean":"2024-05-17T22:01:02.348122+0000","last_became_active":"2024-05-17T22:01:01.345141+0000","last_became_peered":"2024-05-17T22:01:01.345141+0000","last_unstale":"2024-05-17T22:01:02.348122+0000","last_undegraded":"2024-05-17T22:01:02.348122+0000","last_fullsized":"2024-05-17T22:01:02.348122+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:05:33.996730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458889+0000","last_change":"2024-05-17T22:01:01.394881+0000","last_active":"2024-05-17T22:01:03.458889+0000","last_peered":"2024-05-17T22:01:03.458889+0000","last_clean":"2024-05-17T22:01:03.458889+0000","last_became_active":"2024-05-17T22:01:01.394513+0000","last_became_peered":"2024-05-17T22:01:01.394513+0000","last_unstale":"2024-05-17T22:01:03.458889+0000","last_undegraded":"2024-05-17T22:01:03.458889+0000","last_fullsized":"2024-05-17T22:01:03.458889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:42:09.741553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462154+0000","last_change":"2024-05-17T22:01:01.401188+0000","last_active":"2024-05-17T22:01:03.462154+0000","last_peered":"2024-05-17T22:01:03.462154+0000","last_clean":"2024-05-17T22:01:03.462154+0000","last_became_active":"2024-05-17T22:01:01.400487+0000","last_became_peered":"2024-05-17T22:01:01.400487+0000","last_unstale":"2024-05-17T22:01:03.462154+0000","last_undegraded":"2024-05-17T22:01:03.462154+0000","last_fullsized":"2024-05-17T22:01:03.462154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:24.904719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332437+0000","last_change":"2024-05-17T22:01:01.343123+0000","last_active":"2024-05-17T22:01:03.332437+0000","last_peered":"2024-05-17T22:01:03.332437+0000","last_clean":"2024-05-17T22:01:03.332437+0000","last_became_active":"2024-05-17T22:01:01.342760+0000","last_became_peered":"2024-05-17T22:01:01.342760+0000","last_unstale":"2024-05-17T22:01:03.332437+0000","last_undegraded":"2024-05-17T22:01:03.332437+0000","last_fullsized":"2024-05-17T22:01:03.332437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:32.788833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347675+0000","last_change":"2024-05-17T22:01:01.350168+0000","last_active":"2024-05-17T22:01:02.347675+0000","last_peered":"2024-05-17T22:01:02.347675+0000","last_clean":"2024-05-17T22:01:02.347675+0000","last_became_active":"2024-05-17T22:01:01.348753+0000","last_became_peered":"2024-05-17T22:01:01.348753+0000","last_unstale":"2024-05-17T22:01:02.347675+0000","last_undegraded":"2024-05-17T22:01:02.347675+0000","last_fullsized":"2024-05-17T22:01:02.347675+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:51.193518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347753+0000","last_change":"2024-05-17T22:01:01.349951+0000","last_active":"2024-05-17T22:01:02.347753+0000","last_peered":"2024-05-17T22:01:02.347753+0000","last_clean":"2024-05-17T22:01:02.347753+0000","last_became_active":"2024-05-17T22:01:01.349091+0000","last_became_peered":"2024-05-17T22:01:01.349091+0000","last_unstale":"2024-05-17T22:01:02.347753+0000","last_undegraded":"2024-05-17T22:01:02.347753+0000","last_fullsized":"2024-05-17T22:01:02.347753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:42.859600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407887+0000","last_change":"2024-05-17T22:01:01.373818+0000","last_active":"2024-05-17T22:01:02.407887+0000","last_peered":"2024-05-17T22:01:02.407887+0000","last_clean":"2024-05-17T22:01:02.407887+0000","last_became_active":"2024-05-17T22:01:01.373064+0000","last_became_peered":"2024-05-17T22:01:01.373064+0000","last_unstale":"2024-05-17T22:01:02.407887+0000","last_undegraded":"2024-05-17T22:01:02.407887+0000","last_fullsized":"2024-05-17T22:01:02.407887+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:08:52.775863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407849+0000","last_change":"2024-05-17T22:01:01.402527+0000","last_active":"2024-05-17T22:01:02.407849+0000","last_peered":"2024-05-17T22:01:02.407849+0000","last_clean":"2024-05-17T22:01:02.407849+0000","last_became_active":"2024-05-17T22:01:01.401605+0000","last_became_peered":"2024-05-17T22:01:01.401605+0000","last_unstale":"2024-05-17T22:01:02.407849+0000","last_undegraded":"2024-05-17T22:01:02.407849+0000","last_fullsized":"2024-05-17T22:01:02.407849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:27.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348930+0000","last_change":"2024-05-17T22:01:01.383379+0000","last_active":"2024-05-17T22:01:02.348930+0000","last_peered":"2024-05-17T22:01:02.348930+0000","last_clean":"2024-05-17T22:01:02.348930+0000","last_became_active":"2024-05-17T22:01:01.382541+0000","last_became_peered":"2024-05-17T22:01:01.382541+0000","last_unstale":"2024-05-17T22:01:02.348930+0000","last_undegraded":"2024-05-17T22:01:02.348930+0000","last_fullsized":"2024-05-17T22:01:02.348930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.351006+0000","last_change":"2024-05-17T22:01:01.396802+0000","last_active":"2024-05-17T22:01:02.351006+0000","last_peered":"2024-05-17T22:01:02.351006+0000","last_clean":"2024-05-17T22:01:02.351006+0000","last_became_active":"2024-05-17T22:01:01.394908+0000","last_became_peered":"2024-05-17T22:01:01.394908+0000","last_unstale":"2024-05-17T22:01:02.351006+0000","last_undegraded":"2024-05-17T22:01:02.351006+0000","last_fullsized":"2024-05-17T22:01:02.351006+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459076+0000","last_change":"2024-05-17T22:01:01.375228+0000","last_active":"2024-05-17T22:01:03.459076+0000","last_peered":"2024-05-17T22:01:03.459076+0000","last_clean":"2024-05-17T22:01:03.459076+0000","last_became_active":"2024-05-17T22:01:01.374888+0000","last_became_peered":"2024-05-17T22:01:01.374888+0000","last_unstale":"2024-05-17T22:01:03.459076+0000","last_undegraded":"2024-05-17T22:01:03.459076+0000","last_fullsized":"2024-05-17T22:01:03.459076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:41.822295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347962+0000","last_change":"2024-05-17T22:01:01.349859+0000","last_active":"2024-05-17T22:01:02.347962+0000","last_peered":"2024-05-17T22:01:02.347962+0000","last_clean":"2024-05-17T22:01:02.347962+0000","last_became_active":"2024-05-17T22:01:01.348977+0000","last_became_peered":"2024-05-17T22:01:01.348977+0000","last_unstale":"2024-05-17T22:01:02.347962+0000","last_undegraded":"2024-05-17T22:01:02.347962+0000","last_fullsized":"2024-05-17T22:01:02.347962+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:16.696964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458562+0000","last_change":"2024-05-17T22:01:01.356422+0000","last_active":"2024-05-17T22:01:03.458562+0000","last_peered":"2024-05-17T22:01:03.458562+0000","last_clean":"2024-05-17T22:01:03.458562+0000","last_became_active":"2024-05-17T22:01:01.356076+0000","last_became_peered":"2024-05-17T22:01:01.356076+0000","last_unstale":"2024-05-17T22:01:03.458562+0000","last_undegraded":"2024-05-17T22:01:03.458562+0000","last_fullsized":"2024-05-17T22:01:03.458562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:07:13.745478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348658+0000","last_change":"2024-05-17T22:01:01.389882+0000","last_active":"2024-05-17T22:01:02.348658+0000","last_peered":"2024-05-17T22:01:02.348658+0000","last_clean":"2024-05-17T22:01:02.348658+0000","last_became_active":"2024-05-17T22:01:01.389013+0000","last_became_peered":"2024-05-17T22:01:01.389013+0000","last_unstale":"2024-05-17T22:01:02.348658+0000","last_undegraded":"2024-05-17T22:01:02.348658+0000","last_fullsized":"2024-05-17T22:01:02.348658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:46:07.400732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349450+0000","last_change":"2024-05-17T22:01:01.336758+0000","last_active":"2024-05-17T22:01:02.349450+0000","last_peered":"2024-05-17T22:01:02.349450+0000","last_clean":"2024-05-17T22:01:02.349450+0000","last_became_active":"2024-05-17T22:01:01.335920+0000","last_became_peered":"2024-05-17T22:01:01.335920+0000","last_unstale":"2024-05-17T22:01:02.349450+0000","last_undegraded":"2024-05-17T22:01:02.349450+0000","last_fullsized":"2024-05-17T22:01:02.349450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:18:31.323279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461066+0000","last_change":"2024-05-17T22:01:01.393309+0000","last_active":"2024-05-17T22:01:03.461066+0000","last_peered":"2024-05-17T22:01:03.461066+0000","last_clean":"2024-05-17T22:01:03.461066+0000","last_became_active":"2024-05-17T22:01:01.391635+0000","last_became_peered":"2024-05-17T22:01:01.391635+0000","last_unstale":"2024-05-17T22:01:03.461066+0000","last_undegraded":"2024-05-17T22:01:03.461066+0000","last_fullsized":"2024-05-17T22:01:03.461066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:58:13.902831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408744+0000","last_change":"2024-05-17T22:01:01.402351+0000","last_active":"2024-05-17T22:01:02.408744+0000","last_peered":"2024-05-17T22:01:02.408744+0000","last_clean":"2024-05-17T22:01:02.408744+0000","last_became_active":"2024-05-17T22:01:01.401831+0000","last_became_peered":"2024-05-17T22:01:01.401831+0000","last_unstale":"2024-05-17T22:01:02.408744+0000","last_undegraded":"2024-05-17T22:01:02.408744+0000","last_fullsized":"2024-05-17T22:01:02.408744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:03.853003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.409117+0000","last_change":"2024-05-17T22:01:00.316047+0000","last_active":"2024-05-17T22:01:02.409117+0000","last_peered":"2024-05-17T22:01:02.409117+0000","last_clean":"2024-05-17T22:01:02.409117+0000","last_became_active":"2024-05-17T22:01:00.315692+0000","last_became_peered":"2024-05-17T22:01:00.315692+0000","last_unstale":"2024-05-17T22:01:02.409117+0000","last_undegraded":"2024-05-17T22:01:02.409117+0000","last_fullsized":"2024-05-17T22:01:02.409117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:23.893723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348194+0000","last_change":"2024-05-17T22:01:01.346565+0000","last_active":"2024-05-17T22:01:02.348194+0000","last_peered":"2024-05-17T22:01:02.348194+0000","last_clean":"2024-05-17T22:01:02.348194+0000","last_became_active":"2024-05-17T22:01:01.346039+0000","last_became_peered":"2024-05-17T22:01:01.346039+0000","last_unstale":"2024-05-17T22:01:02.348194+0000","last_undegraded":"2024-05-17T22:01:02.348194+0000","last_fullsized":"2024-05-17T22:01:02.348194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:40:39.239245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332766+0000","last_change":"2024-05-17T22:01:00.309435+0000","last_active":"2024-05-17T22:01:03.332766+0000","last_peered":"2024-05-17T22:01:03.332766+0000","last_clean":"2024-05-17T22:01:03.332766+0000","last_became_active":"2024-05-17T22:01:00.309112+0000","last_became_peered":"2024-05-17T22:01:00.309112+0000","last_unstale":"2024-05-17T22:01:03.332766+0000","last_undegraded":"2024-05-17T22:01:03.332766+0000","last_fullsized":"2024-05-17T22:01:03.332766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:27:20.802320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462011+0000","last_change":"2024-05-17T22:01:01.394528+0000","last_active":"2024-05-17T22:01:03.462011+0000","last_peered":"2024-05-17T22:01:03.462011+0000","last_clean":"2024-05-17T22:01:03.462011+0000","last_became_active":"2024-05-17T22:01:01.391743+0000","last_became_peered":"2024-05-17T22:01:01.391743+0000","last_unstale":"2024-05-17T22:01:03.462011+0000","last_undegraded":"2024-05-17T22:01:03.462011+0000","last_fullsized":"2024-05-17T22:01:03.462011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:42.859600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349585+0000","last_change":"2024-05-17T22:01:00.328114+0000","last_active":"2024-05-17T22:01:02.349585+0000","last_peered":"2024-05-17T22:01:02.349585+0000","last_clean":"2024-05-17T22:01:02.349585+0000","last_became_active":"2024-05-17T22:01:00.327381+0000","last_became_peered":"2024-05-17T22:01:00.327381+0000","last_unstale":"2024-05-17T22:01:02.349585+0000","last_undegraded":"2024-05-17T22:01:02.349585+0000","last_fullsized":"2024-05-17T22:01:02.349585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:42:08.730557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408600+0000","last_change":"2024-05-17T22:01:01.377880+0000","last_active":"2024-05-17T22:01:02.408600+0000","last_peered":"2024-05-17T22:01:02.408600+0000","last_clean":"2024-05-17T22:01:02.408600+0000","last_became_active":"2024-05-17T22:01:01.372339+0000","last_became_peered":"2024-05-17T22:01:01.372339+0000","last_unstale":"2024-05-17T22:01:02.408600+0000","last_undegraded":"2024-05-17T22:01:02.408600+0000","last_fullsized":"2024-05-17T22:01:02.408600+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:43:26.614346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462272+0000","last_change":"2024-05-17T22:01:00.310855+0000","last_active":"2024-05-17T22:01:03.462272+0000","last_peered":"2024-05-17T22:01:03.462272+0000","last_clean":"2024-05-17T22:01:03.462272+0000","last_became_active":"2024-05-17T22:01:00.310392+0000","last_became_peered":"2024-05-17T22:01:00.310392+0000","last_unstale":"2024-05-17T22:01:03.462272+0000","last_undegraded":"2024-05-17T22:01:03.462272+0000","last_fullsized":"2024-05-17T22:01:03.462272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:33:03.787565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350222+0000","last_change":"2024-05-17T22:01:01.345139+0000","last_active":"2024-05-17T22:01:02.350222+0000","last_peered":"2024-05-17T22:01:02.350222+0000","last_clean":"2024-05-17T22:01:02.350222+0000","last_became_active":"2024-05-17T22:01:01.344212+0000","last_became_peered":"2024-05-17T22:01:01.344212+0000","last_unstale":"2024-05-17T22:01:02.350222+0000","last_undegraded":"2024-05-17T22:01:02.350222+0000","last_fullsized":"2024-05-17T22:01:02.350222+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:18:11.913763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458973+0000","last_change":"2024-05-17T22:01:00.325464+0000","last_active":"2024-05-17T22:01:03.458973+0000","last_peered":"2024-05-17T22:01:03.458973+0000","last_clean":"2024-05-17T22:01:03.458973+0000","last_became_active":"2024-05-17T22:01:00.325104+0000","last_became_peered":"2024-05-17T22:01:00.325104+0000","last_unstale":"2024-05-17T22:01:03.458973+0000","last_undegraded":"2024-05-17T22:01:03.458973+0000","last_fullsized":"2024-05-17T22:01:03.458973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:23:01.026719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348119+0000","last_change":"2024-05-17T22:01:01.374133+0000","last_active":"2024-05-17T22:01:02.348119+0000","last_peered":"2024-05-17T22:01:02.348119+0000","last_clean":"2024-05-17T22:01:02.348119+0000","last_became_active":"2024-05-17T22:01:01.373225+0000","last_became_peered":"2024-05-17T22:01:01.373225+0000","last_unstale":"2024-05-17T22:01:02.348119+0000","last_undegraded":"2024-05-17T22:01:02.348119+0000","last_fullsized":"2024-05-17T22:01:02.348119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:24.904719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461223+0000","last_change":"2024-05-17T22:01:00.329515+0000","last_active":"2024-05-17T22:01:03.461223+0000","last_peered":"2024-05-17T22:01:03.461223+0000","last_clean":"2024-05-17T22:01:03.461223+0000","last_became_active":"2024-05-17T22:01:00.328555+0000","last_became_peered":"2024-05-17T22:01:00.328555+0000","last_unstale":"2024-05-17T22:01:03.461223+0000","last_undegraded":"2024-05-17T22:01:03.461223+0000","last_fullsized":"2024-05-17T22:01:03.461223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:33.676626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347401+0000","last_change":"2024-05-17T22:01:01.361007+0000","last_active":"2024-05-17T22:01:02.347401+0000","last_peered":"2024-05-17T22:01:02.347401+0000","last_clean":"2024-05-17T22:01:02.347401+0000","last_became_active":"2024-05-17T22:01:01.360759+0000","last_became_peered":"2024-05-17T22:01:01.360759+0000","last_unstale":"2024-05-17T22:01:02.347401+0000","last_undegraded":"2024-05-17T22:01:02.347401+0000","last_fullsized":"2024-05-17T22:01:02.347401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:18:27.118303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459346+0000","last_change":"2024-05-17T22:01:00.328555+0000","last_active":"2024-05-17T22:01:03.459346+0000","last_peered":"2024-05-17T22:01:03.459346+0000","last_clean":"2024-05-17T22:01:03.459346+0000","last_became_active":"2024-05-17T22:01:00.328236+0000","last_became_peered":"2024-05-17T22:01:00.328236+0000","last_unstale":"2024-05-17T22:01:03.459346+0000","last_undegraded":"2024-05-17T22:01:03.459346+0000","last_fullsized":"2024-05-17T22:01:03.459346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:22:36.050481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332175+0000","last_change":"2024-05-17T22:01:01.362035+0000","last_active":"2024-05-17T22:01:03.332175+0000","last_peered":"2024-05-17T22:01:03.332175+0000","last_clean":"2024-05-17T22:01:03.332175+0000","last_became_active":"2024-05-17T22:01:01.361753+0000","last_became_peered":"2024-05-17T22:01:01.361753+0000","last_unstale":"2024-05-17T22:01:03.332175+0000","last_undegraded":"2024-05-17T22:01:03.332175+0000","last_fullsized":"2024-05-17T22:01:03.332175+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:18:11.913763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347357+0000","last_change":"2024-05-17T22:01:00.306375+0000","last_active":"2024-05-17T22:01:02.347357+0000","last_peered":"2024-05-17T22:01:02.347357+0000","last_clean":"2024-05-17T22:01:02.347357+0000","last_became_active":"2024-05-17T22:01:00.305886+0000","last_became_peered":"2024-05-17T22:01:00.305886+0000","last_unstale":"2024-05-17T22:01:02.347357+0000","last_undegraded":"2024-05-17T22:01:02.347357+0000","last_fullsized":"2024-05-17T22:01:02.347357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:11:27.011576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.351165+0000","last_change":"2024-05-17T22:01:01.366828+0000","last_active":"2024-05-17T22:01:02.351165+0000","last_peered":"2024-05-17T22:01:02.351165+0000","last_clean":"2024-05-17T22:01:02.351165+0000","last_became_active":"2024-05-17T22:01:01.365965+0000","last_became_peered":"2024-05-17T22:01:01.365965+0000","last_unstale":"2024-05-17T22:01:02.351165+0000","last_undegraded":"2024-05-17T22:01:02.351165+0000","last_fullsized":"2024-05-17T22:01:02.351165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:27.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349200+0000","last_change":"2024-05-17T22:01:00.327739+0000","last_active":"2024-05-17T22:01:02.349200+0000","last_peered":"2024-05-17T22:01:02.349200+0000","last_clean":"2024-05-17T22:01:02.349200+0000","last_became_active":"2024-05-17T22:01:00.327381+0000","last_became_peered":"2024-05-17T22:01:00.327381+0000","last_unstale":"2024-05-17T22:01:02.349200+0000","last_undegraded":"2024-05-17T22:01:02.349200+0000","last_fullsized":"2024-05-17T22:01:02.349200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:42:48.825530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408944+0000","last_change":"2024-05-17T22:01:01.403914+0000","last_active":"2024-05-17T22:01:02.408944+0000","last_peered":"2024-05-17T22:01:02.408944+0000","last_clean":"2024-05-17T22:01:02.408944+0000","last_became_active":"2024-05-17T22:01:01.403248+0000","last_became_peered":"2024-05-17T22:01:01.403248+0000","last_unstale":"2024-05-17T22:01:02.408944+0000","last_undegraded":"2024-05-17T22:01:02.408944+0000","last_fullsized":"2024-05-17T22:01:02.408944+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:40:54.523318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461873+0000","last_change":"2024-05-17T22:01:00.331079+0000","last_active":"2024-05-17T22:01:03.461873+0000","last_peered":"2024-05-17T22:01:03.461873+0000","last_clean":"2024-05-17T22:01:03.461873+0000","last_became_active":"2024-05-17T22:01:00.330509+0000","last_became_peered":"2024-05-17T22:01:00.330509+0000","last_unstale":"2024-05-17T22:01:03.461873+0000","last_undegraded":"2024-05-17T22:01:03.461873+0000","last_fullsized":"2024-05-17T22:01:03.461873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:00.740087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462098+0000","last_change":"2024-05-17T22:01:01.397260+0000","last_active":"2024-05-17T22:01:03.462098+0000","last_peered":"2024-05-17T22:01:03.462098+0000","last_clean":"2024-05-17T22:01:03.462098+0000","last_became_active":"2024-05-17T22:01:01.393027+0000","last_became_peered":"2024-05-17T22:01:01.393027+0000","last_unstale":"2024-05-17T22:01:03.462098+0000","last_undegraded":"2024-05-17T22:01:03.462098+0000","last_fullsized":"2024-05-17T22:01:03.462098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:16.696964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459173+0000","last_change":"2024-05-17T22:01:00.314472+0000","last_active":"2024-05-17T22:01:03.459173+0000","last_peered":"2024-05-17T22:01:03.459173+0000","last_clean":"2024-05-17T22:01:03.459173+0000","last_became_active":"2024-05-17T22:01:00.314073+0000","last_became_peered":"2024-05-17T22:01:00.314073+0000","last_unstale":"2024-05-17T22:01:03.459173+0000","last_undegraded":"2024-05-17T22:01:03.459173+0000","last_fullsized":"2024-05-17T22:01:03.459173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:02.842007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347708+0000","last_change":"2024-05-17T22:01:01.343893+0000","last_active":"2024-05-17T22:01:02.347708+0000","last_peered":"2024-05-17T22:01:02.347708+0000","last_clean":"2024-05-17T22:01:02.347708+0000","last_became_active":"2024-05-17T22:01:01.343477+0000","last_became_peered":"2024-05-17T22:01:01.343477+0000","last_unstale":"2024-05-17T22:01:02.347708+0000","last_undegraded":"2024-05-17T22:01:02.347708+0000","last_fullsized":"2024-05-17T22:01:02.347708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:13:42.607139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459517+0000","last_change":"2024-05-17T22:01:00.309580+0000","last_active":"2024-05-17T22:01:03.459517+0000","last_peered":"2024-05-17T22:01:03.459517+0000","last_clean":"2024-05-17T22:01:03.459517+0000","last_became_active":"2024-05-17T22:01:00.309263+0000","last_became_peered":"2024-05-17T22:01:00.309263+0000","last_unstale":"2024-05-17T22:01:03.459517+0000","last_undegraded":"2024-05-17T22:01:03.459517+0000","last_fullsized":"2024-05-17T22:01:03.459517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:29.966233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407644+0000","last_change":"2024-05-17T22:01:01.368634+0000","last_active":"2024-05-17T22:01:02.407644+0000","last_peered":"2024-05-17T22:01:02.407644+0000","last_clean":"2024-05-17T22:01:02.407644+0000","last_became_active":"2024-05-17T22:01:01.367805+0000","last_became_peered":"2024-05-17T22:01:01.367805+0000","last_unstale":"2024-05-17T22:01:02.407644+0000","last_undegraded":"2024-05-17T22:01:02.407644+0000","last_fullsized":"2024-05-17T22:01:02.407644+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458548+0000","last_change":"2024-05-17T22:01:00.307058+0000","last_active":"2024-05-17T22:01:03.458548+0000","last_peered":"2024-05-17T22:01:03.458548+0000","last_clean":"2024-05-17T22:01:03.458548+0000","last_became_active":"2024-05-17T22:01:00.306788+0000","last_became_peered":"2024-05-17T22:01:00.306788+0000","last_unstale":"2024-05-17T22:01:03.458548+0000","last_undegraded":"2024-05-17T22:01:03.458548+0000","last_fullsized":"2024-05-17T22:01:03.458548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:47:33.676626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350583+0000","last_change":"2024-05-17T22:01:01.348019+0000","last_active":"2024-05-17T22:01:02.350583+0000","last_peered":"2024-05-17T22:01:02.350583+0000","last_clean":"2024-05-17T22:01:02.350583+0000","last_became_active":"2024-05-17T22:01:01.347176+0000","last_became_peered":"2024-05-17T22:01:01.347176+0000","last_unstale":"2024-05-17T22:01:02.350583+0000","last_undegraded":"2024-05-17T22:01:02.350583+0000","last_fullsized":"2024-05-17T22:01:02.350583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:35:36.659116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461569+0000","last_change":"2024-05-17T22:01:00.323505+0000","last_active":"2024-05-17T22:01:03.461569+0000","last_peered":"2024-05-17T22:01:03.461569+0000","last_clean":"2024-05-17T22:01:03.461569+0000","last_became_active":"2024-05-17T22:01:00.322503+0000","last_became_peered":"2024-05-17T22:01:00.322503+0000","last_unstale":"2024-05-17T22:01:03.461569+0000","last_undegraded":"2024-05-17T22:01:03.461569+0000","last_fullsized":"2024-05-17T22:01:03.461569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:04:00.933298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331844+0000","last_change":"2024-05-17T22:01:01.401267+0000","last_active":"2024-05-17T22:01:03.331844+0000","last_peered":"2024-05-17T22:01:03.331844+0000","last_clean":"2024-05-17T22:01:03.331844+0000","last_became_active":"2024-05-17T22:01:01.400933+0000","last_became_peered":"2024-05-17T22:01:01.400933+0000","last_unstale":"2024-05-17T22:01:03.331844+0000","last_undegraded":"2024-05-17T22:01:03.331844+0000","last_fullsized":"2024-05-17T22:01:03.331844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:30:45.219441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349132+0000","last_change":"2024-05-17T22:01:00.315139+0000","last_active":"2024-05-17T22:01:02.349132+0000","last_peered":"2024-05-17T22:01:02.349132+0000","last_clean":"2024-05-17T22:01:02.349132+0000","last_became_active":"2024-05-17T22:01:00.314569+0000","last_became_peered":"2024-05-17T22:01:00.314569+0000","last_unstale":"2024-05-17T22:01:02.349132+0000","last_undegraded":"2024-05-17T22:01:02.349132+0000","last_fullsized":"2024-05-17T22:01:02.349132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:45:01.337777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347535+0000","last_change":"2024-05-17T22:01:01.346073+0000","last_active":"2024-05-17T22:01:02.347535+0000","last_peered":"2024-05-17T22:01:02.347535+0000","last_clean":"2024-05-17T22:01:02.347535+0000","last_became_active":"2024-05-17T22:01:01.345062+0000","last_became_peered":"2024-05-17T22:01:01.345062+0000","last_unstale":"2024-05-17T22:01:02.347535+0000","last_undegraded":"2024-05-17T22:01:02.347535+0000","last_fullsized":"2024-05-17T22:01:02.347535+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:12:37.799390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347445+0000","last_change":"2024-05-17T22:01:00.328366+0000","last_active":"2024-05-17T22:01:02.347445+0000","last_peered":"2024-05-17T22:01:02.347445+0000","last_clean":"2024-05-17T22:01:02.347445+0000","last_became_active":"2024-05-17T22:01:00.327870+0000","last_became_peered":"2024-05-17T22:01:00.327870+0000","last_unstale":"2024-05-17T22:01:02.347445+0000","last_undegraded":"2024-05-17T22:01:02.347445+0000","last_fullsized":"2024-05-17T22:01:02.347445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:19.261523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349297+0000","last_change":"2024-05-17T22:01:01.350254+0000","last_active":"2024-05-17T22:01:02.349297+0000","last_peered":"2024-05-17T22:01:02.349297+0000","last_clean":"2024-05-17T22:01:02.349297+0000","last_became_active":"2024-05-17T22:01:01.349361+0000","last_became_peered":"2024-05-17T22:01:01.349361+0000","last_unstale":"2024-05-17T22:01:02.349297+0000","last_undegraded":"2024-05-17T22:01:02.349297+0000","last_fullsized":"2024-05-17T22:01:02.349297+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:13:52.122119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461816+0000","last_change":"2024-05-17T22:01:00.321403+0000","last_active":"2024-05-17T22:01:03.461816+0000","last_peered":"2024-05-17T22:01:03.461816+0000","last_clean":"2024-05-17T22:01:03.461816+0000","last_became_active":"2024-05-17T22:01:00.320624+0000","last_became_peered":"2024-05-17T22:01:00.320624+0000","last_unstale":"2024-05-17T22:01:03.461816+0000","last_undegraded":"2024-05-17T22:01:03.461816+0000","last_fullsized":"2024-05-17T22:01:03.461816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:22:36.050481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.409112+0000","last_change":"2024-05-17T22:01:01.385066+0000","last_active":"2024-05-17T22:01:02.409112+0000","last_peered":"2024-05-17T22:01:02.409112+0000","last_clean":"2024-05-17T22:01:02.409112+0000","last_became_active":"2024-05-17T22:01:01.382925+0000","last_became_peered":"2024-05-17T22:01:01.382925+0000","last_unstale":"2024-05-17T22:01:02.409112+0000","last_undegraded":"2024-05-17T22:01:02.409112+0000","last_fullsized":"2024-05-17T22:01:02.409112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:17.554044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349469+0000","last_change":"2024-05-17T22:01:00.329592+0000","last_active":"2024-05-17T22:01:02.349469+0000","last_peered":"2024-05-17T22:01:02.349469+0000","last_clean":"2024-05-17T22:01:02.349469+0000","last_became_active":"2024-05-17T22:01:00.329099+0000","last_became_peered":"2024-05-17T22:01:00.329099+0000","last_unstale":"2024-05-17T22:01:02.349469+0000","last_undegraded":"2024-05-17T22:01:02.349469+0000","last_fullsized":"2024-05-17T22:01:02.349469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:33.113015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459455+0000","last_change":"2024-05-17T22:01:01.335326+0000","last_active":"2024-05-17T22:01:03.459455+0000","last_peered":"2024-05-17T22:01:03.459455+0000","last_clean":"2024-05-17T22:01:03.459455+0000","last_became_active":"2024-05-17T22:01:01.334588+0000","last_became_peered":"2024-05-17T22:01:01.334588+0000","last_unstale":"2024-05-17T22:01:03.459455+0000","last_undegraded":"2024-05-17T22:01:03.459455+0000","last_fullsized":"2024-05-17T22:01:03.459455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:21:39.294671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.462099+0000","last_change":"2024-05-17T22:01:00.332474+0000","last_active":"2024-05-17T22:01:03.462099+0000","last_peered":"2024-05-17T22:01:03.462099+0000","last_clean":"2024-05-17T22:01:03.462099+0000","last_became_active":"2024-05-17T22:01:00.331949+0000","last_became_peered":"2024-05-17T22:01:00.331949+0000","last_unstale":"2024-05-17T22:01:03.462099+0000","last_undegraded":"2024-05-17T22:01:03.462099+0000","last_fullsized":"2024-05-17T22:01:03.462099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:50.714168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347754+0000","last_change":"2024-05-17T22:01:01.347350+0000","last_active":"2024-05-17T22:01:02.347754+0000","last_peered":"2024-05-17T22:01:02.347754+0000","last_clean":"2024-05-17T22:01:02.347754+0000","last_became_active":"2024-05-17T22:01:01.347032+0000","last_became_peered":"2024-05-17T22:01:01.347032+0000","last_unstale":"2024-05-17T22:01:02.347754+0000","last_undegraded":"2024-05-17T22:01:02.347754+0000","last_fullsized":"2024-05-17T22:01:02.347754+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:11:16.386285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408556+0000","last_change":"2024-05-17T22:01:00.327376+0000","last_active":"2024-05-17T22:01:02.408556+0000","last_peered":"2024-05-17T22:01:02.408556+0000","last_clean":"2024-05-17T22:01:02.408556+0000","last_became_active":"2024-05-17T22:01:00.326887+0000","last_became_peered":"2024-05-17T22:01:00.326887+0000","last_unstale":"2024-05-17T22:01:02.408556+0000","last_undegraded":"2024-05-17T22:01:02.408556+0000","last_fullsized":"2024-05-17T22:01:02.408556+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:15.685968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458806+0000","last_change":"2024-05-17T22:01:01.347594+0000","last_active":"2024-05-17T22:01:03.458806+0000","last_peered":"2024-05-17T22:01:03.458806+0000","last_clean":"2024-05-17T22:01:03.458806+0000","last_became_active":"2024-05-17T22:01:01.347271+0000","last_became_peered":"2024-05-17T22:01:01.347271+0000","last_unstale":"2024-05-17T22:01:03.458806+0000","last_undegraded":"2024-05-17T22:01:03.458806+0000","last_fullsized":"2024-05-17T22:01:03.458806+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:37:44.857791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461763+0000","last_change":"2024-05-17T22:01:00.323760+0000","last_active":"2024-05-17T22:01:03.461763+0000","last_peered":"2024-05-17T22:01:03.461763+0000","last_clean":"2024-05-17T22:01:03.461763+0000","last_became_active":"2024-05-17T22:01:00.319194+0000","last_became_peered":"2024-05-17T22:01:00.319194+0000","last_unstale":"2024-05-17T22:01:03.461763+0000","last_undegraded":"2024-05-17T22:01:03.461763+0000","last_fullsized":"2024-05-17T22:01:03.461763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:29.966233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458661+0000","last_change":"2024-05-17T22:01:01.392920+0000","last_active":"2024-05-17T22:01:03.458661+0000","last_peered":"2024-05-17T22:01:03.458661+0000","last_clean":"2024-05-17T22:01:03.458661+0000","last_became_active":"2024-05-17T22:01:01.392494+0000","last_became_peered":"2024-05-17T22:01:01.392494+0000","last_unstale":"2024-05-17T22:01:03.458661+0000","last_undegraded":"2024-05-17T22:01:03.458661+0000","last_fullsized":"2024-05-17T22:01:03.458661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:58:26.754616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.349603+0000","last_change":"2024-05-17T22:01:00.313840+0000","last_active":"2024-05-17T22:01:02.349603+0000","last_peered":"2024-05-17T22:01:02.349603+0000","last_clean":"2024-05-17T22:01:02.349603+0000","last_became_active":"2024-05-17T22:01:00.313278+0000","last_became_peered":"2024-05-17T22:01:00.313278+0000","last_unstale":"2024-05-17T22:01:02.349603+0000","last_undegraded":"2024-05-17T22:01:02.349603+0000","last_fullsized":"2024-05-17T22:01:02.349603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:41:50.182522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458491+0000","last_change":"2024-05-17T22:01:01.392990+0000","last_active":"2024-05-17T22:01:03.458491+0000","last_peered":"2024-05-17T22:01:03.458491+0000","last_clean":"2024-05-17T22:01:03.458491+0000","last_became_active":"2024-05-17T22:01:01.392200+0000","last_became_peered":"2024-05-17T22:01:01.392200+0000","last_unstale":"2024-05-17T22:01:03.458491+0000","last_undegraded":"2024-05-17T22:01:03.458491+0000","last_fullsized":"2024-05-17T22:01:03.458491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:20.238581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.331808+0000","last_change":"2024-05-17T22:01:00.309668+0000","last_active":"2024-05-17T22:01:03.331808+0000","last_peered":"2024-05-17T22:01:03.331808+0000","last_clean":"2024-05-17T22:01:03.331808+0000","last_became_active":"2024-05-17T22:01:00.309344+0000","last_became_peered":"2024-05-17T22:01:00.309344+0000","last_unstale":"2024-05-17T22:01:03.331808+0000","last_undegraded":"2024-05-17T22:01:03.331808+0000","last_fullsized":"2024-05-17T22:01:03.331808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:15.685968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332494+0000","last_change":"2024-05-17T22:01:01.339126+0000","last_active":"2024-05-17T22:01:03.332494+0000","last_peered":"2024-05-17T22:01:03.332494+0000","last_clean":"2024-05-17T22:01:03.332494+0000","last_became_active":"2024-05-17T22:01:01.338816+0000","last_became_peered":"2024-05-17T22:01:01.338816+0000","last_unstale":"2024-05-17T22:01:03.332494+0000","last_undegraded":"2024-05-17T22:01:03.332494+0000","last_fullsized":"2024-05-17T22:01:03.332494+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:21:13.572528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461527+0000","last_change":"2024-05-17T22:01:00.318933+0000","last_active":"2024-05-17T22:01:03.461527+0000","last_peered":"2024-05-17T22:01:03.461527+0000","last_clean":"2024-05-17T22:01:03.461527+0000","last_became_active":"2024-05-17T22:01:00.311013+0000","last_became_peered":"2024-05-17T22:01:00.311013+0000","last_unstale":"2024-05-17T22:01:03.461527+0000","last_undegraded":"2024-05-17T22:01:03.461527+0000","last_fullsized":"2024-05-17T22:01:03.461527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:40:07.783952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.350940+0000","last_change":"2024-05-17T22:01:01.335713+0000","last_active":"2024-05-17T22:01:02.350940+0000","last_peered":"2024-05-17T22:01:02.350940+0000","last_clean":"2024-05-17T22:01:02.350940+0000","last_became_active":"2024-05-17T22:01:01.334908+0000","last_became_peered":"2024-05-17T22:01:01.334908+0000","last_unstale":"2024-05-17T22:01:02.350940+0000","last_undegraded":"2024-05-17T22:01:02.350940+0000","last_fullsized":"2024-05-17T22:01:02.350940+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:43:32.788833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.330561+0000","last_change":"2024-05-17T22:01:00.319773+0000","last_active":"2024-05-17T22:01:03.330561+0000","last_peered":"2024-05-17T22:01:03.330561+0000","last_clean":"2024-05-17T22:01:03.330561+0000","last_became_active":"2024-05-17T22:01:00.313371+0000","last_became_peered":"2024-05-17T22:01:00.313371+0000","last_unstale":"2024-05-17T22:01:03.330561+0000","last_undegraded":"2024-05-17T22:01:03.330561+0000","last_fullsized":"2024-05-17T22:01:03.330561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:47:41.848604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408578+0000","last_change":"2024-05-17T22:01:01.397051+0000","last_active":"2024-05-17T22:01:02.408578+0000","last_peered":"2024-05-17T22:01:02.408578+0000","last_clean":"2024-05-17T22:01:02.408578+0000","last_became_active":"2024-05-17T22:01:01.396030+0000","last_became_peered":"2024-05-17T22:01:01.396030+0000","last_unstale":"2024-05-17T22:01:02.408578+0000","last_undegraded":"2024-05-17T22:01:02.408578+0000","last_fullsized":"2024-05-17T22:01:02.408578+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:46:16.984756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347944+0000","last_change":"2024-05-17T22:01:00.309343+0000","last_active":"2024-05-17T22:01:02.347944+0000","last_peered":"2024-05-17T22:01:02.347944+0000","last_clean":"2024-05-17T22:01:02.347944+0000","last_became_active":"2024-05-17T22:01:00.309062+0000","last_became_peered":"2024-05-17T22:01:00.309062+0000","last_unstale":"2024-05-17T22:01:02.347944+0000","last_undegraded":"2024-05-17T22:01:02.347944+0000","last_fullsized":"2024-05-17T22:01:02.347944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:19:16.312055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.408667+0000","last_change":"2024-05-17T22:01:01.378631+0000","last_active":"2024-05-17T22:01:02.408667+0000","last_peered":"2024-05-17T22:01:02.408667+0000","last_clean":"2024-05-17T22:01:02.408667+0000","last_became_active":"2024-05-17T22:01:01.373285+0000","last_became_peered":"2024-05-17T22:01:01.373285+0000","last_unstale":"2024-05-17T22:01:02.408667+0000","last_undegraded":"2024-05-17T22:01:02.408667+0000","last_fullsized":"2024-05-17T22:01:02.408667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:23:02.037715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.332164+0000","last_change":"2024-05-17T22:01:00.325477+0000","last_active":"2024-05-17T22:01:03.332164+0000","last_peered":"2024-05-17T22:01:03.332164+0000","last_clean":"2024-05-17T22:01:03.332164+0000","last_became_active":"2024-05-17T22:01:00.325208+0000","last_became_peered":"2024-05-17T22:01:00.325208+0000","last_unstale":"2024-05-17T22:01:03.332164+0000","last_undegraded":"2024-05-17T22:01:03.332164+0000","last_fullsized":"2024-05-17T22:01:03.332164+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:46:23.893723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347802+0000","last_change":"2024-05-17T22:01:01.347590+0000","last_active":"2024-05-17T22:01:02.347802+0000","last_peered":"2024-05-17T22:01:02.347802+0000","last_clean":"2024-05-17T22:01:02.347802+0000","last_became_active":"2024-05-17T22:01:01.347239+0000","last_became_peered":"2024-05-17T22:01:01.347239+0000","last_unstale":"2024-05-17T22:01:02.347802+0000","last_undegraded":"2024-05-17T22:01:02.347802+0000","last_fullsized":"2024-05-17T22:01:02.347802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:50:29.242277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.347564+0000","last_change":"2024-05-17T22:01:00.314570+0000","last_active":"2024-05-17T22:01:02.347564+0000","last_peered":"2024-05-17T22:01:02.347564+0000","last_clean":"2024-05-17T22:01:02.347564+0000","last_became_active":"2024-05-17T22:01:00.314145+0000","last_became_peered":"2024-05-17T22:01:00.314145+0000","last_unstale":"2024-05-17T22:01:02.347564+0000","last_undegraded":"2024-05-17T22:01:02.347564+0000","last_fullsized":"2024-05-17T22:01:02.347564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:47:21.456793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461025+0000","last_change":"2024-05-17T22:01:01.394727+0000","last_active":"2024-05-17T22:01:03.461025+0000","last_peered":"2024-05-17T22:01:03.461025+0000","last_clean":"2024-05-17T22:01:03.461025+0000","last_became_active":"2024-05-17T22:01:01.391950+0000","last_became_peered":"2024-05-17T22:01:01.391950+0000","last_unstale":"2024-05-17T22:01:03.461025+0000","last_undegraded":"2024-05-17T22:01:03.461025+0000","last_fullsized":"2024-05-17T22:01:03.461025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:31:35.092742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458168+0000","last_change":"2024-05-17T22:01:00.327489+0000","last_active":"2024-05-17T22:01:03.458168+0000","last_peered":"2024-05-17T22:01:03.458168+0000","last_clean":"2024-05-17T22:01:03.458168+0000","last_became_active":"2024-05-17T22:01:00.327226+0000","last_became_peered":"2024-05-17T22:01:00.327226+0000","last_unstale":"2024-05-17T22:01:03.458168+0000","last_undegraded":"2024-05-17T22:01:03.458168+0000","last_fullsized":"2024-05-17T22:01:03.458168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:40:07.783952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.458191+0000","last_change":"2024-05-17T22:01:01.341387+0000","last_active":"2024-05-17T22:01:03.458191+0000","last_peered":"2024-05-17T22:01:03.458191+0000","last_clean":"2024-05-17T22:01:03.458191+0000","last_became_active":"2024-05-17T22:01:01.341027+0000","last_became_peered":"2024-05-17T22:01:01.341027+0000","last_unstale":"2024-05-17T22:01:03.458191+0000","last_undegraded":"2024-05-17T22:01:03.458191+0000","last_fullsized":"2024-05-17T22:01:03.458191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:26:20.272519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.348240+0000","last_change":"2024-05-17T22:01:00.325512+0000","last_active":"2024-05-17T22:01:02.348240+0000","last_peered":"2024-05-17T22:01:02.348240+0000","last_clean":"2024-05-17T22:01:02.348240+0000","last_became_active":"2024-05-17T22:01:00.325096+0000","last_became_peered":"2024-05-17T22:01:00.325096+0000","last_unstale":"2024-05-17T22:01:02.348240+0000","last_undegraded":"2024-05-17T22:01:02.348240+0000","last_fullsized":"2024-05-17T22:01:02.348240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:29:50.049155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.460973+0000","last_change":"2024-05-17T22:01:01.391883+0000","last_active":"2024-05-17T22:01:03.460973+0000","last_peered":"2024-05-17T22:01:03.460973+0000","last_clean":"2024-05-17T22:01:03.460973+0000","last_became_active":"2024-05-17T22:01:01.388362+0000","last_became_peered":"2024-05-17T22:01:01.388362+0000","last_unstale":"2024-05-17T22:01:03.460973+0000","last_undegraded":"2024-05-17T22:01:03.460973+0000","last_fullsized":"2024-05-17T22:01:03.460973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:31:09.875245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461369+0000","last_change":"2024-05-17T22:01:00.319014+0000","last_active":"2024-05-17T22:01:03.461369+0000","last_peered":"2024-05-17T22:01:03.461369+0000","last_clean":"2024-05-17T22:01:03.461369+0000","last_became_active":"2024-05-17T22:01:00.311177+0000","last_became_peered":"2024-05-17T22:01:00.311177+0000","last_unstale":"2024-05-17T22:01:03.461369+0000","last_undegraded":"2024-05-17T22:01:03.461369+0000","last_fullsized":"2024-05-17T22:01:03.461369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:29:50.049155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.459270+0000","last_change":"2024-05-17T22:01:01.388434+0000","last_active":"2024-05-17T22:01:03.459270+0000","last_peered":"2024-05-17T22:01:03.459270+0000","last_clean":"2024-05-17T22:01:03.459270+0000","last_became_active":"2024-05-17T22:01:01.388045+0000","last_became_peered":"2024-05-17T22:01:01.388045+0000","last_unstale":"2024-05-17T22:01:03.459270+0000","last_undegraded":"2024-05-17T22:01:03.459270+0000","last_fullsized":"2024-05-17T22:01:03.459270+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:01:00.285509+0000","last_clean_scrub_stamp":"2024-05-17T22:01:00.285509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:50:36.694401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:03.461079+0000","last_change":"2024-05-17T22:01:00.319680+0000","last_active":"2024-05-17T22:01:03.461079+0000","last_peered":"2024-05-17T22:01:03.461079+0000","last_clean":"2024-05-17T22:01:03.461079+0000","last_became_active":"2024-05-17T22:01:00.311337+0000","last_became_peered":"2024-05-17T22:01:00.311337+0000","last_unstale":"2024-05-17T22:01:03.461079+0000","last_undegraded":"2024-05-17T22:01:03.461079+0000","last_fullsized":"2024-05-17T22:01:03.461079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:00:59.274513+0000","last_clean_scrub_stamp":"2024-05-17T22:00:59.274513+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:07:54.738653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":194,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-05-17T22:01:02.407834+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:01:02.407834+0000","last_peered":"2024-05-17T22:01:02.407834+0000","last_clean":"2024-05-17T22:01:02.407834+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:01:02.407834+0000","last_undegraded":"2024-05-17T22:01:02.407834+0000","last_fullsized":"2024-05-17T22:01:02.407834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1417216,"data_stored":932167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":341275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":334995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":827392,"data_stored":342795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":835584,"data_stored":343411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":931551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:01:05.831 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:01:05.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T22:01:06.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.020+0000 7f393fb72640 1 Processor -- start 2024-05-17T22:01:06.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 -- start start 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39380b08a0 0x7f393815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f39380b44c0 0x7f39380b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393806f2d0 con 0x7f393806eec0 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f393815c3b0 con 0x7f39380b44c0 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393fb72640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f393806f5a0 con 0x7f39380b08a0 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393e10f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f39380b44c0 0x7f39380b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393d90e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f393806eec0 0x7f39381601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40320/0 (socket says 172.21.15.83:40320) 2024-05-17T22:01:06.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393d90e640 1 -- 172.21.15.83:0/80964971 learned_addr learned my addr 172.21.15.83:0/80964971 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393d10d640 1 --2- 172.21.15.83:0/80964971 >> v2:172.21.15.155:3300/0 conn(0x7f39380b08a0 0x7f393815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393e10f640 1 -- 172.21.15.83:0/80964971 >> v2:172.21.15.155:3300/0 conn(0x7f39380b08a0 msgr2=0x7f393815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393e10f640 1 --2- 172.21.15.83:0/80964971 >> v2:172.21.15.155:3300/0 conn(0x7f39380b08a0 0x7f393815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393e10f640 1 -- 172.21.15.83:0/80964971 >> v1:172.21.15.83:6789/0 conn(0x7f393806eec0 legacy=0x7f39381601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393e10f640 1 -- 172.21.15.83:0/80964971 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393806f880 con 0x7f39380b44c0 2024-05-17T22:01:06.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.024+0000 7f393c90c640 1 -- 172.21.15.83:0/80964971 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931254995 0 0) 0x7f393806f2d0 con 0x7f393806eec0 2024-05-17T22:01:06.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393e10f640 1 --2- 172.21.15.83:0/80964971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b44c0 0x7f39380b48d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f39380740c0 tx=0x7f393001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681a09fdeaa59073 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393c90c640 1 -- 172.21.15.83:0/80964971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300b2070 con 0x7f39380b44c0 2024-05-17T22:01:06.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393c90c640 1 -- 172.21.15.83:0/80964971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39300a5850 con 0x7f39380b44c0 2024-05-17T22:01:06.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393c90c640 1 -- 172.21.15.83:0/80964971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300a5c20 con 0x7f39380b44c0 2024-05-17T22:01:06.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- 172.21.15.83:0/80964971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b44c0 msgr2=0x7f39380b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 --2- 172.21.15.83:0/80964971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b44c0 0x7f39380b48d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f39380740c0 tx=0x7f393001c8a0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- 172.21.15.83:0/80964971 shutdown_connections 2024-05-17T22:01:06.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 --2- 172.21.15.83:0/80964971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b44c0 0x7f39380b48d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 --2- 172.21.15.83:0/80964971 >> v2:172.21.15.155:3300/0 conn(0x7f39380b08a0 0x7f393815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- 172.21.15.83:0/80964971 >> 172.21.15.83:0/80964971 conn(0x7f393806cd90 msgr2=0x7f39380b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:06.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- 172.21.15.83:0/80964971 shutdown_connections 2024-05-17T22:01:06.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- 172.21.15.83:0/80964971 wait complete. 2024-05-17T22:01:06.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 Processor -- start 2024-05-17T22:01:06.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.028+0000 7f393fb72640 1 -- start start 2024-05-17T22:01:06.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 0x7f3938143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39380b44c0 0x7f3938158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3938160a40 con 0x7f39380b08a0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f39381608c0 con 0x7f393806eec0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f39380b38d0 con 0x7f39380b44c0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50104/0 (socket says 172.21.15.83:50104) 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 -- 172.21.15.83:0/522614197 learned_addr learned my addr 172.21.15.83:0/522614197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d90e640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 0x7f3938143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393e10f640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39380b44c0 0x7f3938158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 -- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39380b44c0 msgr2=0x7f3938158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39380b44c0 0x7f3938158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 -- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 msgr2=0x7f3938143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 0x7f3938143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 -- 172.21.15.83:0/522614197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3938161080 con 0x7f39380b08a0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d90e640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 0x7f3938143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d10d640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f392800d5c0 tx=0x7f392800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928004240 con 0x7f39380b08a0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39280043e0 con 0x7f39380b08a0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39380b1c70 con 0x7f39380b08a0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928003840 con 0x7f39380b08a0 2024-05-17T22:01:06.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393815aa20 con 0x7f39380b08a0 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39380b3c60 con 0x7f39380b08a0 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39280039e0 con 0x7f39380b08a0 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 0x7f391c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f393d90e640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 0x7f391c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.032+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f392808f4c0 con 0x7f39380b08a0 2024-05-17T22:01:06.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.036+0000 7f393d90e640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 0x7f391c071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3934007fd0 tx=0x7f3934009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.036+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f392805c440 con 0x7f39380b08a0 2024-05-17T22:01:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.248+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f39380b0f10 con 0x7f39380b08a0 2024-05-17T22:01:06.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.248+0000 7f392e7fc640 1 -- 172.21.15.83:0/522614197 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f392805bd40 con 0x7f39380b08a0 2024-05-17T22:01:06.242 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T22:01:06.243 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T22:01:06.243 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T22:01:06.243 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T22:01:06.243 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T22:01:06.243 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 msgr2=0x7f391c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 0x7f391c071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3934007fd0 tx=0x7f3934009040 comp rx=0 tx=0).stop 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 msgr2=0x7f3938158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f392800d5c0 tx=0x7f392800da80 comp rx=0 tx=0).stop 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 shutdown_connections 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39380b44c0 0x7f3938158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39380b08a0 0x7f3938158140 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f391c06f430 0x7f391c071940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 --2- 172.21.15.83:0/522614197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f393806eec0 0x7f3938143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 >> 172.21.15.83:0/522614197 conn(0x7f393806cd90 msgr2=0x7f39381578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:06.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 shutdown_connections 2024-05-17T22:01:06.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.252+0000 7f393fb72640 1 -- 172.21.15.83:0/522614197 wait complete. 2024-05-17T22:01:06.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 Processor -- start 2024-05-17T22:01:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 -- start start 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e815ffa0 0x7fc3e80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806e3d0 0x7fc3e80d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e80d34e0 con 0x7fc3e80d1150 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e80d3780 con 0x7fc3e815ffa0 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3f056e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e80d3a60 con 0x7fc3e806e3d0 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e815ffa0 0x7fc3e80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e815ffa0 0x7fc3e80d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50112/0 (socket says 172.21.15.83:50112) 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 -- 172.21.15.83:0/282663412 learned_addr learned my addr 172.21.15.83:0/282663412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3eeb0b640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806e3d0 0x7fc3e80d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 -- 172.21.15.83:0/282663412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806e3d0 msgr2=0x7fc3e80d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806e3d0 0x7fc3e80d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 -- 172.21.15.83:0/282663412 >> v1:172.21.15.155:6789/0 conn(0x7fc3e80d1150 legacy=0x7fc3e806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:06.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.416+0000 7fc3ee30a640 1 -- 172.21.15.83:0/282663412 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e80d3d40 con 0x7fc3e815ffa0 2024-05-17T22:01:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3ee30a640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815ffa0 0x7fc3e80d0c10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc3e808e380 tx=0x7fc3d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3feb64cbf22a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3ed308640 1 -- 172.21.15.83:0/282663412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80b0070 con 0x7fc3e815ffa0 2024-05-17T22:01:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3ed308640 1 -- 172.21.15.83:0/282663412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d80ab600 con 0x7fc3e815ffa0 2024-05-17T22:01:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3ed308640 1 -- 172.21.15.83:0/282663412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80ab9d0 con 0x7fc3e815ffa0 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 -- 172.21.15.83:0/282663412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815ffa0 msgr2=0x7fc3e80d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815ffa0 0x7fc3e80d0c10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc3e808e380 tx=0x7fc3d801c520 comp rx=0 tx=0).stop 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 -- 172.21.15.83:0/282663412 shutdown_connections 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e806e3d0 0x7fc3e80d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 --2- 172.21.15.83:0/282663412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815ffa0 0x7fc3e80d0c10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 -- 172.21.15.83:0/282663412 >> 172.21.15.83:0/282663412 conn(0x7fc3e80d28f0 msgr2=0x7fc3e80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 -- 172.21.15.83:0/282663412 shutdown_connections 2024-05-17T22:01:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.420+0000 7fc3f056e640 1 -- 172.21.15.83:0/282663412 wait complete. 2024-05-17T22:01:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 Processor -- start 2024-05-17T22:01:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- start start 2024-05-17T22:01:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e80d1150 0x7fc3e815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e815ffa0 0x7fc3e8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e80d5340 con 0x7fc3e815e900 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e80d51c0 con 0x7fc3e80d1150 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e80d54c0 con 0x7fc3e815ffa0 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50118/0 (socket says 172.21.15.83:50118) 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 -- 172.21.15.83:0/3427320693 learned_addr learned my addr 172.21.15.83:0/3427320693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3edb09640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e815ffa0 0x7fc3e8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 -- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e815ffa0 msgr2=0x7fc3e8156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e815ffa0 0x7fc3e8156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3ee30a640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e80d1150 0x7fc3e815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 -- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e80d1150 msgr2=0x7fc3e815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e80d1150 0x7fc3e815aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 -- 172.21.15.83:0/3427320693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e8159830 con 0x7fc3e815e900 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3eeb0b640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d550 tx=0x7fc3e000da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00125f0 con 0x7fc3e815e900 2024-05-17T22:01:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e0019070 con 0x7fc3e815e900 2024-05-17T22:01:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e8162260 con 0x7fc3e815e900 2024-05-17T22:01:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00129d0 con 0x7fc3e815e900 2024-05-17T22:01:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e8155890 con 0x7fc3e815e900 2024-05-17T22:01:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.424+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e815af60 con 0x7fc3e815e900 2024-05-17T22:01:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.428+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc3e0012b70 con 0x7fc3e815e900 2024-05-17T22:01:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.428+0000 7fc3e6ffd640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 0x7fc3d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.428+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fc3e0090160 con 0x7fc3e815e900 2024-05-17T22:01:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.428+0000 7fc3ee30a640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 0x7fc3d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:06.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.428+0000 7fc3ee30a640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 0x7fc3d4071910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80ad000 tx=0x7fc3d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:06.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.432+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc3e005d0c0 con 0x7fc3e815e900 2024-05-17T22:01:06.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.612+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e80d4260 con 0x7fc3e815e900 2024-05-17T22:01:06.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.612+0000 7fc3e6ffd640 1 -- 172.21.15.83:0/3427320693 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5186 (secure 0 0 0) 0x7fc3e005c9c0 con 0x7fc3e815e900 2024-05-17T22:01:06.604 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:06.605 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:03.754919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1850779527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1850779527},{"type":"v1","addr":"172.21.15.83:6837","nonce":1850779527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":5}]} 2024-05-17T22:01:06.605 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 43 2024-05-17T22:01:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 msgr2=0x7fc3d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 0x7fc3d4071910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80ad000 tx=0x7fc3d8008040 comp rx=0 tx=0).stop 2024-05-17T22:01:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 msgr2=0x7fc3e815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d550 tx=0x7fc3e000da10 comp rx=0 tx=0).stop 2024-05-17T22:01:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 shutdown_connections 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e815e900 0x7fc3e815ed10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e815ffa0 0x7fc3e8156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d406f400 0x7fc3d4071910 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 --2- 172.21.15.83:0/3427320693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e80d1150 0x7fc3e815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 >> 172.21.15.83:0/3427320693 conn(0x7fc3e80d28f0 msgr2=0x7fc3e80b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 shutdown_connections 2024-05-17T22:01:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:06.616+0000 7fc3f056e640 1 -- 172.21.15.83:0/3427320693 wait complete. 2024-05-17T22:01:07.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:01:00.724476+0000', 'modified': '2024-05-17T22:01:03.754919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6072}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6072': {'gid': 6072, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/1850779527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1850779527}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1850779527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6072, 'qdb_cluster': [6072]} 2024-05-17T22:01:07.028 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:01:07.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:01:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.196+0000 7fa7af799640 1 Processor -- start 2024-05-17T22:01:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.196+0000 7fa7af799640 1 -- start start 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7af799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7a815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7af799640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a80b5640 con 0x7fa7a815d530 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7af799640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a80b58e0 con 0x7fa7a80cf830 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7af799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a8070870 con 0x7fa7a80cdf40 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7add36640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa7a815d530 0x7fa7a81663d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43636/0 (socket says 172.21.15.83:43636) 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7add36640 1 -- 172.21.15.83:0/2852783795 learned_addr learned my addr 172.21.15.83:0/2852783795 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7a815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398327268 0 0) 0x7fa7a80b5640 con 0x7fa7a815d530 2024-05-17T22:01:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa794003530 con 0x7fa7a815d530 2024-05-17T22:01:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1140863099 0 0) 0x7fa794003530 con 0x7fa7a815d530 2024-05-17T22:01:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 msgr2=0x7fa7a815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7a815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 >> v1:172.21.15.155:6789/0 conn(0x7fa7a80cf830 legacy=0x7fa7a80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a806f060 con 0x7fa7a815d530 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fa7a0001aa0 con 0x7fa7a815d530 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 >> v1:172.21.15.83:6789/0 conn(0x7fa7a815d530 legacy=0x7fa7a81663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7940039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa794003fa0 0x7fa794004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794004d50 0x7fa7940051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa79400bb60 con 0x7fa794003fa0 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa79400bd10 con 0x7fa794004d50 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa79400bf10 con 0x7fa7a80cdf40 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa794003fa0 0x7fa794004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7940039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 -- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 msgr2=0x7fa7940039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a80cdf40 0x7fa7940039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7add36640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794004d50 0x7fa7940051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 -- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794004d50 msgr2=0x7fa7940051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794004d50 0x7fa7940051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.200+0000 7fa7acd34640 1 -- 172.21.15.83:0/2852783795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa794003530 con 0x7fa794003fa0 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa794003fa0 0x7fa794004810 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ab0 tx=0x7fa79801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31566381781ef41 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2070 con 0x7fa794003fa0 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980a5660 con 0x7fa794003fa0 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa79ffff640 1 -- 172.21.15.83:0/2852783795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980096b0 con 0x7fa794003fa0 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- 172.21.15.83:0/2852783795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa794003fa0 msgr2=0x7fa794004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 --2- 172.21.15.83:0/2852783795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa794003fa0 0x7fa794004810 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ab0 tx=0x7fa79801ca70 comp rx=0 tx=0).stop 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7add36640 1 -- 172.21.15.83:0/2852783795 reap_dead start 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- 172.21.15.83:0/2852783795 shutdown_connections 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- 172.21.15.83:0/2852783795 >> 172.21.15.83:0/2852783795 conn(0x7fa7a806cd90 msgr2=0x7fa7a80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- 172.21.15.83:0/2852783795 shutdown_connections 2024-05-17T22:01:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- 172.21.15.83:0/2852783795 wait complete. 2024-05-17T22:01:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 Processor -- start 2024-05-17T22:01:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.204+0000 7fa7af799640 1 -- start start 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794003fa0 0x7fa7a8144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a815f260 0x7fa7a815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a815dac0 con 0x7fa7a8157c60 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a815d940 con 0x7fa794003fa0 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a815dc40 con 0x7fa7a815f260 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39690/0 (socket says 172.21.15.83:39690) 2024-05-17T22:01:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 -- 172.21.15.83:0/610974923 learned_addr learned my addr 172.21.15.83:0/610974923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7acd34640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a815f260 0x7fa7a815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7ad535640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794003fa0 0x7fa7a8144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 -- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a815f260 msgr2=0x7fa7a815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a815f260 0x7fa7a815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 -- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794003fa0 msgr2=0x7fa7a8144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794003fa0 0x7fa7a8144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 -- 172.21.15.83:0/610974923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a81585b0 con 0x7fa7a8157c60 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7add36640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000f90 tx=0x7fa7a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 -- 172.21.15.83:0/610974923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0004d70 con 0x7fa7a8157c60 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 -- 172.21.15.83:0/610974923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a000bb20 con 0x7fa7a8157c60 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a8154480 con 0x7fa7a8157c60 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 -- 172.21.15.83:0/610974923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0007070 con 0x7fa7a8157c60 2024-05-17T22:01:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a8154a40 con 0x7fa7a8157c60 2024-05-17T22:01:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 -- 172.21.15.83:0/610974923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa7a001b020 con 0x7fa7a8157c60 2024-05-17T22:01:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 0x7fa78c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa79dffb640 1 -- 172.21.15.83:0/610974923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fa7a008f3c0 con 0x7fa7a8157c60 2024-05-17T22:01:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.208+0000 7fa7ad535640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 0x7fa78c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7ad535640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 0x7fa78c071910 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000f70 tx=0x7fa7a4000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 Processor -- start 2024-05-17T22:01:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- start start 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f17978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f1797df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618f1635ca0 con 0x5618f1630d30 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618f15a8110 con 0x5618f1651fa0 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5618f1597cb0 con 0x5618f1647600 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39700/0 (socket says 172.21.15.83:39700) 2024-05-17T22:01:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 -- 172.21.15.83:0/2090963175 learned_addr learned my addr 172.21.15.83:0/2090963175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f1797df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f17978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 -- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 msgr2=0x5618f1797df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f1797df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 -- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 msgr2=0x5618f17978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f17978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 -- 172.21.15.83:0/2090963175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f155c370 con 0x5618f1630d30 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f17978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7add36640 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0001e00 tx=0x7fa7a00583a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcebd9b48cab7dc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa792ffd640 1 -- 172.21.15.83:0/2090963175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0064340 con 0x5618f1630d30 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa792ffd640 1 -- 172.21.15.83:0/2090963175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a005ffd0 con 0x5618f1630d30 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa792ffd640 1 -- 172.21.15.83:0/2090963175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00649f0 con 0x5618f1630d30 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2090963175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 msgr2=0x5618f1798330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0001e00 tx=0x7fa7a00583a0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2090963175 shutdown_connections 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f1798330 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f1797df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2090963175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f17978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2090963175 >> 172.21.15.83:0/2090963175 conn(0x5618f1259460 msgr2=0x5618f174ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2090963175 shutdown_connections 2024-05-17T22:01:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.212+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2090963175 wait complete. 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- start start 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 Processor -- start 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1630d30 0x5618f17542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f1759460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618f1553d40 con 0x5618f1647600 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618f159bcf0 con 0x5618f1651fa0 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5618f1540e90 con 0x5618f1630d30 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 10 client.0 _ll_register_callbacks cb 0x5618f1575fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:07.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39708/0 (socket says 172.21.15.83:39708) 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 -- 172.21.15.83:0/890966335 learned_addr learned my addr 172.21.15.83:0/890966335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7ad535640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1630d30 0x5618f17542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7add36640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f1759460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 -- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1630d30 msgr2=0x5618f17542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1630d30 0x5618f17542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 -- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 msgr2=0x5618f1759460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f1759460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 -- 172.21.15.83:0/890966335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f15666e0 con 0x5618f1647600 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7add36640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f1759460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7acd34640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ab0 tx=0x7fa798019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee93c1aa93053f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2040 con 0x5618f1647600 2024-05-17T22:01:07.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fa7980c2aa0 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a005ffd0 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa7980c9050 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 10 client.6189 fetch_fsmap learned FSMap version 43 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5618f1556690 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a005ffd0 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fa7980cc020 con 0x5618f1647600 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 10 client.6189 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 10 client.6189 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 20 client.6189 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f123e7d0 0x5618f179e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 4 client.6189 mds_command: new command op to 6072 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 --> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f1533a00 con 0x5618f123e7d0 2024-05-17T22:01:07.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7ad535640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f123e7d0 0x5618f179e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa787fff640 10 client.6189 ms_handle_connect on v2:172.21.15.83:6836/1850779527 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.216+0000 7fa7ad535640 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f123e7d0 0x5618f179e920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad11aec993906176 server_cookie=5c12ce43bcfdd0b3 in_seq=0 out_seq=0 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa787fff640 1 -- 172.21.15.83:0/890966335 <== mds.0 v2:172.21.15.83:6836/1850779527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618f1533a00 con 0x5618f123e7d0 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa787fff640 10 client.6189 handle_command_reply: tid=0 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 client.6189 shutdown 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 msgr2=0x5618f1754810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ab0 tx=0x7fa798019810 comp rx=0 tx=0).stop 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 shutdown_connections 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f123e7d0 0x5618f179e920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1651fa0 0x5618f1759460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa787fff640 0 client.6189 ms_handle_reset on v2:172.21.15.83:6836/1850779527 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1647600 0x5618f1754810 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/890966335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1630d30 0x5618f17542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 >> 172.21.15.83:0/890966335 conn(0x5618f1259460 msgr2=0x5618f174ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 shutdown_connections 2024-05-17T22:01:07.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 1 -- 172.21.15.83:0/890966335 wait complete. 2024-05-17T22:01:07.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.220+0000 7fa7b6a13000 20 client.6189 trim_cache size 0 max 16384 2024-05-17T22:01:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.236+0000 7fa7b6a13000 1 Processor -- start 2024-05-17T22:01:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.236+0000 7fa7b6a13000 1 -- start start 2024-05-17T22:01:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1647600 0x5618f1753280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1651fa0 0x5618f17537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81fd500 con 0x5618f1630d30 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81f24b0 con 0x5618f1647600 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a81f2b00 con 0x5618f1651fa0 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39724/0 (socket says 172.21.15.83:39724) 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 -- 172.21.15.83:0/667747950 learned_addr learned my addr 172.21.15.83:0/667747950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 -- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1651fa0 msgr2=0x5618f17537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1651fa0 0x5618f17537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 -- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1647600 msgr2=0x5618f1753280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1647600 0x5618f1753280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 -- 172.21.15.83:0/667747950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f161ecb0 con 0x5618f1630d30 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7ad535640 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005300 tx=0x7fa7a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7301b9fa20ac6e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7927fc640 1 -- 172.21.15.83:0/667747950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a401c040 con 0x5618f1630d30 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7927fc640 1 -- 172.21.15.83:0/667747950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4016480 con 0x5618f1630d30 2024-05-17T22:01:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7927fc640 1 -- 172.21.15.83:0/667747950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4016830 con 0x5618f1630d30 2024-05-17T22:01:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- 172.21.15.83:0/667747950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 msgr2=0x5618f174fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4005300 tx=0x7fa7a4015000 comp rx=0 tx=0).stop 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- 172.21.15.83:0/667747950 shutdown_connections 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1651fa0 0x5618f17537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1647600 0x5618f1753280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/667747950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1630d30 0x5618f174fac0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- 172.21.15.83:0/667747950 >> 172.21.15.83:0/667747950 conn(0x5618f1259460 msgr2=0x5618f17adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- 172.21.15.83:0/667747950 shutdown_connections 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- 172.21.15.83:0/667747950 wait complete. 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 -- start start 2024-05-17T22:01:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.240+0000 7fa7b6a13000 1 Processor -- start 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1630d30 0x5618f180eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f180f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618f1566c10 con 0x5618f1651fa0 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5618f1757c20 con 0x5618f1630d30 2024-05-17T22:01:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5618f161e820 con 0x5618f1647600 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 10 client.0 _ll_register_callbacks cb 0x7fa794003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39734/0 (socket says 172.21.15.83:39734) 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 -- 172.21.15.83:0/2896460911 learned_addr learned my addr 172.21.15.83:0/2896460911 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1630d30 0x5618f180eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7acd34640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f180f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 -- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 msgr2=0x5618f180f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f180f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 -- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1630d30 msgr2=0x5618f180eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1630d30 0x5618f180eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 -- 172.21.15.83:0/2896460911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f15574f0 con 0x5618f1651fa0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7add36640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0058e90 tx=0x7fa7a008d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4566219b565c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0058270 con 0x5618f1651fa0 2024-05-17T22:01:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fa7a0064760 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a4016480 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa7a0064340 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 10 client.6195 fetch_fsmap learned FSMap version 43 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5618f15614f0 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4016480 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fa7a005ae80 con 0x5618f1651fa0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 10 client.6195 fetch_fsmap finished waiting for FSMap version 43 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 10 client.6195 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 20 client.6195 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f122c830 0x5618f174e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 4 client.6195 mds_command: new command op to 6072 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 --> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] -- command(tid 0: {"prefix": "status"}) -- 0x5618f15aa060 con 0x5618f122c830 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f122c830 0x5618f174e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 10 client.6195 ms_handle_connect on v2:172.21.15.83:6836/1850779527 2024-05-17T22:01:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7ad535640 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f122c830 0x5618f174e8a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6e080d9bef96f7 server_cookie=28241b421463538d in_seq=0 out_seq=0 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 1 -- 172.21.15.83:0/2896460911 <== mds.0 v2:172.21.15.83:6836/1850779527 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5618f15aa060 con 0x5618f122c830 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 10 client.6195 handle_command_reply: tid=0 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 client.6195 shutdown 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 msgr2=0x5618f180f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0058e90 tx=0x7fa7a008d7a0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 shutdown_connections 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:6836/1850779527,v1:172.21.15.83:6837/1850779527] conn(0x5618f122c830 0x5618f174e8a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618f1651fa0 0x5618f180f930 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa787fff640 0 client.6195 ms_handle_reset on v2:172.21.15.83:6836/1850779527 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5618f1647600 0x5618f180f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 --2- 172.21.15.83:0/2896460911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618f1630d30 0x5618f180eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 >> 172.21.15.83:0/2896460911 conn(0x5618f1259460 msgr2=0x5618f1797180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 shutdown_connections 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 1 -- 172.21.15.83:0/2896460911 wait complete. 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.244+0000 7fa7b6a13000 20 client.6195 trim_cache size 0 max 16384 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6072, 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:01:07.243 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.9309369109999999, 2024-05-17T22:01:07.244 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 43, 2024-05-17T22:01:07.244 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 67, 2024-05-17T22:01:07.244 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 67, 2024-05-17T22:01:07.244 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.885494045 2024-05-17T22:01:07.244 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 msgr2=0x7fa78c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 0x7fa78c071910 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4000f70 tx=0x7fa7a4000ee0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 msgr2=0x7fa7a8158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000f90 tx=0x7fa7a000b4a0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 shutdown_connections 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7a8157c60 0x7fa7a8158070 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7a815f260 0x7fa7a815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78c06f400 0x7fa78c071910 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 --2- 172.21.15.83:0/610974923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa794003fa0 0x7fa7a8144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 >> 172.21.15.83:0/610974923 conn(0x7fa7a806cd90 msgr2=0x7fa7a8072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 shutdown_connections 2024-05-17T22:01:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.248+0000 7fa7af799640 1 -- 172.21.15.83:0/610974923 wait complete. 2024-05-17T22:01:07.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:01:07.272 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:01:07.280+0000 7f35e6a3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:01:07.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 Processor -- start 2024-05-17T22:01:07.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 -- start start 2024-05-17T22:01:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3d480afef0 0x7f3d480b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d480b0300 con 0x7f3d480b5390 2024-05-17T22:01:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d480b05b0 con 0x7f3d480b3030 2024-05-17T22:01:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d5053a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3d480b11c0 con 0x7f3d480afef0 2024-05-17T22:01:07.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4dad5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3d480b5390 0x7f3d480b57a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43648/0 (socket says 172.21.15.83:43648) 2024-05-17T22:01:07.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4dad5640 1 -- 172.21.15.83:0/3886407405 learned_addr learned my addr 172.21.15.83:0/3886407405 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677585233 0 0) 0x7f3d480b0300 con 0x7f3d480b5390 2024-05-17T22:01:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4e2d6640 1 --2- 172.21.15.83:0/3886407405 >> v2:172.21.15.155:3300/0 conn(0x7f3d480afef0 0x7f3d480b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d30005740 con 0x7f3d480b5390 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2434217528 0 0) 0x7f3d30005740 con 0x7f3d480b5390 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 >> v2:172.21.15.155:3300/0 conn(0x7f3d480afef0 msgr2=0x7f3d480b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 --2- 172.21.15.83:0/3886407405 >> v2:172.21.15.155:3300/0 conn(0x7f3d480afef0 0x7f3d480b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 >> v1:172.21.15.155:6789/0 conn(0x7f3d480b3030 legacy=0x7f3d481665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d480b0890 con 0x7f3d480b5390 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f3d44001700 con 0x7f3d480b5390 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 >> v1:172.21.15.83:6789/0 conn(0x7f3d480b5390 legacy=0x7f3d480b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d30005c70 0x7f3d300060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d30006a20 0x7f3d30006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d3000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480b11c0 con 0x7f3d30007570 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3000e4a0 con 0x7f3d30005c70 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.408+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3000e6a0 con 0x7f3d30006a20 2024-05-17T22:01:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d3000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4e2d6640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d30005c70 0x7f3d300060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4dad5640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d30006a20 0x7f3d30006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 -- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d30006a20 msgr2=0x7f3d30006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d30006a20 0x7f3d30006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 -- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d30005c70 msgr2=0x7f3d300060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d30005c70 0x7f3d300060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 -- 172.21.15.83:0/3886407405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d30005740 con 0x7f3d30007570 2024-05-17T22:01:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 -- 172.21.15.83:0/3886407405 reap_dead start 2024-05-17T22:01:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4ead7640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d3000dbe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d48072e30 tx=0x7f3d3801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1c0d66b0f9ff6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380013e0 con 0x7f3d30007570 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d380052d0 con 0x7f3d30007570 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d4d2d4640 1 -- 172.21.15.83:0/3886407405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003840 con 0x7f3d30007570 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 -- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 msgr2=0x7f3d3000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d3000dbe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d48072e30 tx=0x7f3d3801c350 comp rx=0 tx=0).stop 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 -- 172.21.15.83:0/3886407405 shutdown_connections 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 --2- 172.21.15.83:0/3886407405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d3000dbe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 -- 172.21.15.83:0/3886407405 >> 172.21.15.83:0/3886407405 conn(0x7f3d4809b8b0 msgr2=0x7f3d480b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 -- 172.21.15.83:0/3886407405 shutdown_connections 2024-05-17T22:01:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.412+0000 7f3d5053a640 1 -- 172.21.15.83:0/3886407405 wait complete. 2024-05-17T22:01:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 Processor -- start 2024-05-17T22:01:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- start start 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4815bf80 0x7f3d481f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d481f9700 0x7f3d48140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480b4950 con 0x7f3d30007570 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3d480b0e50 con 0x7f3d4815bf80 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3d480b4ad0 con 0x7f3d481f9700 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39756/0 (socket says 172.21.15.83:39756) 2024-05-17T22:01:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 -- 172.21.15.83:0/888557236 learned_addr learned my addr 172.21.15.83:0/888557236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4ead7640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d481f9700 0x7f3d48140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4dad5640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4815bf80 0x7f3d481f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 -- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d481f9700 msgr2=0x7f3d48140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d481f9700 0x7f3d48140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 -- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4815bf80 msgr2=0x7f3d481f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4815bf80 0x7f3d481f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 -- 172.21.15.83:0/888557236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481f9b10 con 0x7f3d30007570 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d4e2d6640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009f40 tx=0x7f3d400115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4001a020 con 0x7f3d30007570 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d40016470 con 0x7f3d30007570 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d48154690 con 0x7f3d30007570 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40016840 con 0x7f3d30007570 2024-05-17T22:01:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.416+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d48154c50 con 0x7f3d30007570 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d480b4e60 con 0x7f3d30007570 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3d4001d020 con 0x7f3d30007570 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d3effd640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 0x7f3d34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f3d4008ff80 con 0x7f3d30007570 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d4dad5640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 0x7f3d34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:07.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.420+0000 7f3d4dad5640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 0x7f3d34071940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3d44001d80 tx=0x7f3d44001c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.424+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3d40024030 con 0x7f3d30007570 2024-05-17T22:01:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.604+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3d480b8400 con 0x7f3d30007570 2024-05-17T22:01:07.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d3effd640 1 -- 172.21.15.83:0/888557236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f3d481f9b10 con 0x7f3d30007570 2024-05-17T22:01:07.599 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 msgr2=0x7f3d34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 0x7f3d34071940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3d44001d80 tx=0x7f3d44001c90 comp rx=0 tx=0).stop 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 msgr2=0x7f3d48153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009f40 tx=0x7f3d400115a0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 shutdown_connections 2024-05-17T22:01:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d481f9700 0x7f3d48140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3d3406f430 0x7f3d34071940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d4815bf80 0x7f3d481f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 --2- 172.21.15.83:0/888557236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d30007570 0x7f3d48153e40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 >> 172.21.15.83:0/888557236 conn(0x7f3d4809b8b0 msgr2=0x7f3d4806ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 shutdown_connections 2024-05-17T22:01:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07.608+0000 7f3d5053a640 1 -- 172.21.15.83:0/888557236 wait complete. 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:01:07.619 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:07.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:07.619 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:07.667 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.668 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:07.669 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:01:07.670 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:07.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:07.670 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:01:07.721 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:01:07.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:07.722 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:01:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:07.777 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:07.777 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:07.777 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:01:07.777 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:01:07.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:01:07.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:01:07.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:07.896 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:01:07.943 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:01:07.943 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:01:07.944 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:07.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:07.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:01:07.996 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:01:07.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:01:07.997 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:08.043 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:08.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:01:08.043 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:08.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:08.152 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:08.200 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:08.201 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:08.202 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:08.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:08.203 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:08.251 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:01:08.251 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:01:08.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:08.310 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:08.359 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:08.359 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:08.359 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:08.359 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:08.359 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:08.360 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:08.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:08.362 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:08.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:01:08.412+0000 7fae45383580 -1 init, newargv = 0x7fae340a9070 newargc=15 2024-05-17T22:01:08.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41183]: starting ceph client 2024-05-17T22:01:08.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41183]: starting fuse 2024-05-17T22:01:09.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:09.524 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:09.572 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:09.573 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:09.574 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:09.575 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:09.575 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:09.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:09.576 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:09.624 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:01:09.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:01:09.624 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:01:09.625 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:09.626 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:01:09.626 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:01:09.626 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:01:09.626 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:01:09.626 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:01:09.627 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:01:09.745 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.41183.asok 2024-05-17T22:01:09.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:01:09.757 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41183.asok 2024-05-17T22:01:09.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:09.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41183.asok status 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "41183", 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:01:09.933 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6204, 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6204 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2916074842 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:01:09.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2916074842 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6204 192.168.0.1:0/2916074842", 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2916074842", 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 43, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 67, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:01:09.935 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:09.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:09.949 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:01:09.949 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:09.955 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:01:09.955 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:09.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:09.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:10.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:10.012 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-17T22:01:10.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:10.068 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-17T22:01:10.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:10.122 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-17T22:01:10.177 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:01:10.177 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:01:10.177 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:01:10.177 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:10.178 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-17T22:01:10.178 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:01:10.285 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:01:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:01:10.296 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:01:10.296 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:01:10.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:01:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.548+0000 7fadfbd83640 1 Processor -- start 2024-05-17T22:01:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.548+0000 7fadfbd83640 1 -- start start 2024-05-17T22:01:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadfbd83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fadf415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadfbd83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf40b5640 con 0x7fadf40cf830 2024-05-17T22:01:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadfbd83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf40b58e0 con 0x7fadf415d530 2024-05-17T22:01:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadfbd83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fadf4070870 con 0x7fadf40cdf40 2024-05-17T22:01:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fadf40cf830 0x7fadf40b3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43664/0 (socket says 172.21.15.83:43664) 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 -- 172.21.15.83:0/453201822 learned_addr learned my addr 172.21.15.83:0/453201822 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf931e640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fadf415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060906966 0 0) 0x7fadf40b5640 con 0x7fadf40cf830 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fade0003530 con 0x7fadf40cf830 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2909198047 0 0) 0x7fade0003530 con 0x7fadf40cf830 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 msgr2=0x7fadf415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fadf415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 >> v1:172.21.15.155:6789/0 conn(0x7fadf415d530 legacy=0x7fadf41663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf406f060 con 0x7fadf40cf830 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fadf0001720 con 0x7fadf40cf830 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 >> v1:172.21.15.83:6789/0 conn(0x7fadf40cf830 legacy=0x7fadf40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fade0003a40 0x7fade0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fade00052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fade000bb60 con 0x7fade0003a40 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fade000bd10 con 0x7fade0004810 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fade000bf10 con 0x7fadf40cdf40 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fade0003a40 0x7fade0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf931e640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadfa320640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fade00052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 -- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 msgr2=0x7fade00052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadf40cdf40 0x7fade00052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 -- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fade0004810 msgr2=0x7fade0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fade0004810 0x7fade0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.552+0000 7fadf9b1f640 1 -- 172.21.15.83:0/453201822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade0003530 con 0x7fade0003a40 2024-05-17T22:01:10.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fade0003a40 0x7fade0003eb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c210 tx=0x7fadf001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7eed77a636d39c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007d60 con 0x7fade0003a40 2024-05-17T22:01:10.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf00a9020 con 0x7fade0003a40 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadf8b1d640 1 -- 172.21.15.83:0/453201822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0005b20 con 0x7fade0003a40 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 -- 172.21.15.83:0/453201822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fade0003a40 msgr2=0x7fade0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 --2- 172.21.15.83:0/453201822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fade0003a40 0x7fade0003eb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c210 tx=0x7fadf001cba0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfa320640 1 -- 172.21.15.83:0/453201822 reap_dead start 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 -- 172.21.15.83:0/453201822 shutdown_connections 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 -- 172.21.15.83:0/453201822 >> 172.21.15.83:0/453201822 conn(0x7fadf406cd90 msgr2=0x7fadf40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 -- 172.21.15.83:0/453201822 shutdown_connections 2024-05-17T22:01:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.556+0000 7fadfbd83640 1 -- 172.21.15.83:0/453201822 wait complete. 2024-05-17T22:01:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 Processor -- start 2024-05-17T22:01:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- start start 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fade0003a40 0x7fadf414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 0x7fadf40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4168690 con 0x7fadf4165a40 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4168510 con 0x7fadf41672d0 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fadf4168810 con 0x7fade0003a40 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39782/0 (socket says 172.21.15.83:39782) 2024-05-17T22:01:10.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 -- 172.21.15.83:0/853852954 learned_addr learned my addr 172.21.15.83:0/853852954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fade0003a40 0x7fadf414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfa320640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 0x7fadf40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 -- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fade0003a40 msgr2=0x7fadf414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fade0003a40 0x7fadf414be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 -- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 msgr2=0x7fadf40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 0x7fadf40ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 -- 172.21.15.83:0/853852954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf41676e0 con 0x7fadf4165a40 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfa320640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 0x7fadf40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadf931e640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fade40115c0 tx=0x7fade4011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4004220 con 0x7fadf4165a40 2024-05-17T22:01:10.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade40043c0 con 0x7fadf4165a40 2024-05-17T22:01:10.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf41649a0 con 0x7fadf4165a40 2024-05-17T22:01:10.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade400f8f0 con 0x7fadf4165a40 2024-05-17T22:01:10.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.560+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf4164f60 con 0x7fadf4165a40 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf40b3aa0 con 0x7fadf4165a40 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fade400fa90 con 0x7fadf4165a40 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadea7fc640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 0x7fadd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 0x7fadd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fade408f4e0 con 0x7fadf4165a40 2024-05-17T22:01:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.564+0000 7fadf9b1f640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 0x7fadd8071940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fadf00aa000 tx=0x7fadf0001a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.568+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fadf41676e0 con 0x7fadf4165a40 2024-05-17T22:01:10.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.740+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fadf4164730 con 0x7fadf4165a40 2024-05-17T22:01:10.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.740+0000 7fadea7fc640 1 -- 172.21.15.83:0/853852954 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7fade4019030 con 0x7fadf4165a40 2024-05-17T22:01:10.732 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 msgr2=0x7fadd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 0x7fadd8071940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fadf00aa000 tx=0x7fadf0001a20 comp rx=0 tx=0).stop 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 msgr2=0x7fadf4166d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fade40115c0 tx=0x7fade4011a80 comp rx=0 tx=0).stop 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 shutdown_connections 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fadd806f430 0x7fadd8071940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf41672d0 0x7fadf40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf4165a40 0x7fadf4166d90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 --2- 172.21.15.83:0/853852954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fade0003a40 0x7fadf414be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 >> 172.21.15.83:0/853852954 conn(0x7fadf406cd90 msgr2=0x7fadf40b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:10.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 shutdown_connections 2024-05-17T22:01:10.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.744+0000 7fadfbd83640 1 -- 172.21.15.83:0/853852954 wait complete. 2024-05-17T22:01:10.754 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:01:10.754 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:01:10.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:01:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 Processor -- start 2024-05-17T22:01:10.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 -- start start 2024-05-17T22:01:10.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d600ce610 0x7f7d600cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d601665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d600d0c20 con 0x7f7d600b7570 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7d600cda70 con 0x7f7d600ce610 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d65a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d600d0f00 con 0x7f7d600cef60 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7d600b7570 0x7f7d600cdf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:43674/0 (socket says 172.21.15.83:43674) 2024-05-17T22:01:10.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 -- 172.21.15.83:0/1076590631 learned_addr learned my addr 172.21.15.83:0/1076590631 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5ffff640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d601665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5effd640 1 --2- 172.21.15.83:0/1076590631 >> v2:172.21.15.155:3300/0 conn(0x7f7d600ce610 0x7f7d600cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603453685 0 0) 0x7f7d600d0c20 con 0x7f7d600b7570 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d48003530 con 0x7f7d600b7570 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1697975441 0 0) 0x7f7d48003530 con 0x7f7d600b7570 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 msgr2=0x7f7d601665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d601665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 >> v2:172.21.15.155:3300/0 conn(0x7f7d600ce610 msgr2=0x7f7d600cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 --2- 172.21.15.83:0/1076590631 >> v2:172.21.15.155:3300/0 conn(0x7f7d600ce610 0x7f7d600cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d600d11e0 con 0x7f7d600b7570 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f7d50001700 con 0x7f7d600b7570 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 >> v1:172.21.15.83:6789/0 conn(0x7f7d600b7570 legacy=0x7f7d600cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d48003a10 0x7f7d48003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48004810 0x7f7d48004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d48005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d600cda70 con 0x7f7d48003a10 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d600d0f00 con 0x7f7d48004810 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4800ba80 con 0x7f7d600cef60 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d48003a10 0x7f7d48003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5ffff640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d48005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5effd640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48004810 0x7f7d48004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 -- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 msgr2=0x7f7d48005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d600cef60 0x7f7d48005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 -- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48004810 msgr2=0x7f7d48004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d48004810 0x7f7d48004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.964+0000 7f7d5f7fe640 1 -- 172.21.15.83:0/1076590631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48003530 con 0x7f7d48003a10 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d48003a10 0x7f7d48003e80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c2c0 tx=0x7f7d5001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ddb42ab1486b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50007d60 con 0x7f7d48003a10 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d500a9070 con 0x7f7d48003a10 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d5e7fc640 1 -- 172.21.15.83:0/1076590631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50005d40 con 0x7f7d48003a10 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 -- 172.21.15.83:0/1076590631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d48003a10 msgr2=0x7f7d48003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 --2- 172.21.15.83:0/1076590631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d48003a10 0x7f7d48003e80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c2c0 tx=0x7f7d5001cca0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d5ffff640 1 -- 172.21.15.83:0/1076590631 reap_dead start 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 -- 172.21.15.83:0/1076590631 shutdown_connections 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 -- 172.21.15.83:0/1076590631 >> 172.21.15.83:0/1076590631 conn(0x7f7d600b4170 msgr2=0x7f7d600b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 -- 172.21.15.83:0/1076590631 shutdown_connections 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.968+0000 7f7d65a71640 1 -- 172.21.15.83:0/1076590631 wait complete. 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 Processor -- start 2024-05-17T22:01:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- start start 2024-05-17T22:01:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d60152e50 0x7f7d6014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d600b3190 0x7f7d6009f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60166f80 con 0x7f7d600af450 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60166e00 con 0x7f7d600b3190 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7d60167100 con 0x7f7d60152e50 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39798/0 (socket says 172.21.15.83:39798) 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 -- 172.21.15.83:0/209805895 learned_addr learned my addr 172.21.15.83:0/209805895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d60152e50 0x7f7d6014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 -- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d60152e50 msgr2=0x7f7d6014c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d60152e50 0x7f7d6014c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 -- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d600b3190 msgr2=0x7f7d6009f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5ffff640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d600b3190 0x7f7d6009f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d600b3190 0x7f7d6009f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 -- 172.21.15.83:0/209805895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6014cac0 con 0x7f7d600af450 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d5effd640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d54011590 tx=0x7f7d54011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54004220 con 0x7f7d600af450 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d540043c0 con 0x7f7d600af450 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d600b35a0 con 0x7f7d600af450 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54015930 con 0x7f7d600af450 2024-05-17T22:01:10.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d600aff80 con 0x7f7d600af450 2024-05-17T22:01:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.972+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600cea20 con 0x7f7d600af450 2024-05-17T22:01:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.976+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d54015ad0 con 0x7f7d600af450 2024-05-17T22:01:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.976+0000 7f7d47fff640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 0x7f7d4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.976+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f7d5408f8a0 con 0x7f7d600af450 2024-05-17T22:01:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.976+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 0x7f7d4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.976+0000 7f7d5f7fe640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 0x7f7d4c071940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c2c0 tx=0x7f7d500b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:10.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:10.980+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7d54019030 con 0x7f7d600af450 2024-05-17T22:01:11.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.160+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d6006d680 con 0x7f7d600af450 2024-05-17T22:01:11.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.160+0000 7f7d47fff640 1 -- 172.21.15.83:0/209805895 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) ==== 80+0+0 (secure 0 0 0) 0x7f7d5405c8b0 con 0x7f7d600af450 2024-05-17T22:01:11.153 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 msgr2=0x7f7d4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 0x7f7d4c071940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c2c0 tx=0x7f7d500b0040 comp rx=0 tx=0).stop 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 msgr2=0x7f7d600b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d54011590 tx=0x7f7d54011a50 comp rx=0 tx=0).stop 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 shutdown_connections 2024-05-17T22:01:11.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7d4c06f430 0x7f7d4c071940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d600b3190 0x7f7d6009f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d600af450 0x7f7d600b2c50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 --2- 172.21.15.83:0/209805895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d60152e50 0x7f7d6014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 >> 172.21.15.83:0/209805895 conn(0x7f7d600b4170 msgr2=0x7f7d6006c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 shutdown_connections 2024-05-17T22:01:11.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:11.164+0000 7f7d65a71640 1 -- 172.21.15.83:0/209805895 wait complete. 2024-05-17T22:01:11.174 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:01:11.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:11.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:11.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41183]: fuse finished with error 0 and tester_r 0 2024-05-17T22:01:11.187 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:01:17.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:01:17.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:17.289 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:01:17.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 Processor -- start 2024-05-17T22:01:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 -- start start 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 0x7fee380b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee3806c790 0x7fee3806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee380b3f20 con 0x7fee3815ec80 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee3815e7f0 con 0x7fee3806c790 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3ceb4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee3806d0e0 con 0x7fee380af3e0 2024-05-17T22:01:17.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee35d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fee3815ec80 0x7fee380b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35100/0 (socket says 172.21.15.83:35100) 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee35d9b640 1 -- 172.21.15.83:0/1627167140 learned_addr learned my addr 172.21.15.83:0/1627167140 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee36d9d640 1 --2- 172.21.15.83:0/1627167140 >> v2:172.21.15.155:3300/0 conn(0x7fee3806c790 0x7fee3806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3659c640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 0x7fee380b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984041501 0 0) 0x7fee20001120 con 0x7fee3815ec80 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.503+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee24003530 con 0x7fee3815ec80 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960283266 0 0) 0x7fee24003530 con 0x7fee3815ec80 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 msgr2=0x7fee380b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 0x7fee380b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 >> v2:172.21.15.155:3300/0 conn(0x7fee3806c790 msgr2=0x7fee3806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 --2- 172.21.15.83:0/1627167140 >> v2:172.21.15.155:3300/0 conn(0x7fee3806c790 0x7fee3806cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee380af930 con 0x7fee3815ec80 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fee200018d0 con 0x7fee3815ec80 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 >> v1:172.21.15.83:6789/0 conn(0x7fee3815ec80 legacy=0x7fee380b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee24003a60 0x7fee24003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24004810 0x7fee24004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 0x7fee24005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee3815e7f0 con 0x7fee24003a60 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee3806d0e0 con 0x7fee24004810 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee2400bab0 con 0x7fee380af3e0 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee24003a60 0x7fee24003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee35d9b640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24004810 0x7fee24004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 -- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 msgr2=0x7fee24005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee380af3e0 0x7fee24005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 -- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24004810 msgr2=0x7fee24004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee24004810 0x7fee24004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 -- 172.21.15.83:0/1627167140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee24003530 con 0x7fee24003a60 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3659c640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee24003a60 0x7fee24003ed0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fee30009fc0 tx=0x7fee3001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad470bb7719ec77 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee300b2070 con 0x7fee24003a60 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee300a59a0 con 0x7fee24003a60 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3559a640 1 -- 172.21.15.83:0/1627167140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30009c10 con 0x7fee24003a60 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3ceb4640 1 -- 172.21.15.83:0/1627167140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee24003a60 msgr2=0x7fee24003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/1627167140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee24003a60 0x7fee24003ed0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fee30009fc0 tx=0x7fee3001ccc0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee36d9d640 1 -- 172.21.15.83:0/1627167140 reap_dead start 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3ceb4640 1 -- 172.21.15.83:0/1627167140 shutdown_connections 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3ceb4640 1 -- 172.21.15.83:0/1627167140 >> 172.21.15.83:0/1627167140 conn(0x7fee3809b8b0 msgr2=0x7fee380b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.507+0000 7fee3ceb4640 1 -- 172.21.15.83:0/1627167140 shutdown_connections 2024-05-17T22:01:17.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 -- 172.21.15.83:0/1627167140 wait complete. 2024-05-17T22:01:17.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 Processor -- start 2024-05-17T22:01:17.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 -- start start 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee24003a60 0x7fee381f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 0x7fee3815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee381581a0 con 0x7fee380b1bf0 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee38158020 con 0x7fee3815d180 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3ceb4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee38158320 con 0x7fee24003a60 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46210/0 (socket says 172.21.15.83:46210) 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 -- 172.21.15.83:0/3141181822 learned_addr learned my addr 172.21.15.83:0/3141181822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee3659c640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee24003a60 0x7fee381f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee35d9b640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 0x7fee3815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 -- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee24003a60 msgr2=0x7fee381f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee24003a60 0x7fee381f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 -- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 msgr2=0x7fee3815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 0x7fee3815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 -- 172.21.15.83:0/3141181822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee380b2540 con 0x7fee380b1bf0 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee35d9b640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 0x7fee3815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.511+0000 7fee36d9d640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fee380b03b0 tx=0x7fee2800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800c620 con 0x7fee380b1bf0 2024-05-17T22:01:17.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2801a070 con 0x7fee380b1bf0 2024-05-17T22:01:17.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee381f7560 con 0x7fee380b1bf0 2024-05-17T22:01:17.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800ca20 con 0x7fee380b1bf0 2024-05-17T22:01:17.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381fb180 con 0x7fee380b1bf0 2024-05-17T22:01:17.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3806cba0 con 0x7fee380b1bf0 2024-05-17T22:01:17.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee2800cbc0 con 0x7fee380b1bf0 2024-05-17T22:01:17.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 0x7fee18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:17.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5767+0+0 (secure 0 0 0) 0x7fee2808f360 con 0x7fee380b1bf0 2024-05-17T22:01:17.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee3659c640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 0x7fee18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:17.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.515+0000 7fee3659c640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 0x7fee18071910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fee300af000 tx=0x7fee300a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:17.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.519+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fee380b2540 con 0x7fee380b1bf0 2024-05-17T22:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.715+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee3809c0c0 con 0x7fee380b1bf0 2024-05-17T22:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.715+0000 7fee2effd640 1 -- 172.21.15.83:0/3141181822 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5186 (secure 0 0 0) 0x7fee2805c2f0 con 0x7fee380b1bf0 2024-05-17T22:01:17.709 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:17.709 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:03.754919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/1850779527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1850779527},{"type":"v1","addr":"172.21.15.83:6837","nonce":1850779527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":5}]} 2024-05-17T22:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 43 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 msgr2=0x7fee18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 0x7fee18071910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fee300af000 tx=0x7fee300a7040 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 msgr2=0x7fee380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fee380b03b0 tx=0x7fee2800ec00 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 shutdown_connections 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee380b1bf0 0x7fee380b2000 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee1806f400 0x7fee18071910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee3815d180 0x7fee3815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 --2- 172.21.15.83:0/3141181822 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee24003a60 0x7fee381f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 >> 172.21.15.83:0/3141181822 conn(0x7fee3809b8b0 msgr2=0x7fee38160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 shutdown_connections 2024-05-17T22:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:17.719+0000 7fee3ceb4640 1 -- 172.21.15.83:0/3141181822 wait complete. 2024-05-17T22:01:17.731 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T22:01:17.916 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:01:17.916 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:01:17.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:01:18.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.071+0000 7f245b790640 1 Processor -- start 2024-05-17T22:01:18.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.071+0000 7f245b790640 1 -- start start 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245b790640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 0x7f24540d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245b790640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24540d10d0 con 0x7f2454073590 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245b790640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2454070120 con 0x7f2454143f20 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245b790640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f24540b4110 con 0x7f24540b3d00 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f2459d2d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2454073590 0x7f24541447f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35106/0 (socket says 172.21.15.83:35106) 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f2459d2d640 1 -- 172.21.15.83:0/594116769 learned_addr learned my addr 172.21.15.83:0/594116769 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 0x7f24540d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933238098 0 0) 0x7f24540d10d0 con 0x7f2454073590 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440003530 con 0x7f2454073590 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3414330116 0 0) 0x7f2440003530 con 0x7f2454073590 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 msgr2=0x7f24540d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 0x7f24540d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 >> v1:172.21.15.155:6789/0 conn(0x7f2454143f20 legacy=0x7f2454072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2454070400 con 0x7f2454073590 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f244c001a80 con 0x7f2454073590 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 >> v1:172.21.15.83:6789/0 conn(0x7f2454073590 legacy=0x7f24541447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2440003a40 0x7f2440003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2440004810 0x7f2440004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 0x7f24400052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24540b4110 con 0x7f2440003a40 2024-05-17T22:01:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f244000bb90 con 0x7f2440004810 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f244000bd90 con 0x7f24540b3d00 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2440003a40 0x7f2440003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f2458d2b640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2440004810 0x7f2440004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 -- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 msgr2=0x7f24400052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f24540b3d00 0x7f24400052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 -- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2440004810 msgr2=0x7f2440004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2440004810 0x7f2440004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f245952c640 1 -- 172.21.15.83:0/594116769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2440003530 con 0x7f2440003a40 2024-05-17T22:01:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.075+0000 7f2458d2b640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2440004810 0x7f2440004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:18.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245952c640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2440003a40 0x7f2440003eb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2450009ae0 tx=0x7f245001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a688f48c38bd0c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500096b0 con 0x7f2440003a40 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24500a6490 con 0x7f2440003a40 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f244bfff640 1 -- 172.21.15.83:0/594116769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500a6880 con 0x7f2440003a40 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- 172.21.15.83:0/594116769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2440003a40 msgr2=0x7f2440003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 --2- 172.21.15.83:0/594116769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2440003a40 0x7f2440003eb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2450009ae0 tx=0x7f245001cab0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f2459d2d640 1 -- 172.21.15.83:0/594116769 reap_dead start 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- 172.21.15.83:0/594116769 shutdown_connections 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- 172.21.15.83:0/594116769 >> 172.21.15.83:0/594116769 conn(0x7f245406cd90 msgr2=0x7f24540b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- 172.21.15.83:0/594116769 shutdown_connections 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- 172.21.15.83:0/594116769 wait complete. 2024-05-17T22:01:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 Processor -- start 2024-05-17T22:01:18.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.079+0000 7f245b790640 1 -- start start 2024-05-17T22:01:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f245415d310 0x7f24540b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2454161630 0x7f2454161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2454070c00 con 0x7f245415f2c0 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2454169690 con 0x7f2454161630 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2454161f80 con 0x7f245415d310 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46220/0 (socket says 172.21.15.83:46220) 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 -- 172.21.15.83:0/2709335335 learned_addr learned my addr 172.21.15.83:0/2709335335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245952c640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f245415d310 0x7f24540b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2458d2b640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2454161630 0x7f2454161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 -- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f245415d310 msgr2=0x7f24540b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f245415d310 0x7f24540b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 -- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2454161630 msgr2=0x7f2454161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2454161630 0x7f2454161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 -- 172.21.15.83:0/2709335335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24541fa8e0 con 0x7f245415f2c0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2459d2d640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2454072370 tx=0x7f244c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c018070 con 0x7f245415f2c0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c00ab00 con 0x7f245415f2c0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24541fabc0 con 0x7f245415f2c0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c007030 con 0x7f245415f2c0 2024-05-17T22:01:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541fb180 con 0x7f245415f2c0 2024-05-17T22:01:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24541447f0 con 0x7f245415f2c0 2024-05-17T22:01:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f244c01b070 con 0x7f245415f2c0 2024-05-17T22:01:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f2449ffb640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 0x7f2438071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.083+0000 7f245952c640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 0x7f2438071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.087+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5767+0+0 (secure 0 0 0) 0x7f244c08eb20 con 0x7f245415f2c0 2024-05-17T22:01:18.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.087+0000 7f245952c640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 0x7f2438071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24500ad000 tx=0x7f24500046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.091+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f244c05baf0 con 0x7f245415f2c0 2024-05-17T22:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.275+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f24540b0a70 con 0x7f245415f2c0 2024-05-17T22:01:18.268 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:01:18.275+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.279+0000 7f2449ffb640 1 -- 172.21.15.83:0/2709335335 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f244c05b3f0 con 0x7f245415f2c0 2024-05-17T22:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:01:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 msgr2=0x7f2438071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 0x7f2438071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f24500ad000 tx=0x7f24500046d0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 msgr2=0x7f245415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2454072370 tx=0x7f244c00ced0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 shutdown_connections 2024-05-17T22:01:18.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f245415f2c0 0x7f245415f6d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f243806f400 0x7f2438071910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2454161630 0x7f2454161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 --2- 172.21.15.83:0/2709335335 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f245415d310 0x7f24540b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 >> 172.21.15.83:0/2709335335 conn(0x7f245406cd90 msgr2=0x7f2454142d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 shutdown_connections 2024-05-17T22:01:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.283+0000 7f245b790640 1 -- 172.21.15.83:0/2709335335 wait complete. 2024-05-17T22:01:18.278 INFO:tasks.ceph.mds.a.smithi083.stderr: -3388> 2024-05-17T22:01:07.280+0000 7f35e6a3b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:01:18.296 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-17T22:01:18.338 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:01:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.367+0000 7f0c111d5080 1 Processor -- start 2024-05-17T22:01:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.367+0000 7f0c111d5080 1 -- start start 2024-05-17T22:01:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36186a0 0x55b3b3643af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b3b36eb690 con 0x55b3b36d1c60 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b3b35641b0 con 0x55b3b36186a0 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b3b36eb810 con 0x55b3b3617c00 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46268/0 (socket says 172.21.15.83:46268) 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/70419475 learned_addr learned my addr 172.21.15.83:0/70419475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0acb7640 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c03fff640 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36186a0 0x55b3b3643af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 msgr2=0x55b3b36e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36186a0 msgr2=0x55b3b3643af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36186a0 0x55b3b3643af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/70419475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b3562760 con 0x55b3b36d1c60 2024-05-17T22:01:18.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c460 tx=0x7f0c0400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd55f2c65744f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c09cb5640 1 -- 172.21.15.83:0/70419475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040116b0 con 0x55b3b36d1c60 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c09cb5640 1 -- 172.21.15.83:0/70419475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04017070 con 0x55b3b36d1c60 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c09cb5640 1 -- 172.21.15.83:0/70419475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04011ab0 con 0x55b3b36d1c60 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- 172.21.15.83:0/70419475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 msgr2=0x55b3b3654850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c460 tx=0x7f0c0400c920 comp rx=0 tx=0).stop 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- 172.21.15.83:0/70419475 shutdown_connections 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36d1c60 0x55b3b3654850 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36186a0 0x55b3b3643af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 --2- 172.21.15.83:0/70419475 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- 172.21.15.83:0/70419475 >> 172.21.15.83:0/70419475 conn(0x55b3b3600e40 msgr2=0x55b3b365edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- 172.21.15.83:0/70419475 shutdown_connections 2024-05-17T22:01:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- 172.21.15.83:0/70419475 wait complete. 2024-05-17T22:01:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 Processor -- start 2024-05-17T22:01:18.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.371+0000 7f0c111d5080 1 -- start start 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 0x55b3b3775420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36d1c60 0x55b3b3618c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b3b3642e70 con 0x55b3b36186a0 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b3b3642bd0 con 0x55b3b36d1c60 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b3b3643110 con 0x55b3b3617c00 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 0x55b3b3775420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 0x55b3b3775420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46282/0 (socket says 172.21.15.83:46282) 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 -- 172.21.15.83:0/1778316060 learned_addr learned my addr 172.21.15.83:0/1778316060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36d1c60 0x55b3b3618c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0acb7640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 -- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 msgr2=0x55b3b36e93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b3b3617c00 0x55b3b36e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 -- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36d1c60 msgr2=0x55b3b3618c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36d1c60 0x55b3b3618c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 -- 172.21.15.83:0/1778316060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b353ec10 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3b36d1c60 0x55b3b3618c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c03fff640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 0x55b3b3775420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0bf401c2f0 tx=0x7f0bf401a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 -- 172.21.15.83:0/1778316060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40b4040 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 -- 172.21.15.83:0/1778316060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf40bf070 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 -- 172.21.15.83:0/1778316060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40af840 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b3b3775e80 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b37763c0 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 -- 172.21.15.83:0/1778316060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bf40af9e0 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0bec06dde0 0x7f0bec0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c08cb3640 1 -- 172.21.15.83:0/1778316060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f0bf412bf90 con 0x55b3b36186a0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0acb7640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0bec06dde0 0x7f0bec0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b3b3775960 0x55b3b377edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) -- 0x55b3b377f310 con 0x55b3b3775960 2024-05-17T22:01:18.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0acb7640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0bec06dde0 0x7f0bec0702f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009530 tx=0x7f0bfc005710 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.375+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b3b3775960 0x55b3b377edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.379+0000 7f0c0b4b8640 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b3b3775960 0x55b3b377edd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/1778316060 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v70'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0c04011910 con 0x55b3b3775960 2024-05-17T22:01:18.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b3b3775960 msgr2=0x55b3b377edd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b3b3775960 0x55b3b377edd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0bec06dde0 msgr2=0x7f0bec0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0bec06dde0 0x7f0bec0702f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009530 tx=0x7f0bfc005710 comp rx=0 tx=0).stop 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 msgr2=0x55b3b3775420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 --2- 172.21.15.83:0/1778316060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3b36186a0 0x55b3b3775420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0bf401c2f0 tx=0x7f0bf401a9e0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c0b4b8640 1 -- 172.21.15.83:0/1778316060 reap_dead start 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 shutdown_connections 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 >> 172.21.15.83:0/1778316060 conn(0x55b3b3600e40 msgr2=0x55b3b365ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 shutdown_connections 2024-05-17T22:01:18.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.387+0000 7f0c111d5080 1 -- 172.21.15.83:0/1778316060 wait complete. 2024-05-17T22:01:18.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:01:18.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 Processor -- start 2024-05-17T22:01:18.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 -- start start 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40b88b0 0x7f2fd40d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 0x7f2fd409be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd409c350 con 0x7f2fd40ce130 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40d1a40 con 0x7f2fd40b88b0 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd9a47640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd409d040 con 0x7f2fd409ba00 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40b88b0 0x7f2fd40d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2fd40b88b0 0x7f2fd40d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46288/0 (socket says 172.21.15.83:46288) 2024-05-17T22:01:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.583+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/104537280 learned_addr learned my addr 172.21.15.83:0/104537280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/104537280 >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 0x7f2fd409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/104537280 >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 msgr2=0x7f2fd409be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd2ffd640 1 --2- 172.21.15.83:0/104537280 >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 0x7f2fd409be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/104537280 >> v1:172.21.15.155:6789/0 conn(0x7f2fd40ce130 legacy=0x7f2fd40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/104537280 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd409c710 con 0x7f2fd40b88b0 2024-05-17T22:01:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/104537280 >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 0x7f2fd409be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd2ffd640 1 --2- 172.21.15.83:0/104537280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40d11f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fd406d560 tx=0x7f2fc401c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3967dd8cbe9e5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd27fc640 1 -- 172.21.15.83:0/104537280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40b0070 con 0x7f2fd40b88b0 2024-05-17T22:01:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd27fc640 1 -- 172.21.15.83:0/104537280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc40ab600 con 0x7f2fd40b88b0 2024-05-17T22:01:18.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd27fc640 1 -- 172.21.15.83:0/104537280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40ab9d0 con 0x7f2fd40b88b0 2024-05-17T22:01:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 -- 172.21.15.83:0/104537280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40b88b0 msgr2=0x7f2fd40d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/104537280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40d11f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fd406d560 tx=0x7f2fc401c220 comp rx=0 tx=0).stop 2024-05-17T22:01:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 -- 172.21.15.83:0/104537280 shutdown_connections 2024-05-17T22:01:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/104537280 >> v2:172.21.15.155:3300/0 conn(0x7f2fd409ba00 0x7f2fd409be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/104537280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40d11f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 -- 172.21.15.83:0/104537280 >> 172.21.15.83:0/104537280 conn(0x7f2fd40d3620 msgr2=0x7f2fd40d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 -- 172.21.15.83:0/104537280 shutdown_connections 2024-05-17T22:01:18.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.587+0000 7f2fd9a47640 1 -- 172.21.15.83:0/104537280 wait complete. 2024-05-17T22:01:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 Processor -- start 2024-05-17T22:01:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 -- start start 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 0x7f2fd40b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40b2dc0 con 0x7f2fd409ba00 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40cf060 con 0x7f2fd40b88b0 2024-05-17T22:01:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd9a47640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40b2f40 con 0x7f2fd40ce130 2024-05-17T22:01:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46296/0 (socket says 172.21.15.83:46296) 2024-05-17T22:01:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/3344314714 learned_addr learned my addr 172.21.15.83:0/3344314714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.591+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 0x7f2fd40b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 msgr2=0x7f2fd40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 0x7f2fd40b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 msgr2=0x7f2fd40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 -- 172.21.15.83:0/3344314714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd4159620 con 0x7f2fd409ba00 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd37fe640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 0x7f2fd40b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd2ffd640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40ae040 tx=0x7f2fc401aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40ba020 con 0x7f2fd409ba00 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc40b6620 con 0x7f2fd409ba00 2024-05-17T22:01:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd4159900 con 0x7f2fd409ba00 2024-05-17T22:01:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40b0040 con 0x7f2fd409ba00 2024-05-17T22:01:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd41521a0 con 0x7f2fd409ba00 2024-05-17T22:01:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd409be10 con 0x7f2fd409ba00 2024-05-17T22:01:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2fc40bd020 con 0x7f2fd409ba00 2024-05-17T22:01:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 0x7f2fb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f2fc4129ab0 con 0x7f2fd409ba00 2024-05-17T22:01:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.595+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 0x7f2fb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.599+0000 7f2fca7fc640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 0x7f2fb4071910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40d0150 tx=0x7f2fbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:18.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.599+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2fc40c4070 con 0x7f2fd409ba00 2024-05-17T22:01:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:18.783+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2fd40d0740 con 0x7f2fd409ba00 2024-05-17T22:01:19.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.279+0000 7f2fd0ff9640 1 -- 172.21.15.83:0/3344314714 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v45) ==== 162+0+0 (secure 0 0 0) 0x7f2fd409c350 con 0x7f2fd409ba00 2024-05-17T22:01:19.274 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:01:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 msgr2=0x7f2fb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 0x7f2fb4071910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40d0150 tx=0x7f2fbc008040 comp rx=0 tx=0).stop 2024-05-17T22:01:19.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 msgr2=0x7f2fd4158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40ae040 tx=0x7f2fc401aa60 comp rx=0 tx=0).stop 2024-05-17T22:01:19.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 shutdown_connections 2024-05-17T22:01:19.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fd40ce130 0x7f2fd40b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2fb406f400 0x7f2fb4071910 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fd40b88b0 0x7f2fd40b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 --2- 172.21.15.83:0/3344314714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd409ba00 0x7f2fd4158a10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 >> 172.21.15.83:0/3344314714 conn(0x7f2fd40d3620 msgr2=0x7f2fd40b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.283+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 shutdown_connections 2024-05-17T22:01:19.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.287+0000 7f2fd9a47640 1 -- 172.21.15.83:0/3344314714 wait complete. 2024-05-17T22:01:19.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 Processor -- start 2024-05-17T22:01:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 -- start start 2024-05-17T22:01:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7b415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b41465d0 con 0x7fb7b415e6b0 2024-05-17T22:01:19.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b406ef30 con 0x7fb7b40706e0 2024-05-17T22:01:19.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ba6c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b406f210 con 0x7fb7b41461c0 2024-05-17T22:01:19.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ab7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7b415e6b0 0x7fb7b406ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35122/0 (socket says 172.21.15.83:35122) 2024-05-17T22:01:19.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7ab7fe640 1 -- 172.21.15.83:0/1663864811 learned_addr learned my addr 172.21.15.83:0/1663864811 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7b415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.459+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342807099 0 0) 0x7fb7b41465d0 con 0x7fb7b415e6b0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7a0003530 con 0x7fb7b415e6b0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 845296250 0 0) 0x7fb7a0003530 con 0x7fb7b415e6b0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 msgr2=0x7fb7b415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7b415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 >> v1:172.21.15.155:6789/0 conn(0x7fb7b40706e0 legacy=0x7fb7b41605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b406f4f0 con 0x7fb7b415e6b0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fb79c001700 con 0x7fb7b415e6b0 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 >> v1:172.21.15.83:6789/0 conn(0x7fb7b415e6b0 legacy=0x7fb7b406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0003a40 0x7fb7a0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a0004810 0x7fb7a0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b406f210 con 0x7fb7a0004810 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a00055a0 con 0x7fb7a0003a40 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a000bf10 con 0x7fb7b41461c0 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a0004810 0x7fb7a0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0003a40 0x7fb7a0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b8c5f640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 -- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 msgr2=0x7fb7a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 -- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0003a40 msgr2=0x7fb7a0003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0003a40 0x7fb7a0003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 -- 172.21.15.83:0/1663864811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a0003530 con 0x7fb7a0004810 2024-05-17T22:01:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b8c5f640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b41461c0 0x7fb7a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:19.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a0004810 0x7fb7a0004c20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c240 tx=0x7fb79c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db16762652f74bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c007d70 con 0x7fb7a0004810 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c005ae0 con 0x7fb7a0004810 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b37fe640 1 -- 172.21.15.83:0/1663864811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c007070 con 0x7fb7a0004810 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/1663864811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a0004810 msgr2=0x7fb7a0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/1663864811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a0004810 0x7fb7a0004c20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c240 tx=0x7fb79c01f330 comp rx=0 tx=0).stop 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7b8c5f640 1 -- 172.21.15.83:0/1663864811 reap_dead start 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/1663864811 shutdown_connections 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/1663864811 >> 172.21.15.83:0/1663864811 conn(0x7fb7b406cd90 msgr2=0x7fb7b40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/1663864811 shutdown_connections 2024-05-17T22:01:19.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.463+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/1663864811 wait complete. 2024-05-17T22:01:19.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 Processor -- start 2024-05-17T22:01:19.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 -- start start 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 0x7fb7b40aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 0x7fb7b413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4160cb0 con 0x7fb7b4072e30 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b406fab0 con 0x7fb7a0004810 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ba6c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b4160e30 con 0x7fb7b413d980 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46304/0 (socket says 172.21.15.83:46304) 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 -- 172.21.15.83:0/2776641647 learned_addr learned my addr 172.21.15.83:0/2776641647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 0x7fb7b413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 0x7fb7b40aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 -- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 msgr2=0x7fb7b413cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 0x7fb7b413cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 -- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 msgr2=0x7fb7b40aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 0x7fb7b40aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 -- 172.21.15.83:0/2776641647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b4073780 con 0x7fb7b4072e30 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7ab7fe640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 0x7fb7b413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 0x7fb7b40aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.467+0000 7fb7b8c5f640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b3c90 tx=0x7fb7ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac018070 con 0x7fb7b4072e30 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac006da0 con 0x7fb7b4072e30 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b413e0d0 con 0x7fb7b4072e30 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac003bc0 con 0x7fb7b4072e30 2024-05-17T22:01:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b413e690 con 0x7fb7b4072e30 2024-05-17T22:01:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b413de40 con 0x7fb7b4072e30 2024-05-17T22:01:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7ac01b070 con 0x7fb7b4072e30 2024-05-17T22:01:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 0x7fb794071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 0x7fb794071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fb7ac08e680 con 0x7fb7b4072e30 2024-05-17T22:01:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.471+0000 7fb7b3fff640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 0x7fb794071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b11f0 tx=0x7fb7a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.475+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb7b4073780 con 0x7fb7b4072e30 2024-05-17T22:01:19.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7b415e170 con 0x7fb7b4072e30 2024-05-17T22:01:19.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7b17fa640 1 -- 172.21.15.83:0/2776641647 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5180 (secure 0 0 0) 0x7fb7ac05b570 con 0x7fb7b4072e30 2024-05-17T22:01:19.663 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:19.664 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:19.290065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-17T22:01:19.664 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 msgr2=0x7fb794071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 0x7fb794071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b11f0 tx=0x7fb7a4008040 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 msgr2=0x7fb7b4073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b3c90 tx=0x7fb7ac00dc00 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 shutdown_connections 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7b4072e30 0x7fb7b4073240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7b413d980 0x7fb7b413cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb79406f430 0x7fb794071940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 --2- 172.21.15.83:0/2776641647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a0004810 0x7fb7b40aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 >> 172.21.15.83:0/2776641647 conn(0x7fb7b406cd90 msgr2=0x7fb7b40cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 shutdown_connections 2024-05-17T22:01:19.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:19.671+0000 7fb7ba6c2640 1 -- 172.21.15.83:0/2776641647 wait complete. 2024-05-17T22:01:20.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:01:00.724476+0000', 'modified': '2024-05-17T22:01:19.290065+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6234}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6234': {'gid': 6234, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/1116165089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1116165089}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1116165089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:01:20.087 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6234:up:replay 2024-05-17T22:01:21.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:21.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 Processor -- start 2024-05-17T22:01:21.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 -- start start 2024-05-17T22:01:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd3bc06dcc0 0x7fd3bc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3bc0d4260 0x7fd3bc0d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3bc06e610 con 0x7fd3bc15ffa0 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd3bc0d34e0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3c1aad640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd3bc0d3780 con 0x7fd3bc06dcc0 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3bc0d4260 0x7fd3bc0d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd3bc0d4260 0x7fd3bc0d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46308/0 (socket says 172.21.15.83:46308) 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 -- 172.21.15.83:0/1894214623 learned_addr learned my addr 172.21.15.83:0/1894214623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3baffd640 1 --2- 172.21.15.83:0/1894214623 >> v2:172.21.15.155:3300/0 conn(0x7fd3bc06dcc0 0x7fd3bc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 -- 172.21.15.83:0/1894214623 >> v2:172.21.15.155:3300/0 conn(0x7fd3bc06dcc0 msgr2=0x7fd3bc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 --2- 172.21.15.83:0/1894214623 >> v2:172.21.15.155:3300/0 conn(0x7fd3bc06dcc0 0x7fd3bc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 -- 172.21.15.83:0/1894214623 >> v1:172.21.15.155:6789/0 conn(0x7fd3bc15ffa0 legacy=0x7fd3bc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:21.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.255+0000 7fd3bbfff640 1 -- 172.21.15.83:0/1894214623 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc0d3a60 con 0x7fd3bc0d4260 2024-05-17T22:01:21.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3bbfff640 1 --2- 172.21.15.83:0/1894214623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc0d4670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0b5260 tx=0x7fd3b401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e28a506a7fb8e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:21.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3ba7fc640 1 -- 172.21.15.83:0/1894214623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40a5510 con 0x7fd3bc0d4260 2024-05-17T22:01:21.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3ba7fc640 1 -- 172.21.15.83:0/1894214623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b40a56b0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3ba7fc640 1 -- 172.21.15.83:0/1894214623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40a5a80 con 0x7fd3bc0d4260 2024-05-17T22:01:21.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1894214623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 msgr2=0x7fd3bc0d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1894214623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc0d4670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0b5260 tx=0x7fd3b401c810 comp rx=0 tx=0).stop 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1894214623 shutdown_connections 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1894214623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc0d4670 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1894214623 >> v2:172.21.15.155:3300/0 conn(0x7fd3bc06dcc0 0x7fd3bc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1894214623 >> 172.21.15.83:0/1894214623 conn(0x7fd3bc0d28f0 msgr2=0x7fd3bc0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1894214623 shutdown_connections 2024-05-17T22:01:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.259+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1894214623 wait complete. 2024-05-17T22:01:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 Processor -- start 2024-05-17T22:01:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- start start 2024-05-17T22:01:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3bc06dcc0 0x7fd3bc145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3bc15ffa0 0x7fd3bc163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc0d17e0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc0d1660 con 0x7fd3bc15ffa0 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3bc16cb80 con 0x7fd3bc06dcc0 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46310/0 (socket says 172.21.15.83:46310) 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 -- 172.21.15.83:0/1369214558 learned_addr learned my addr 172.21.15.83:0/1369214558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3bb7fe640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3bc06dcc0 0x7fd3bc145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3bbfff640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3bc15ffa0 0x7fd3bc163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 -- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3bc06dcc0 msgr2=0x7fd3bc145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3bc06dcc0 0x7fd3bc145820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 -- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3bc15ffa0 msgr2=0x7fd3bc163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3bc15ffa0 0x7fd3bc163770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 -- 172.21.15.83:0/1369214558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc148e50 con 0x7fd3bc0d4260 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3baffd640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d620 tx=0x7fd3b000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000b620 con 0x7fd3bc0d4260 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0019070 con 0x7fd3bc0d4260 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc15a320 con 0x7fd3bc0d4260 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000b9e0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.263+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc15a8e0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc06e0d0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3b000bb80 con 0x7fd3bc0d4260 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3a3fff640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 0x7fd3a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fd3b008f7b0 con 0x7fd3bc0d4260 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3bb7fe640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 0x7fd3a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.267+0000 7fd3bb7fe640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 0x7fd3a8071940 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00a750 tx=0x7fd3ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.271+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd3bc148e50 con 0x7fd3bc0d4260 2024-05-17T22:01:21.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.455+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3bc0cf390 con 0x7fd3bc0d4260 2024-05-17T22:01:21.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.455+0000 7fd3a3fff640 1 -- 172.21.15.83:0/1369214558 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5180 (secure 0 0 0) 0x7fd3b005c610 con 0x7fd3bc0d4260 2024-05-17T22:01:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:21.449 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:20.301677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-17T22:01:21.450 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 48 2024-05-17T22:01:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 msgr2=0x7fd3a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 0x7fd3a8071940 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00a750 tx=0x7fd3ac008040 comp rx=0 tx=0).stop 2024-05-17T22:01:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 msgr2=0x7fd3bc148910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000d620 tx=0x7fd3b000dae0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 shutdown_connections 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd3a806f430 0x7fd3a8071940 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3bc15ffa0 0x7fd3bc163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc0d4260 0x7fd3bc148910 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 --2- 172.21.15.83:0/1369214558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3bc06dcc0 0x7fd3bc145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 >> 172.21.15.83:0/1369214558 conn(0x7fd3bc0d28f0 msgr2=0x7fd3bc0b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 shutdown_connections 2024-05-17T22:01:21.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:21.459+0000 7fd3c1aad640 1 -- 172.21.15.83:0/1369214558 wait complete. 2024-05-17T22:01:21.874 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:01:00.724476+0000', 'modified': '2024-05-17T22:01:20.301677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6234}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6234': {'gid': 6234, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6837/1116165089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1116165089}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1116165089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:01:21.874 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6234:up:rejoin 2024-05-17T22:01:22.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:01:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.035+0000 7fc71f1e2640 1 Processor -- start 2024-05-17T22:01:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 -- start start 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7181461c0 0x7fc71815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 0x7fc7181605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7181465d0 con 0x7fc71815e6b0 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc71806ef30 con 0x7fc7181461c0 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71f1e2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc71806f210 con 0x7fc7180706e0 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7181461c0 0x7fc71815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc717fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc71815e6b0 0x7fc71806ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35128/0 (socket says 172.21.15.83:35128) 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7181461c0 0x7fc71815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46314/0 (socket says 172.21.15.83:46314) 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc717fff640 1 -- 172.21.15.83:0/3380370686 learned_addr learned my addr 172.21.15.83:0/3380370686 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71d77f640 1 --2- 172.21.15.83:0/3380370686 >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 0x7fc7181605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 -- 172.21.15.83:0/3380370686 >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 msgr2=0x7fc7181605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 --2- 172.21.15.83:0/3380370686 >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 0x7fc7181605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 -- 172.21.15.83:0/3380370686 >> v1:172.21.15.83:6789/0 conn(0x7fc71815e6b0 legacy=0x7fc71806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc7177fe640 1 -- 172.21.15.83:0/3380370686 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324311998 0 0) 0x7fc7181465d0 con 0x7fc71815e6b0 2024-05-17T22:01:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71cf7e640 1 -- 172.21.15.83:0/3380370686 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71806f4f0 con 0x7fc7181461c0 2024-05-17T22:01:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.039+0000 7fc71d77f640 1 --2- 172.21.15.83:0/3380370686 >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 0x7fc7181605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71cf7e640 1 --2- 172.21.15.83:0/3380370686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7181461c0 0x7fc71815e170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc718072c00 tx=0x7fc70801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd55a5d12e9f4486 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc7177fe640 1 -- 172.21.15.83:0/3380370686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080b2070 con 0x7fc7181461c0 2024-05-17T22:01:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc7177fe640 1 -- 172.21.15.83:0/3380370686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7080a9070 con 0x7fc7181461c0 2024-05-17T22:01:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc7177fe640 1 -- 172.21.15.83:0/3380370686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080ad880 con 0x7fc7181461c0 2024-05-17T22:01:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- 172.21.15.83:0/3380370686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7181461c0 msgr2=0x7fc71815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/3380370686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7181461c0 0x7fc71815e170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc718072c00 tx=0x7fc70801c450 comp rx=0 tx=0).stop 2024-05-17T22:01:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- 172.21.15.83:0/3380370686 shutdown_connections 2024-05-17T22:01:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/3380370686 >> v2:172.21.15.155:3300/0 conn(0x7fc7180706e0 0x7fc7181605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/3380370686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7181461c0 0x7fc71815e170 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- 172.21.15.83:0/3380370686 >> 172.21.15.83:0/3380370686 conn(0x7fc71806cd90 msgr2=0x7fc7180b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- 172.21.15.83:0/3380370686 shutdown_connections 2024-05-17T22:01:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- 172.21.15.83:0/3380370686 wait complete. 2024-05-17T22:01:23.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 Processor -- start 2024-05-17T22:01:23.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.043+0000 7fc71f1e2640 1 -- start start 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 0x7fc718159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc71815e6b0 0x7fc71815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc718169940 con 0x7fc7180706e0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180d12b0 con 0x7fc7181461c0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc718169ac0 con 0x7fc71815e6b0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46318/0 (socket says 172.21.15.83:46318) 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 -- 172.21.15.83:0/620263971 learned_addr learned my addr 172.21.15.83:0/620263971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc717fff640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 0x7fc718159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71d77f640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc71815e6b0 0x7fc71815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 -- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc71815e6b0 msgr2=0x7fc71815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc71815e6b0 0x7fc71815a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 -- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 msgr2=0x7fc718159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 0x7fc718159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 -- 172.21.15.83:0/620263971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc718166d40 con 0x7fc7180706e0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc717fff640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 0x7fc718159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71cf7e640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc70801a080 tx=0x7fc70801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080b2050 con 0x7fc7180706e0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7080b3040 con 0x7fc7180706e0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc718165ca0 con 0x7fc7180706e0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708004420 con 0x7fc7180706e0 2024-05-17T22:01:23.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc718166260 con 0x7fc7180706e0 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7181605a0 con 0x7fc7180706e0 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.047+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc7080c0060 con 0x7fc7180706e0 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.051+0000 7fc7157fa640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 0x7fc704071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.051+0000 7fc717fff640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 0x7fc704071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.051+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fc708128ee0 con 0x7fc7180706e0 2024-05-17T22:01:23.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.051+0000 7fc717fff640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 0x7fc704071910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7181670c0 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.055+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc708016070 con 0x7fc7180706e0 2024-05-17T22:01:23.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.239+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7180767c0 con 0x7fc7180706e0 2024-05-17T22:01:23.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.239+0000 7fc7157fa640 1 -- 172.21.15.83:0/620263971 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5187 (secure 0 0 0) 0x7fc7080f5cc0 con 0x7fc7180706e0 2024-05-17T22:01:23.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:01:23.234 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:22.305427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:01:23.234 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 msgr2=0x7fc704071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 0x7fc704071910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7181670c0 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 msgr2=0x7fc7180730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc70801a080 tx=0x7fc70801ac90 comp rx=0 tx=0).stop 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 shutdown_connections 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc71815e6b0 0x7fc71815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc70406f400 0x7fc704071910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7181461c0 0x7fc718159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 --2- 172.21.15.83:0/620263971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7180706e0 0x7fc7180730c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 >> 172.21.15.83:0/620263971 conn(0x7fc71806cd90 msgr2=0x7fc71815eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 shutdown_connections 2024-05-17T22:01:23.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.243+0000 7fc71f1e2640 1 -- 172.21.15.83:0/620263971 wait complete. 2024-05-17T22:01:23.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:01:00.724476+0000', 'modified': '2024-05-17T22:01:22.305427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6234}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6234': {'gid': 6234, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6837/1116165089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1116165089}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1116165089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6234, 'qdb_cluster': [6234]} 2024-05-17T22:01:23.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:01:23.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:01:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 Processor -- start 2024-05-17T22:01:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 -- start start 2024-05-17T22:01:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06680af3e0 0x7f06680b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06680b3f20 con 0x7f066815ec80 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06680af930 con 0x7f066806c790 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f0670070640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f066815e7f0 con 0x7f06680af3e0 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f066d60b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f066815ec80 0x7f06680b49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35132/0 (socket says 172.21.15.83:35132) 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f066d60b640 1 -- 172.21.15.83:0/3236658263 learned_addr learned my addr 172.21.15.83:0/3236658263 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.823+0000 7f066de0c640 1 --2- 172.21.15.83:0/3236658263 >> v2:172.21.15.83:3300/0 conn(0x7f06680af3e0 0x7f06680b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066de0c640 1 -- 172.21.15.83:0/3236658263 >> v1:172.21.15.155:6789/0 conn(0x7f066806c790 legacy=0x7f066806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066de0c640 1 -- 172.21.15.83:0/3236658263 >> v1:172.21.15.83:6789/0 conn(0x7f066815ec80 legacy=0x7f06680b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066ce0a640 1 -- 172.21.15.83:0/3236658263 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187485376 0 0) 0x7f0658001120 con 0x7f066815ec80 2024-05-17T22:01:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066de0c640 1 -- 172.21.15.83:0/3236658263 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066815e1a0 con 0x7f06680af3e0 2024-05-17T22:01:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066de0c640 1 --2- 172.21.15.83:0/3236658263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f06680b39e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f06640097c0 tx=0x7f066401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cb68bc5765ca95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066ce0a640 1 -- 172.21.15.83:0/3236658263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640b0070 con 0x7f06680af3e0 2024-05-17T22:01:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066ce0a640 1 -- 172.21.15.83:0/3236658263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06640ab600 con 0x7f06680af3e0 2024-05-17T22:01:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f066ce0a640 1 -- 172.21.15.83:0/3236658263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640a4e60 con 0x7f06680af3e0 2024-05-17T22:01:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 -- 172.21.15.83:0/3236658263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 msgr2=0x7f06680b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 --2- 172.21.15.83:0/3236658263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f06680b39e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f06640097c0 tx=0x7f066401c410 comp rx=0 tx=0).stop 2024-05-17T22:01:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 -- 172.21.15.83:0/3236658263 shutdown_connections 2024-05-17T22:01:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 --2- 172.21.15.83:0/3236658263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f06680b39e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 -- 172.21.15.83:0/3236658263 >> 172.21.15.83:0/3236658263 conn(0x7f066809b8b0 msgr2=0x7f06680b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 -- 172.21.15.83:0/3236658263 shutdown_connections 2024-05-17T22:01:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.827+0000 7f0670070640 1 -- 172.21.15.83:0/3236658263 wait complete. 2024-05-17T22:01:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 Processor -- start 2024-05-17T22:01:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 -- start start 2024-05-17T22:01:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 0x7f06681545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066815ec80 0x7f0668156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06681583b0 con 0x7f06680af3e0 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0668158230 con 0x7f066815ec80 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f06681521f0 con 0x7f066806c790 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46344/0 (socket says 172.21.15.83:46344) 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 -- 172.21.15.83:0/2824481659 learned_addr learned my addr 172.21.15.83:0/2824481659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066de0c640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 0x7f06681545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066e60d640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066815ec80 0x7f0668156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 -- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 msgr2=0x7f06681545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 0x7f06681545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 -- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066815ec80 msgr2=0x7f0668156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066815ec80 0x7f0668156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 -- 172.21.15.83:0/2824481659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668155100 con 0x7f06680af3e0 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066de0c640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 0x7f06681545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f066d60b640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f065800c430 tx=0x7f065800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f065e7fc640 1 -- 172.21.15.83:0/2824481659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580074e0 con 0x7f06680af3e0 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f065e7fc640 1 -- 172.21.15.83:0/2824481659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06580047f0 con 0x7f06680af3e0 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f065e7fc640 1 -- 172.21.15.83:0/2824481659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004be0 con 0x7f06680af3e0 2024-05-17T22:01:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.831+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06681553e0 con 0x7f06680af3e0 2024-05-17T22:01:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0668151f30 con 0x7f06680af3e0 2024-05-17T22:01:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f065e7fc640 1 -- 172.21.15.83:0/2824481659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0658011070 con 0x7f06680af3e0 2024-05-17T22:01:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f065e7fc640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 0x7f064c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f066de0c640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 0x7f064c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f065e7fc640 1 -- 172.21.15.83:0/2824481659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f065808eef0 con 0x7f06680af3e0 2024-05-17T22:01:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f066de0c640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 0x7f064c071940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06640ad000 tx=0x7f0664008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f06772ea000 1 Processor -- start 2024-05-17T22:01:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.835+0000 7f06772ea000 1 -- start start 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f99600 0x5639920e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563991f87ca0 con 0x563991f82d30 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563991efa110 con 0x563991fa3fa0 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563991ee9cb0 con 0x563991f99600 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46356/0 (socket says 172.21.15.83:46356) 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 -- 172.21.15.83:0/593837360 learned_addr learned my addr 172.21.15.83:0/593837360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066d60b640 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f99600 0x5639920e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 -- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f99600 msgr2=0x5639920e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066de0c640 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f99600 0x5639920e9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 -- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 msgr2=0x5639920e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 -- 172.21.15.83:0/593837360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563991eae370 con 0x563991f82d30 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f066e60d640 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0660007d00 tx=0x7f066000c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f0eaeb4df3adf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f064b7fe640 1 -- 172.21.15.83:0/593837360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660015020 con 0x563991f82d30 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f064b7fe640 1 -- 172.21.15.83:0/593837360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066000ce10 con 0x563991f82d30 2024-05-17T22:01:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f064b7fe640 1 -- 172.21.15.83:0/593837360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066000d040 con 0x563991f82d30 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- 172.21.15.83:0/593837360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 msgr2=0x5639920ea330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0660007d00 tx=0x7f066000c540 comp rx=0 tx=0).stop 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- 172.21.15.83:0/593837360 shutdown_connections 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f82d30 0x5639920ea330 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f99600 0x5639920e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 --2- 172.21.15.83:0/593837360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- 172.21.15.83:0/593837360 >> 172.21.15.83:0/593837360 conn(0x563991bab460 msgr2=0x56399209ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- 172.21.15.83:0/593837360 shutdown_connections 2024-05-17T22:01:23.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- 172.21.15.83:0/593837360 wait complete. 2024-05-17T22:01:23.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 -- start start 2024-05-17T22:01:23.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.839+0000 7f06772ea000 1 Processor -- start 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920ab460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563991ea5d40 con 0x563991f99600 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563991eedcf0 con 0x563991fa3fa0 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563991e92e90 con 0x563991f82d30 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 10 client.0 _ll_register_callbacks cb 0x563991ec7fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46358/0 (socket says 172.21.15.83:46358) 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 -- 172.21.15.83:0/4152803963 learned_addr learned my addr 172.21.15.83:0/4152803963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066e60d640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066de0c640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 -- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 msgr2=0x5639920a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 -- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 msgr2=0x5639920ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 -- 172.21.15.83:0/4152803963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563991eb86e0 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066d60b640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0658000f70 tx=0x7f065805be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c87037715e372f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065805a8b0 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066000ce10 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f065805aa50 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0658056fc0 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 10 client.6285 fetch_fsmap learned FSMap version 49 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x563991ea8690 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066000ce10 con 0x563991f99600 2024-05-17T22:01:23.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f0658057160 con 0x563991f99600 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 10 client.6285 fetch_fsmap finished waiting for FSMap version 49 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 10 client.6285 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 20 client.6285 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b907d0 0x5639920f0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 4 client.6285 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563991e85a00 con 0x563991b907d0 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066de0c640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b907d0 0x5639920f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f066de0c640 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b907d0 0x5639920f0920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bdd055aa7a388ba server_cookie=cf944d74e484a83c in_seq=0 out_seq=0 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 10 client.6285 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 1 -- 172.21.15.83:0/4152803963 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563991e85a00 con 0x563991b907d0 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 10 client.6285 handle_command_reply: tid=0 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 client.6285 shutdown 2024-05-17T22:01:23.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 msgr2=0x5639920a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0658000f70 tx=0x7f065805be90 comp rx=0 tx=0).stop 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 shutdown_connections 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991fa3fa0 0x5639920ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b907d0 0x5639920f0920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a67d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f0648ff9640 0 client.6285 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 --2- 172.21.15.83:0/4152803963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 >> 172.21.15.83:0/4152803963 conn(0x563991bab460 msgr2=0x56399209ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 shutdown_connections 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.843+0000 7f06772ea000 1 -- 172.21.15.83:0/4152803963 wait complete. 2024-05-17T22:01:23.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.847+0000 7f06772ea000 20 client.6285 trim_cache size 0 max 16384 2024-05-17T22:01:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 Processor -- start 2024-05-17T22:01:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 -- start start 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f99600 0x5639920e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0668202820 con 0x563991fa3fa0 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f066809fa70 con 0x563991f99600 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f06682021d0 con 0x563991f82d30 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46360/0 (socket says 172.21.15.83:46360) 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 -- 172.21.15.83:0/695827082 learned_addr learned my addr 172.21.15.83:0/695827082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 -- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 msgr2=0x5639920e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 -- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f99600 msgr2=0x5639920e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f99600 0x5639920e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 -- 172.21.15.83:0/695827082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639920f79a0 con 0x563991fa3fa0 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f066e60d640 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0660013000 tx=0x7f0660011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c4f69ff325927f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f064affd640 1 -- 172.21.15.83:0/695827082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660015050 con 0x563991fa3fa0 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f064affd640 1 -- 172.21.15.83:0/695827082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066000cd80 con 0x563991fa3fa0 2024-05-17T22:01:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.863+0000 7f064affd640 1 -- 172.21.15.83:0/695827082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660003ba0 con 0x563991fa3fa0 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/695827082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 msgr2=0x5639920ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0660013000 tx=0x7f0660011000 comp rx=0 tx=0).stop 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/695827082 shutdown_connections 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991fa3fa0 0x5639920ffea0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f99600 0x5639920e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- 172.21.15.83:0/695827082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991f82d30 0x5639920e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/695827082 >> 172.21.15.83:0/695827082 conn(0x563991bab460 msgr2=0x5639920f71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/695827082 shutdown_connections 2024-05-17T22:01:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/695827082 wait complete. 2024-05-17T22:01:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- start start 2024-05-17T22:01:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 Processor -- start 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 0x56399209fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991fa3fa0 0x5639920a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563991eed000 con 0x563991f99600 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5639920a9c20 con 0x563991f82d30 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563991ef8ee0 con 0x563991fa3fa0 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 10 client.0 _ll_register_callbacks cb 0x7f0658057510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46374/0 (socket says 172.21.15.83:46374) 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 -- 172.21.15.83:0/4097753782 learned_addr learned my addr 172.21.15.83:0/4097753782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066de0c640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 0x56399209fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066e60d640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991fa3fa0 0x5639920a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 -- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991fa3fa0 msgr2=0x5639920a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991fa3fa0 0x5639920a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 -- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 msgr2=0x56399209fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 0x56399209fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 -- 172.21.15.83:0/4097753782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563991f70cb0 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066de0c640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 0x56399209fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f066d60b640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f065808d230 tx=0x7f0658000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba2fe81055d583 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065805adc0 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f065805a320 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066000cd80 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f065805a540 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 10 client.6291 fetch_fsmap learned FSMap version 49 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x563991ea94f0 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066000cd80 con 0x563991f99600 2024-05-17T22:01:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f0658070cb0 con 0x563991f99600 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 10 client.6291 fetch_fsmap finished waiting for FSMap version 49 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 10 client.6291 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 20 client.6291 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b7e830 0x5639920a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 4 client.6291 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.867+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "status"}) -- 0x563991efc060 con 0x563991b7e830 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f066de0c640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b7e830 0x5639920a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0648ff9640 10 client.6291 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f066de0c640 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b7e830 0x5639920a1380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a47f7986c80f831 server_cookie=17d2bcdbb98f85af in_seq=0 out_seq=0 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0648ff9640 1 -- 172.21.15.83:0/4097753782 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563991efc060 con 0x563991b7e830 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0648ff9640 10 client.6291 handle_command_reply: tid=0 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 client.6291 shutdown 2024-05-17T22:01:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 msgr2=0x5639920a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f065808d230 tx=0x7f0658000ee0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 shutdown_connections 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563991fa3fa0 0x5639920a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563991b7e830 0x5639920a1380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563991f99600 0x5639920a03c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 --2- 172.21.15.83:0/4097753782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563991f82d30 0x56399209fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0648ff9640 0 client.6291 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 >> 172.21.15.83:0/4097753782 conn(0x563991bab460 msgr2=0x5639920e8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 shutdown_connections 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 1 -- 172.21.15.83:0/4097753782 wait complete. 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f06772ea000 20 client.6291 trim_cache size 0 max 16384 2024-05-17T22:01:23.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6234, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.5756962210000003, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 49, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 70, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 70, 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.5227730309999998 2024-05-17T22:01:23.868 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 msgr2=0x7f064c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 0x7f064c071940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f06640ad000 tx=0x7f0664008040 comp rx=0 tx=0).stop 2024-05-17T22:01:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 msgr2=0x7f0668155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f065800c430 tx=0x7f065800c8f0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 shutdown_connections 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f064c06f430 0x7f064c071940 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066815ec80 0x7f0668156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06680af3e0 0x7f0668155ec0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 --2- 172.21.15.83:0/2824481659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066806c790 0x7f06681545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 >> 172.21.15.83:0/2824481659 conn(0x7f066809b8b0 msgr2=0x7f06680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 shutdown_connections 2024-05-17T22:01:23.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:23.871+0000 7f0670070640 1 -- 172.21.15.83:0/2824481659 wait complete. 2024-05-17T22:01:23.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:01:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 Processor -- start 2024-05-17T22:01:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 -- start start 2024-05-17T22:01:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff370073540 0x7ff370168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37015e790 con 0x7ff3700d1040 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37015ea40 con 0x7ff370144080 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff375c60640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff37015ed20 con 0x7ff370073540 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3700d1040 0x7ff37015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35146/0 (socket says 172.21.15.83:35146) 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36f7fe640 1 -- 172.21.15.83:0/1983984449 learned_addr learned my addr 172.21.15.83:0/1983984449 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36ffff640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff370073540 0x7ff370168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094365962 0 0) 0x7ff37015e790 con 0x7ff3700d1040 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff358003530 con 0x7ff3700d1040 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 875487711 0 0) 0x7ff358003530 con 0x7ff3700d1040 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff370073540 msgr2=0x7ff370168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff370073540 0x7ff370168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 >> v1:172.21.15.155:6789/0 conn(0x7ff370144080 legacy=0x7ff370072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37015f000 con 0x7ff3700d1040 2024-05-17T22:01:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ff364001700 con 0x7ff3700d1040 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 >> v1:172.21.15.83:6789/0 conn(0x7ff3700d1040 legacy=0x7ff37015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358003a40 0x7ff358003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff358004810 0x7ff358004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff358005280 0x7ff35800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff37015ed20 con 0x7ff358004810 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff35800bfc0 con 0x7ff358003a40 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff35800c1c0 con 0x7ff358005280 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36effd640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff358004810 0x7ff358004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.031+0000 7ff36f7fe640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358003a40 0x7ff358003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 -- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff358005280 msgr2=0x7ff35800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff358005280 0x7ff35800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 -- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358003a40 msgr2=0x7ff358003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358003a40 0x7ff358003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 -- 172.21.15.83:0/1983984449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff358003530 con 0x7ff358004810 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36effd640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff358004810 0x7ff358004c20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff3700ceb70 tx=0x7ff36001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a533fddb5c17a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600013e0 con 0x7ff358004810 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3600052d0 con 0x7ff358004810 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36e7fc640 1 -- 172.21.15.83:0/1983984449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360003840 con 0x7ff358004810 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 -- 172.21.15.83:0/1983984449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff358004810 msgr2=0x7ff358004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 --2- 172.21.15.83:0/1983984449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff358004810 0x7ff358004c20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff3700ceb70 tx=0x7ff36001c390 comp rx=0 tx=0).stop 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff36ffff640 1 -- 172.21.15.83:0/1983984449 reap_dead start 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 -- 172.21.15.83:0/1983984449 shutdown_connections 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 -- 172.21.15.83:0/1983984449 >> 172.21.15.83:0/1983984449 conn(0x7ff37006cd90 msgr2=0x7ff3700b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 -- 172.21.15.83:0/1983984449 shutdown_connections 2024-05-17T22:01:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.035+0000 7ff375c60640 1 -- 172.21.15.83:0/1983984449 wait complete. 2024-05-17T22:01:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 Processor -- start 2024-05-17T22:01:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- start start 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358004810 0x7ff37015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3700ab200 0x7ff3700ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff370144490 con 0x7ff3700aa310 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff37006f210 con 0x7ff358004810 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff370073540 con 0x7ff3700ab200 2024-05-17T22:01:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46382/0 (socket says 172.21.15.83:46382) 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 -- 172.21.15.83:0/2706961649 learned_addr learned my addr 172.21.15.83:0/2706961649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36f7fe640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358004810 0x7ff37015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36ffff640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3700ab200 0x7ff3700ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 -- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3700ab200 msgr2=0x7ff3700ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3700ab200 0x7ff3700ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 -- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358004810 msgr2=0x7ff37015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358004810 0x7ff37015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 -- 172.21.15.83:0/2706961649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff370160290 con 0x7ff3700aa310 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff36effd640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff36001aa90 tx=0x7ff358005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600bf070 con 0x7ff3700aa310 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff360007250 con 0x7ff3700aa310 2024-05-17T22:01:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff370160570 con 0x7ff3700aa310 2024-05-17T22:01:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360007a80 con 0x7ff3700aa310 2024-05-17T22:01:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.039+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff370160b30 con 0x7ff3700aa310 2024-05-17T22:01:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff370072e30 con 0x7ff3700aa310 2024-05-17T22:01:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3600c2070 con 0x7ff3700aa310 2024-05-17T22:01:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff357fff640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff36f7fe640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7ff3601290f0 con 0x7ff3700aa310 2024-05-17T22:01:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.043+0000 7ff36f7fe640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 0x7ff35c071940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff364001ec0 tx=0x7ff364009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.047+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff3600160f0 con 0x7ff3700aa310 2024-05-17T22:01:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.231+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3700b38c0 con 0x7ff3700aa310 2024-05-17T22:01:24.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.231+0000 7ff357fff640 1 -- 172.21.15.83:0/2706961649 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7ff3600fae20 con 0x7ff3700aa310 2024-05-17T22:01:24.225 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 msgr2=0x7ff35c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 0x7ff35c071940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff364001ec0 tx=0x7ff364009040 comp rx=0 tx=0).stop 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 msgr2=0x7ff3700ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff36001aa90 tx=0x7ff358005840 comp rx=0 tx=0).stop 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 shutdown_connections 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3700ab200 0x7ff3700ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3700aa310 0x7ff3700ae450 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 --2- 172.21.15.83:0/2706961649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff358004810 0x7ff37015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 >> 172.21.15.83:0/2706961649 conn(0x7ff37006cd90 msgr2=0x7ff370144610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 shutdown_connections 2024-05-17T22:01:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24.235+0000 7ff375c60640 1 -- 172.21.15.83:0/2706961649 wait complete. 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:01:24.245 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:01:24.246 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:01:24.246 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:24.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.246 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.252 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.253 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.254 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:01:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:01:24.255 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:01:24.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.255 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:01:24.289 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:01:24.295+0000 7f7d9e130640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:01:24.299 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:01:24.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.301 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:01:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:01:24.352 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:01:24.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:01:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:01:24.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.471 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:01:24.518 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:01:24.519 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:01:24.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:24.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:01:24.574 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:01:24.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:01:24.574 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:24.623 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:01:24.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:01:24.624 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:24.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:24.732 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:24.783 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:24.784 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:24.785 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:24.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.787 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:24.835 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:01:24.835 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:01:24.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:24.894 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:24.943 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:24.944 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:24.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:24.945 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:24.986 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:01:24.991+0000 7f0fe27f9580 -1 init, newargv = 0x7f0fcc01b580 newargc=15 2024-05-17T22:01:24.986 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41627]: starting ceph client 2024-05-17T22:01:24.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41627]: starting fuse 2024-05-17T22:01:26.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:01:26.108 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:01:26.157 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:01:26.158 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:26.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.160 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:01:26.203 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:01:26.204 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:01:26.204 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:01:26.205 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.206 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:01:26.206 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:01:26.328 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.41627.asok 2024-05-17T22:01:26.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:01:26.339 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41627.asok 2024-05-17T22:01:26.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41627.asok status 2024-05-17T22:01:26.517 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:26.517 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:01:26.517 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:01:26.517 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:01:26.517 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "41627", 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6303, 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6303 2024-05-17T22:01:26.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 63555347 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 63555347 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6303 192.168.0.1:0/63555347", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/63555347", 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 49, 2024-05-17T22:01:26.519 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 70, 2024-05-17T22:01:26.520 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:01:26.520 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:01:26.520 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:01:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:26.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.533 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:01:26.533 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:26.539 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:01:26.540 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:26.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:26.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.594 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-17T22:01:26.650 INFO:teuthology.orchestra.run.smithi083.stdout:file_undamaged 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> import json 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> import sys 2024-05-17T22:01:26.651 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> try: 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> except OSError as e: 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> sys.exit(e.errno) 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> print(json.dumps( 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> indent=2)) 2024-05-17T22:01:26.652 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:01:26.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:26 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-17T22:01:26.776 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-17T22:01:26.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:26.776 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-17T22:01:26.890 INFO:teuthology.orchestra.run.smithi083.stdout:2 2024-05-17T22:01:26.891 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:01:26.891 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:01:26.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:01:27.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 Processor -- start 2024-05-17T22:01:27.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 -- start start 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80b4810 0x7f2ca80d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 0x7f2ca815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca80d2000 0x7f2ca80d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca80d4170 con 0x7f2ca80d2000 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca80d4450 con 0x7f2ca80d0290 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cb0414640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80d4730 con 0x7f2ca80b4810 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cae9b1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca80d2000 0x7f2ca80d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cae9b1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ca80d2000 0x7f2ca80d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50806/0 (socket says 172.21.15.83:50806) 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cae9b1640 1 -- 172.21.15.83:0/1279499524 learned_addr learned my addr 172.21.15.83:0/1279499524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cad9af640 1 --2- 172.21.15.83:0/1279499524 >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 0x7f2ca815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.103+0000 7f2cae1b0640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80b4810 0x7f2ca80d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 -- 172.21.15.83:0/1279499524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80b4810 msgr2=0x7f2ca80d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80b4810 0x7f2ca80d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 -- 172.21.15.83:0/1279499524 >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 msgr2=0x7f2ca815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 --2- 172.21.15.83:0/1279499524 >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 0x7f2ca815fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 -- 172.21.15.83:0/1279499524 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca80d4a10 con 0x7f2ca80d2000 2024-05-17T22:01:27.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cad9af640 1 --2- 172.21.15.83:0/1279499524 >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 0x7f2ca815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:01:27.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cae9b1640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d2000 0x7f2ca80d2410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2ca808eb60 tx=0x7f2ca001d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ff630a9783b5f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cad1ae640 1 -- 172.21.15.83:0/1279499524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a5890 con 0x7f2ca80d2000 2024-05-17T22:01:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cad1ae640 1 -- 172.21.15.83:0/1279499524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca00a5a30 con 0x7f2ca80d2000 2024-05-17T22:01:27.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cad1ae640 1 -- 172.21.15.83:0/1279499524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a5e00 con 0x7f2ca80d2000 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 -- 172.21.15.83:0/1279499524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d2000 msgr2=0x7f2ca80d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d2000 0x7f2ca80d2410 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2ca808eb60 tx=0x7f2ca001d450 comp rx=0 tx=0).stop 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 -- 172.21.15.83:0/1279499524 shutdown_connections 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d2000 0x7f2ca80d2410 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1279499524 >> v2:172.21.15.155:3300/0 conn(0x7f2ca80d0290 0x7f2ca815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1279499524 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80b4810 0x7f2ca80d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 -- 172.21.15.83:0/1279499524 >> 172.21.15.83:0/1279499524 conn(0x7f2ca806e010 msgr2=0x7f2ca806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 -- 172.21.15.83:0/1279499524 shutdown_connections 2024-05-17T22:01:27.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.107+0000 7f2cb0414640 1 -- 172.21.15.83:0/1279499524 wait complete. 2024-05-17T22:01:27.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 Processor -- start 2024-05-17T22:01:27.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 -- start start 2024-05-17T22:01:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca80b4810 0x7f2ca8161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 0x7f2ca8157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80b7aa0 con 0x7f2ca80d0290 2024-05-17T22:01:27.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8160050 con 0x7f2ca80b4810 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80b7c20 con 0x7f2ca80d2000 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50808/0 (socket says 172.21.15.83:50808) 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 -- 172.21.15.83:0/1222349857 learned_addr learned my addr 172.21.15.83:0/1222349857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cae1b0640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca80b4810 0x7f2ca8161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cae9b1640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 0x7f2ca8157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 -- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 msgr2=0x7f2ca8157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 0x7f2ca8157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 -- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca80b4810 msgr2=0x7f2ca8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca80b4810 0x7f2ca8161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 -- 172.21.15.83:0/1222349857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca8162b90 con 0x7f2ca80d0290 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cae9b1640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 0x7f2ca8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:27.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cad9af640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d830 tx=0x7f2ca400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400b6c0 con 0x7f2ca80d0290 2024-05-17T22:01:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca4019070 con 0x7f2ca80d0290 2024-05-17T22:01:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400ba80 con 0x7f2ca80d0290 2024-05-17T22:01:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.111+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81628b0 con 0x7f2ca80d0290 2024-05-17T22:01:27.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca8159db0 con 0x7f2ca80d0290 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca8069b70 con 0x7f2ca80d0290 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ca401e460 con 0x7f2ca80d0290 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2c9effd640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 0x7f2c8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f2ca408f9f0 con 0x7f2ca80d0290 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2cae1b0640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 0x7f2c8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.115+0000 7f2cae1b0640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 0x7f2c8c071910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b930 tx=0x7f2c98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.119+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2ca405c850 con 0x7f2ca80d0290 2024-05-17T22:01:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.291+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ca80b8080 con 0x7f2ca80d0290 2024-05-17T22:01:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.291+0000 7f2c9effd640 1 -- 172.21.15.83:0/1222349857 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7f2ca405c150 con 0x7f2ca80d0290 2024-05-17T22:01:27.286 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:01:27.286 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:01:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:01:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 msgr2=0x7f2c8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 0x7f2c8c071910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b930 tx=0x7f2c98008040 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 msgr2=0x7f2ca8157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d830 tx=0x7f2ca400dcf0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 shutdown_connections 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ca80d2000 0x7f2ca8157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca80d0290 0x7f2ca8157a30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2c8c06f400 0x7f2c8c071910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 --2- 172.21.15.83:0/1222349857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ca80b4810 0x7f2ca8161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 >> 172.21.15.83:0/1222349857 conn(0x7f2ca806e010 msgr2=0x7f2ca80b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 shutdown_connections 2024-05-17T22:01:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.295+0000 7f2cb0414640 1 -- 172.21.15.83:0/1222349857 wait complete. 2024-05-17T22:01:27.308 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:01:27.308 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:01:27.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:01:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 Processor -- start 2024-05-17T22:01:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 -- start start 2024-05-17T22:01:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 0x7fa79c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa79c143f20 0x7fa79c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa79c0737d0 con 0x7fa79c0733c0 2024-05-17T22:01:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa79c0b4110 con 0x7fa79c143f20 2024-05-17T22:01:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.519+0000 7fa7a2988640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c070770 con 0x7fa79c0b3d00 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa79c143f20 0x7fa79c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 0x7fa79c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa79c143f20 0x7fa79c072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:45058/0 (socket says 172.21.15.83:45058) 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 -- 172.21.15.83:0/3375025648 learned_addr learned my addr 172.21.15.83:0/3375025648 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa7a0f25640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa79c0733c0 0x7fa79c06eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:45650/0 (socket says 172.21.15.83:45650) 2024-05-17T22:01:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 -- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 msgr2=0x7fa79c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 0x7fa79c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 -- 172.21.15.83:0/3375025648 >> v1:172.21.15.155:6789/0 conn(0x7fa79c0733c0 legacy=0x7fa79c06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 -- 172.21.15.83:0/3375025648 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c0d10d0 con 0x7fa79c143f20 2024-05-17T22:01:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79bfff640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 0x7fa79c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:01:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79b7fe640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c143f20 0x7fa79c072e80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa790009870 tx=0x7fa79001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9395eea8627ba66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79affd640 1 -- 172.21.15.83:0/3375025648 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7900ab650 con 0x7fa79c143f20 2024-05-17T22:01:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79affd640 1 -- 172.21.15.83:0/3375025648 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7900b0070 con 0x7fa79c143f20 2024-05-17T22:01:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.523+0000 7fa79affd640 1 -- 172.21.15.83:0/3375025648 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7900aba10 con 0x7fa79c143f20 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c143f20 msgr2=0x7fa79c072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c143f20 0x7fa79c072e80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa790009870 tx=0x7fa79001c560 comp rx=0 tx=0).stop 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- 172.21.15.83:0/3375025648 shutdown_connections 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c143f20 0x7fa79c072e80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- 172.21.15.83:0/3375025648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0b3d00 0x7fa79c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- 172.21.15.83:0/3375025648 >> 172.21.15.83:0/3375025648 conn(0x7fa79c06cd90 msgr2=0x7fa79c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- 172.21.15.83:0/3375025648 shutdown_connections 2024-05-17T22:01:27.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- 172.21.15.83:0/3375025648 wait complete. 2024-05-17T22:01:27.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 Processor -- start 2024-05-17T22:01:27.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- start start 2024-05-17T22:01:27.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 0x7fa79c1f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c0b3d00 0x7fa79c13ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c070c00 con 0x7fa79c143f20 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c144330 con 0x7fa79c0b3d00 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.527+0000 7fa7a2988640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c1f7230 con 0x7fa79c0733c0 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50814/0 (socket says 172.21.15.83:50814) 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 -- 172.21.15.83:0/228227991 learned_addr learned my addr 172.21.15.83:0/228227991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa79bfff640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 0x7fa79c1f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa79b7fe640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c0b3d00 0x7fa79c13ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 -- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 msgr2=0x7fa79c1f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 0x7fa79c1f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 -- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c0b3d00 msgr2=0x7fa79c13ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c0b3d00 0x7fa79c13ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 -- 172.21.15.83:0/228227991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c13f8e0 con 0x7fa79c143f20 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa79bfff640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 0x7fa79c1f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a0f25640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa79400c6b0 tx=0x7fa79400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794018070 con 0x7fa79c143f20 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa794004d70 con 0x7fa79c143f20 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c13fbc0 con 0x7fa79c143f20 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79400a860 con 0x7fa79c143f20 2024-05-17T22:01:27.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c140180 con 0x7fa79c143f20 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c0d0a10 con 0x7fa79c143f20 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa79401b070 con 0x7fa79c143f20 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 0x7fa788071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fa79408e830 con 0x7fa79c143f20 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa79bfff640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 0x7fa788071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:27.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.531+0000 7fa79bfff640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 0x7fa788071910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009a40 tx=0x7fa78c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:27.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.535+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa79405b720 con 0x7fa79c143f20 2024-05-17T22:01:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa79c13f670 con 0x7fa79c143f20 2024-05-17T22:01:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa798ff9640 1 -- 172.21.15.83:0/228227991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7fa79405b020 con 0x7fa79c143f20 2024-05-17T22:01:27.703 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:01:27.703 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:01:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 msgr2=0x7fa788071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 0x7fa788071910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009a40 tx=0x7fa78c008040 comp rx=0 tx=0).stop 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 msgr2=0x7fa79c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa79400c6b0 tx=0x7fa79400cb70 comp rx=0 tx=0).stop 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 shutdown_connections 2024-05-17T22:01:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa79c143f20 0x7fa79c15b580 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa78806f400 0x7fa788071910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa79c0b3d00 0x7fa79c13ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 --2- 172.21.15.83:0/228227991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79c0733c0 0x7fa79c1f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 >> 172.21.15.83:0/228227991 conn(0x7fa79c06cd90 msgr2=0x7fa79c13cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 shutdown_connections 2024-05-17T22:01:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:27.711+0000 7fa7a2988640 1 -- 172.21.15.83:0/228227991 wait complete. 2024-05-17T22:01:27.724 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:01:27.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:27.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:01:27.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[41627]: fuse finished with error 0 and tester_r 0 2024-05-17T22:01:27.736 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:01:29.321 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:01:29.327+0000 7fb429809640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T22:01:30.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:01:30.397 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:01:33.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:01:33.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:33.838 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:01:33.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:33.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-17T22:01:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 Processor -- start 2024-05-17T22:01:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 -- start start 2024-05-17T22:01:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a400d1a80 0x7f7a400d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a4006cea0 0x7f7a400af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4006f7c0 con 0x7f7a400ce110 2024-05-17T22:01:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7a400d0680 con 0x7f7a400d1a80 2024-05-17T22:01:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a44f28640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a40160420 con 0x7f7a4006cea0 2024-05-17T22:01:34.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3e59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a400d1a80 0x7f7a400d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a400ce110 0x7f7a4006c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51074/0 (socket says 172.21.15.83:51074) 2024-05-17T22:01:34.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/3561798436 learned_addr learned my addr 172.21.15.83:0/3561798436 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a4006cea0 0x7f7a400af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3d59a640 1 -- 172.21.15.83:0/3561798436 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131676868 0 0) 0x7f7a4006f7c0 con 0x7f7a400ce110 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3d59a640 1 -- 172.21.15.83:0/3561798436 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a28003530 con 0x7f7a400ce110 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3e59c640 1 -- 172.21.15.83:0/3561798436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a4006cea0 msgr2=0x7f7a400af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a4006cea0 0x7f7a400af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3e59c640 1 -- 172.21.15.83:0/3561798436 >> v1:172.21.15.83:6789/0 conn(0x7f7a400ce110 legacy=0x7f7a4006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.063+0000 7f7a3e59c640 1 -- 172.21.15.83:0/3561798436 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a400cd670 con 0x7f7a400d1a80 2024-05-17T22:01:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400d0140 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009d90 tx=0x7f7a3801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eb67f7b442c196 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a3d59a640 1 -- 172.21.15.83:0/3561798436 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380b2070 con 0x7f7a400d1a80 2024-05-17T22:01:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a3d59a640 1 -- 172.21.15.83:0/3561798436 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a380a5830 con 0x7f7a400d1a80 2024-05-17T22:01:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a3d59a640 1 -- 172.21.15.83:0/3561798436 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380a5c00 con 0x7f7a400d1a80 2024-05-17T22:01:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 -- 172.21.15.83:0/3561798436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 msgr2=0x7f7a400d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400d0140 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009d90 tx=0x7f7a3801c970 comp rx=0 tx=0).stop 2024-05-17T22:01:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 -- 172.21.15.83:0/3561798436 shutdown_connections 2024-05-17T22:01:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a4006cea0 0x7f7a400af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3561798436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400d0140 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 -- 172.21.15.83:0/3561798436 >> 172.21.15.83:0/3561798436 conn(0x7f7a400b4100 msgr2=0x7f7a400b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 -- 172.21.15.83:0/3561798436 shutdown_connections 2024-05-17T22:01:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.067+0000 7f7a44f28640 1 -- 172.21.15.83:0/3561798436 wait complete. 2024-05-17T22:01:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 Processor -- start 2024-05-17T22:01:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- start start 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a400ce110 0x7f7a400b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 0x7f7a400b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400d22f0 con 0x7f7a400d1a80 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a400afd30 con 0x7f7a4014f340 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a400b7ee0 con 0x7f7a400ce110 2024-05-17T22:01:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50820/0 (socket says 172.21.15.83:50820) 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/3780175898 learned_addr learned my addr 172.21.15.83:0/3780175898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a400ce110 0x7f7a400b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 0x7f7a400b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a400ce110 msgr2=0x7f7a400b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a400ce110 0x7f7a400b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 msgr2=0x7f7a400b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 0x7f7a400b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/3780175898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a400b27c0 con 0x7f7a400d1a80 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 0x7f7a400b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c460 tx=0x7f7a3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a2effd640 1 -- 172.21.15.83:0/3780175898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34017070 con 0x7f7a400d1a80 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a2effd640 1 -- 172.21.15.83:0/3780175898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34005cf0 con 0x7f7a400d1a80 2024-05-17T22:01:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4015ec10 con 0x7f7a400d1a80 2024-05-17T22:01:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a2effd640 1 -- 172.21.15.83:0/3780175898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34007270 con 0x7f7a400d1a80 2024-05-17T22:01:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.071+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a400b72b0 con 0x7f7a400d1a80 2024-05-17T22:01:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a2effd640 1 -- 172.21.15.83:0/3780175898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a3401a070 con 0x7f7a400d1a80 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a2effd640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 0x7f7a20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a2effd640 1 -- 172.21.15.83:0/3780175898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f7a3408e510 con 0x7f7a400d1a80 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 0x7f7a20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a4c1a2000 1 Processor -- start 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 0x7f7a20071940 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a380af000 tx=0x7f7a380a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.075+0000 7f7a4c1a2000 1 -- start start 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b55060 0x564c32ca6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32ca6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c32a5b6c0 con 0x564c32b3eb30 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564c32ab5fd0 con 0x564c32b2fc20 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x564c32aa5b70 con 0x564c32b55060 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50836/0 (socket says 172.21.15.83:50836) 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/1090194686 learned_addr learned my addr 172.21.15.83:0/1090194686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32ca6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b55060 msgr2=0x564c32ca6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b55060 0x564c32ca6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b55060 0x564c32ca6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 msgr2=0x564c32ca6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32ca6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 -- 172.21.15.83:0/1090194686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c32af0f50 con 0x564c32b3eb30 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32ca6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005840 tx=0x7f7a3405a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec160b5d8643cd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a1ffff640 1 -- 172.21.15.83:0/1090194686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3405f010 con 0x564c32b3eb30 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a1ffff640 1 -- 172.21.15.83:0/1090194686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34063a30 con 0x564c32b3eb30 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a1ffff640 1 -- 172.21.15.83:0/1090194686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3408e310 con 0x564c32b3eb30 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1090194686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 msgr2=0x564c32ca6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7a34005840 tx=0x7f7a3405a5b0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1090194686 shutdown_connections 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32ca6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b3eb30 0x564c32ca6550 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1090194686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b55060 0x564c32ca6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1090194686 >> 172.21.15.83:0/1090194686 conn(0x564c32b75d60 msgr2=0x564c32c5b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1090194686 shutdown_connections 2024-05-17T22:01:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1090194686 wait complete. 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 -- start start 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.079+0000 7f7a4c1a2000 1 Processor -- start 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32c68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b55060 0x564c32c68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c32a61e20 con 0x564c32b2fc20 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564c32aa9bb0 con 0x564c32b55060 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x564c3276f860 con 0x564c32b3eb30 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 10 client.0 _ll_register_callbacks cb 0x564c32c47d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:34.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50852/0 (socket says 172.21.15.83:50852) 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 -- 172.21.15.83:0/841531347 learned_addr learned my addr 172.21.15.83:0/841531347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 -- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 msgr2=0x564c32c68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32c68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b55060 0x564c32c68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 -- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b55060 msgr2=0x564c32c68aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b55060 0x564c32c68aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 -- 172.21.15.83:0/841531347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c32a5e6c0 con 0x564c32b2fc20 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7a3801ce50 tx=0x7f7a3801b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df134868d9a41ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380b2040 con 0x564c32b2fc20 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f7a380b85c0 con 0x564c32b2fc20 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a34063a30 con 0x564c32b2fc20 2024-05-17T22:01:34.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7a380b87c0 con 0x564c32b2fc20 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 10 client.6327 fetch_fsmap learned FSMap version 50 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x564c32b76170 con 0x564c32b2fc20 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34063a30 con 0x564c32b2fc20 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f7a380b8960 con 0x564c32b2fc20 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 10 client.6327 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 10 client.6327 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 20 client.6327 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3274cb10 0x564c32cb75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 4 client.6327 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c32a41af0 con 0x564c3274cb10 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3274cb10 0x564c32cb75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 10 client.6327 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a3dd9b640 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3274cb10 0x564c32cb75d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39abc615923c23a server_cookie=21d9b12c2e9760bd in_seq=0 out_seq=0 2024-05-17T22:01:34.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/841531347 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c32a41af0 con 0x564c3274cb10 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 10 client.6327 handle_command_reply: tid=0 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 client.6327 shutdown 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 msgr2=0x564c32c68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7a3801ce50 tx=0x7f7a3801b1b0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 shutdown_connections 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b55060 0x564c32c68aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32c68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3274cb10 0x564c32cb75d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/841531347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b2fc20 0x564c32c68020 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a1d7fa640 0 client.6327 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 >> 172.21.15.83:0/841531347 conn(0x564c32b75d60 msgr2=0x564c32c5b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 shutdown_connections 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/841531347 wait complete. 2024-05-17T22:01:34.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.083+0000 7f7a4c1a2000 20 client.6327 trim_cache size 0 max 16384 2024-05-17T22:01:34.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 Processor -- start 2024-05-17T22:01:34.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 -- start start 2024-05-17T22:01:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b2fc20 0x564c32c5cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b3eb30 0x564c32c5d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40205f80 con 0x564c32b55060 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40204fe0 con 0x564c32b3eb30 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a40157a70 con 0x564c32b2fc20 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50862/0 (socket says 172.21.15.83:50862) 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/2835407151 learned_addr learned my addr 172.21.15.83:0/2835407151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b2fc20 msgr2=0x564c32c5cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b2fc20 0x564c32c5cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b3eb30 msgr2=0x564c32c5d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b3eb30 0x564c32c5d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/2835407151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c32b2dbe0 con 0x564c32b55060 2024-05-17T22:01:34.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000e8c0 tx=0x7f7a3000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10e0f8cc0081681 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a1f7fe640 1 -- 172.21.15.83:0/2835407151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300109c0 con 0x564c32b55060 2024-05-17T22:01:34.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a1f7fe640 1 -- 172.21.15.83:0/2835407151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30010b60 con 0x564c32b55060 2024-05-17T22:01:34.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.103+0000 7f7a1f7fe640 1 -- 172.21.15.83:0/2835407151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30014450 con 0x564c32b55060 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/2835407151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 msgr2=0x564c32cd1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000e8c0 tx=0x7f7a3000ed80 comp rx=0 tx=0).stop 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/2835407151 shutdown_connections 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32cd1480 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b3eb30 0x564c32c5d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/2835407151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b2fc20 0x564c32c5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/2835407151 >> 172.21.15.83:0/2835407151 conn(0x564c32b75d60 msgr2=0x564c32c5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/2835407151 shutdown_connections 2024-05-17T22:01:34.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/2835407151 wait complete. 2024-05-17T22:01:34.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- start start 2024-05-17T22:01:34.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 Processor -- start 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32d19460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32d199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c32a61e20 con 0x564c32b55060 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564c32c65790 con 0x564c32b2fc20 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x564c32a83770 con 0x564c32b3eb30 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 10 client.0 _ll_register_callbacks cb 0x7f7a380b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:34.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50868/0 (socket says 172.21.15.83:50868) 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/1246347800 learned_addr learned my addr 172.21.15.83:0/1246347800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 msgr2=0x564c32d199a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32d199a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32d19460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 msgr2=0x564c32d19460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32d19460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 -- 172.21.15.83:0/1246347800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c32b04110 con 0x564c32b55060 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a3ed9d640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x564c32a73ab0 tx=0x7f7a30004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae3e93b663b3aef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30021070 con 0x564c32b55060 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f7a30004220 con 0x564c32b55060 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a30010b60 con 0x564c32b55060 2024-05-17T22:01:34.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7a30004420 con 0x564c32b55060 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 10 client.6333 fetch_fsmap learned FSMap version 50 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x564c32a43e30 con 0x564c32b55060 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.107+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30010b60 con 0x564c32b55060 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f7a3002f050 con 0x564c32b55060 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 10 client.6333 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 10 client.6333 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 20 client.6333 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3273ab70 0x564c32cd48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 4 client.6333 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x564c32ab7f20 con 0x564c3273ab70 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3273ab70 0x564c32cd48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a1d7fa640 10 client.6333 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a3e59c640 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3273ab70 0x564c32cd48a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb32de57a47f213e server_cookie=955042f33b785997 in_seq=0 out_seq=0 2024-05-17T22:01:34.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a1d7fa640 1 -- 172.21.15.83:0/1246347800 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c32ab7f20 con 0x564c3273ab70 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a1d7fa640 10 client.6333 handle_command_reply: tid=0 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 client.6333 shutdown 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 msgr2=0x564c32d19ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x564c32a73ab0 tx=0x7f7a30004740 comp rx=0 tx=0).stop 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 shutdown_connections 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x564c3273ab70 0x564c32cd48a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c32b55060 0x564c32d19ee0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564c32b3eb30 0x564c32d199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a1d7fa640 0 client.6333 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 --2- 172.21.15.83:0/1246347800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c32b2fc20 0x564c32d19460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 >> 172.21.15.83:0/1246347800 conn(0x564c32b75d60 msgr2=0x564c32c5c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 shutdown_connections 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 1 -- 172.21.15.83:0/1246347800 wait complete. 2024-05-17T22:01:34.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a4c1a2000 20 client.6333 trim_cache size 0 max 16384 2024-05-17T22:01:34.111 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:34.111 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:01:34.111 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "3ab0ba70-945c-4097-b36b-7972c240a4d1", 2024-05-17T22:01:34.111 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:01:34.111 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 msgr2=0x7f7a20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 0x7f7a20071940 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a380af000 tx=0x7f7a380a7070 comp rx=0 tx=0).stop 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 msgr2=0x7f7a400afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c460 tx=0x7f7a3400c920 comp rx=0 tx=0).stop 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 shutdown_connections 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a2006f430 0x7f7a20071940 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a4014f340 0x7f7a400b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a400d1a80 0x7f7a400afeb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 --2- 172.21.15.83:0/3780175898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a400ce110 0x7f7a400b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 >> 172.21.15.83:0/3780175898 conn(0x7f7a400b4100 msgr2=0x7f7a40070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 shutdown_connections 2024-05-17T22:01:34.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.111+0000 7f7a44f28640 1 -- 172.21.15.83:0/3780175898 wait complete. 2024-05-17T22:01:34.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:01:34.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-17T22:01:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.275+0000 7f1e5ee24640 1 Processor -- start 2024-05-17T22:01:34.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.275+0000 7f1e5ee24640 1 -- start start 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5ee24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e580cf830 0x7f1e580b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5ee24640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cdf40 0x7f1e5815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5ee24640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e580b5640 con 0x7f1e5815d530 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5ee24640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1e580b58e0 con 0x7f1e580cf830 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5ee24640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5806f060 con 0x7f1e580cdf40 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e580cf830 0x7f1e580b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5d3c1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1e5815d530 0x7f1e5806f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51090/0 (socket says 172.21.15.83:51090) 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/1192564891 learned_addr learned my addr 172.21.15.83:0/1192564891 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e57fff640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cdf40 0x7f1e5815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e577fe640 1 -- 172.21.15.83:0/1192564891 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3520997754 0 0) 0x7f1e580b5640 con 0x7f1e5815d530 2024-05-17T22:01:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e577fe640 1 -- 172.21.15.83:0/1192564891 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e44005740 con 0x7f1e5815d530 2024-05-17T22:01:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/1192564891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cdf40 msgr2=0x7f1e5815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cdf40 0x7f1e5815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/1192564891 >> v1:172.21.15.83:6789/0 conn(0x7f1e5815d530 legacy=0x7f1e5806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/1192564891 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5806f340 con 0x7f1e580cf830 2024-05-17T22:01:34.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cf830 0x7f1e580b3aa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097c0 tx=0x7f1e5001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa8e922e422132e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e577fe640 1 -- 172.21.15.83:0/1192564891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500b1070 con 0x7f1e580cf830 2024-05-17T22:01:34.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e577fe640 1 -- 172.21.15.83:0/1192564891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500ac410 con 0x7f1e580cf830 2024-05-17T22:01:34.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.279+0000 7f1e577fe640 1 -- 172.21.15.83:0/1192564891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500ac7e0 con 0x7f1e580cf830 2024-05-17T22:01:34.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- 172.21.15.83:0/1192564891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cf830 msgr2=0x7f1e580b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cf830 0x7f1e580b3aa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097c0 tx=0x7f1e5001c4a0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- 172.21.15.83:0/1192564891 shutdown_connections 2024-05-17T22:01:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cdf40 0x7f1e5815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/1192564891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cf830 0x7f1e580b3aa0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- 172.21.15.83:0/1192564891 >> 172.21.15.83:0/1192564891 conn(0x7f1e5806cd90 msgr2=0x7f1e580b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- 172.21.15.83:0/1192564891 shutdown_connections 2024-05-17T22:01:34.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- 172.21.15.83:0/1192564891 wait complete. 2024-05-17T22:01:34.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 Processor -- start 2024-05-17T22:01:34.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- start start 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cf830 0x7f1e581477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5815d530 0x7f1e58156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58073210 con 0x7f1e580cdf40 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58073090 con 0x7f1e5815d530 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5ee24640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e58073390 con 0x7f1e580cf830 2024-05-17T22:01:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5cbc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5cbc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50888/0 (socket says 172.21.15.83:50888) 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.283+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/2413911582 learned_addr learned my addr 172.21.15.83:0/2413911582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e57fff640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cf830 0x7f1e581477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cf830 msgr2=0x7f1e581477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cf830 0x7f1e581477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5815d530 msgr2=0x7f1e58156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5815d530 0x7f1e58156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5815d530 0x7f1e58156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/2413911582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e58154ea0 con 0x7f1e580cdf40 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1e500af040 tx=0x7f1e5001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 -- 172.21.15.83:0/2413911582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500b1050 con 0x7f1e580cdf40 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 -- 172.21.15.83:0/2413911582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500b2040 con 0x7f1e580cdf40 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e581514e0 con 0x7f1e580cdf40 2024-05-17T22:01:34.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 -- 172.21.15.83:0/2413911582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50006e20 con 0x7f1e580cdf40 2024-05-17T22:01:34.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e58151aa0 con 0x7f1e580cdf40 2024-05-17T22:01:34.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 -- 172.21.15.83:0/2413911582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e500bf060 con 0x7f1e580cdf40 2024-05-17T22:01:34.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 0x7f1e38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e557fa640 1 -- 172.21.15.83:0/2413911582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f1e50128ff0 con 0x7f1e580cdf40 2024-05-17T22:01:34.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e57fff640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 0x7f1e38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.287+0000 7f1e6609e000 1 Processor -- start 2024-05-17T22:01:34.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- start start 2024-05-17T22:01:34.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e57fff640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 0x7f1e38071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e58073da0 tx=0x7f1e40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e8c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e8c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56403e765a90 con 0x56403e760f80 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56403e6d9120 con 0x56403e74e090 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56403e6c8cc0 con 0x56403e74d880 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50904/0 (socket says 172.21.15.83:50904) 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/770572667 learned_addr learned my addr 172.21.15.83:0/770572667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 msgr2=0x56403e8c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e8c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e8c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 msgr2=0x56403e8c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e8c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 -- 172.21.15.83:0/770572667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403e685110 con 0x56403e760f80 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1e500a6010 tx=0x7f1e500f5160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97edede62b861f84 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e4e7fc640 1 -- 172.21.15.83:0/770572667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500f5650 con 0x56403e760f80 2024-05-17T22:01:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e4e7fc640 1 -- 172.21.15.83:0/770572667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500f9a00 con 0x56403e760f80 2024-05-17T22:01:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e4e7fc640 1 -- 172.21.15.83:0/770572667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500fe4b0 con 0x56403e760f80 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- 172.21.15.83:0/770572667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 msgr2=0x56403e8c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1e500a6010 tx=0x7f1e500f5160 comp rx=0 tx=0).stop 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- 172.21.15.83:0/770572667 shutdown_connections 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e8c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e8c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 --2- 172.21.15.83:0/770572667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8c6e20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- 172.21.15.83:0/770572667 >> 172.21.15.83:0/770572667 conn(0x56403e79c010 msgr2=0x56403e87c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- 172.21.15.83:0/770572667 shutdown_connections 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.291+0000 7f1e6609e000 1 -- 172.21.15.83:0/770572667 wait complete. 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- start start 2024-05-17T22:01:34.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 Processor -- start 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e883a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e883f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56403e684ec0 con 0x56403e760f80 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56403e6ccd00 con 0x56403e74e090 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56403e392680 con 0x56403e74d880 2024-05-17T22:01:34.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 10 client.0 _ll_register_callbacks cb 0x56403e7801c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50908/0 (socket says 172.21.15.83:50908) 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/3746814673 learned_addr learned my addr 172.21.15.83:0/3746814673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e883a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e57fff640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e883f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 msgr2=0x56403e883a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e883a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 msgr2=0x56403e883f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e883f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/3746814673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403e687890 con 0x56403e760f80 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e57fff640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e883f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800e810 tx=0x7f1e4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd9340c1a6d3c80 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800c930 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f1e4800cad0 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e500f9a00 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1e48020050 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 10 client.6348 fetch_fsmap learned FSMap version 50 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x56403e689650 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500f9a00 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f1e48023020 con 0x56403e760f80 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 10 client.6348 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 10 client.6348 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 20 client.6348 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e36f930 0x56403e8cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 4 client.6348 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:01:34.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56403e664c40 con 0x56403e36f930 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e36f930 0x56403e8cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e377fe640 10 client.6348 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.295+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e36f930 0x56403e8cde90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299a591c823805b6 server_cookie=6f7434c8ae9b9606 in_seq=0 out_seq=0 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e377fe640 1 -- 172.21.15.83:0/3746814673 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56403e664c40 con 0x56403e36f930 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e377fe640 10 client.6348 handle_command_reply: tid=0 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 client.6348 shutdown 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 msgr2=0x56403e888be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800e810 tx=0x7f1e4800ecd0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 shutdown_connections 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e36f930 0x56403e8cde90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e888be0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e883f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e377fe640 0 client.6348 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 --2- 172.21.15.83:0/3746814673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e883a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 >> 172.21.15.83:0/3746814673 conn(0x56403e79c010 msgr2=0x56403e87c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 shutdown_connections 2024-05-17T22:01:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 1 -- 172.21.15.83:0/3746814673 wait complete. 2024-05-17T22:01:34.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.299+0000 7f1e6609e000 20 client.6348 trim_cache size 0 max 16384 2024-05-17T22:01:34.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.315+0000 7f1e6609e000 1 Processor -- start 2024-05-17T22:01:34.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.315+0000 7f1e6609e000 1 -- start start 2024-05-17T22:01:34.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e935bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e9360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e582013e0 con 0x56403e760f80 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5814e1f0 con 0x56403e74e090 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5814a4b0 con 0x56403e74d880 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50922/0 (socket says 172.21.15.83:50922) 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/230978863 learned_addr learned my addr 172.21.15.83:0/230978863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e935bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 msgr2=0x56403e935bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e935bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e57fff640 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e9360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 msgr2=0x56403e9360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e9360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/230978863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403e880630 con 0x56403e760f80 2024-05-17T22:01:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001690 tx=0x7f1e48007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05dcdc93b6ebd52 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e4dffb640 1 -- 172.21.15.83:0/230978863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800cb30 con 0x56403e760f80 2024-05-17T22:01:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e4dffb640 1 -- 172.21.15.83:0/230978863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e48018070 con 0x56403e760f80 2024-05-17T22:01:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e4dffb640 1 -- 172.21.15.83:0/230978863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48013550 con 0x56403e760f80 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- 172.21.15.83:0/230978863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 msgr2=0x56403e936630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001690 tx=0x7f1e48007000 comp rx=0 tx=0).stop 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- 172.21.15.83:0/230978863 shutdown_connections 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e936630 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74e090 0x56403e9360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 --2- 172.21.15.83:0/230978863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74d880 0x56403e935bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- 172.21.15.83:0/230978863 >> 172.21.15.83:0/230978863 conn(0x56403e79c010 msgr2=0x56403e8829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- 172.21.15.83:0/230978863 shutdown_connections 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- 172.21.15.83:0/230978863 wait complete. 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 -- start start 2024-05-17T22:01:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.319+0000 7f1e6609e000 1 Processor -- start 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74d880 0x56403e87e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74e090 0x56403e87ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56403e68ff10 con 0x56403e760f80 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58152c60 con 0x56403e74d880 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56403e788d50 con 0x56403e74e090 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 10 client.0 _ll_register_callbacks cb 0x7f1e48001800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:01:34.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50932/0 (socket says 172.21.15.83:50932) 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/1329040971 learned_addr learned my addr 172.21.15.83:0/1329040971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74d880 0x56403e87e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e57fff640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74e090 0x56403e87ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74e090 msgr2=0x56403e87ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74e090 0x56403e87ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74d880 msgr2=0x56403e87e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74d880 0x56403e87e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 -- 172.21.15.83:0/1329040971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403e6688c0 con 0x56403e760f80 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5d3c1640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001660 tx=0x7f1e48005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fde279284b2d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48013990 con 0x56403e760f80 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f1e48013b30 con 0x56403e760f80 2024-05-17T22:01:34.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48018070 con 0x56403e760f80 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1e4803e050 con 0x56403e760f80 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 10 client.6354 fetch_fsmap learned FSMap version 50 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x56403e683fa0 con 0x56403e760f80 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e48018070 con 0x56403e760f80 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f1e48041020 con 0x56403e760f80 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 10 client.6354 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 10 client.6354 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 20 client.6354 populate_metadata read hostname 'smithi083' 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e35d990 0x56403e8f7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 4 client.6354 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56403e6db070 con 0x56403e35d990 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e35d990 0x56403e8f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 10 client.6354 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5cbc0640 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e35d990 0x56403e8f7710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e4ff8a50f77995 server_cookie=be4d3a4a872a0b13 in_seq=0 out_seq=0 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 1 -- 172.21.15.83:0/1329040971 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56403e6db070 con 0x56403e35d990 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 10 client.6354 handle_command_reply: tid=0 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 client.6354 shutdown 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 msgr2=0x56403e8f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001660 tx=0x7f1e48005b80 comp rx=0 tx=0).stop 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 shutdown_connections 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x56403e35d990 0x56403e8f7710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56403e760f80 0x56403e8f6d80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56403e74e090 0x56403e87ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e377fe640 0 client.6354 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 --2- 172.21.15.83:0/1329040971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403e74d880 0x56403e87e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 >> 172.21.15.83:0/1329040971 conn(0x56403e79c010 msgr2=0x56403e883390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 shutdown_connections 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 1 -- 172.21.15.83:0/1329040971 wait complete. 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e6609e000 20 client.6354 trim_cache size 0 max 16384 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "3ab0ba70-945c-4097-b36b-7972c240a4d1": { 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/subdir", 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "3ab0ba70-945c-4097-b36b-7972c240a4d1", 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-05-17T22:01:34.322 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:01:34.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:01:34.323 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:01:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 msgr2=0x7f1e38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 0x7f1e38071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1e58073da0 tx=0x7f1e40008040 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 msgr2=0x7f1e5814fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1e500af040 tx=0x7f1e5001ace0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 shutdown_connections 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1e3806f400 0x7f1e38071910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5815d530 0x7f1e58156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e580cf830 0x7f1e581477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 --2- 172.21.15.83:0/2413911582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e580cdf40 0x7f1e5814fbd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.323+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 >> 172.21.15.83:0/2413911582 conn(0x7f1e5806cd90 msgr2=0x7f1e580b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.327+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 shutdown_connections 2024-05-17T22:01:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:01:34.327+0000 7f1e5ee24640 1 -- 172.21.15.83:0/2413911582 wait complete. 2024-05-17T22:01:34.337 INFO:tasks.cephfs.filesystem:scrub status for tag:3ab0ba70-945c-4097-b36b-7972c240a4d1 - {'path': '/subdir', 'tag': '3ab0ba70-945c-4097-b36b-7972c240a4d1', 'options': 'recursive'} 2024-05-17T22:02:00.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:02:00.505 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:02:04.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:04.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-17T22:02:04.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 Processor -- start 2024-05-17T22:02:04.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 -- start start 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 0x7f50f41601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f406f2d0 con 0x7f50f41606f0 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f50f406f5a0 con 0x7f50f415be70 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50fa55a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f50f406f880 con 0x7f50f406eec0 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50eb7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 0x7f50f41601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:34208/0 (socket says 172.21.15.83:34208) 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 -- 172.21.15.83:0/3396030383 learned_addr learned my addr 172.21.15.83:0/3396030383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 -- 172.21.15.83:0/3396030383 >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 msgr2=0x7f50f41601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3396030383 >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 0x7f50f41601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 -- 172.21.15.83:0/3396030383 >> v1:172.21.15.155:6789/0 conn(0x7f50f41606f0 legacy=0x7f50f40b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50f3fff640 1 -- 172.21.15.83:0/3396030383 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f406fb60 con 0x7f50f406eec0 2024-05-17T22:02:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.511+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3396030383 >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 0x7f50f41601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:02:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3396030383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009870 tx=0x7f50e401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a107842264cc4a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50f37fe640 1 -- 172.21.15.83:0/3396030383 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40ab5f0 con 0x7f50f406eec0 2024-05-17T22:02:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50f37fe640 1 -- 172.21.15.83:0/3396030383 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e40b0070 con 0x7f50f406eec0 2024-05-17T22:02:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50f37fe640 1 -- 172.21.15.83:0/3396030383 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40ab9b0 con 0x7f50f406eec0 2024-05-17T22:02:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 -- 172.21.15.83:0/3396030383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 msgr2=0x7f50f40b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 --2- 172.21.15.83:0/3396030383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009870 tx=0x7f50e401c560 comp rx=0 tx=0).stop 2024-05-17T22:02:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 -- 172.21.15.83:0/3396030383 shutdown_connections 2024-05-17T22:02:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 --2- 172.21.15.83:0/3396030383 >> v2:172.21.15.155:3300/0 conn(0x7f50f415be70 0x7f50f41601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 --2- 172.21.15.83:0/3396030383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f406eec0 0x7f50f40b08a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 -- 172.21.15.83:0/3396030383 >> 172.21.15.83:0/3396030383 conn(0x7f50f406cd90 msgr2=0x7f50f40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 -- 172.21.15.83:0/3396030383 shutdown_connections 2024-05-17T22:02:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.515+0000 7f50fa55a640 1 -- 172.21.15.83:0/3396030383 wait complete. 2024-05-17T22:02:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 Processor -- start 2024-05-17T22:02:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- start start 2024-05-17T22:02:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f415be70 0x7f50f41fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f41606f0 0x7f50f40cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4073000 con 0x7f50f406eec0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4072e80 con 0x7f50f415be70 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4073180 con 0x7f50f41606f0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58628/0 (socket says 172.21.15.83:58628) 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 -- 172.21.15.83:0/1425012664 learned_addr learned my addr 172.21.15.83:0/1425012664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f415be70 0x7f50f41fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f8af7640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f41606f0 0x7f50f40cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 -- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f41606f0 msgr2=0x7f50f40cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f41606f0 0x7f50f40cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 -- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f415be70 msgr2=0x7f50f41fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f415be70 0x7f50f41fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 -- 172.21.15.83:0/1425012664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f40aadd0 con 0x7f50f406eec0 2024-05-17T22:02:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f3fff640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ad040 tx=0x7f50e401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f17fa640 1 -- 172.21.15.83:0/1425012664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40be3f0 con 0x7f50f406eec0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f17fa640 1 -- 172.21.15.83:0/1425012664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e40be590 con 0x7f50f406eec0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f40ab0b0 con 0x7f50f406eec0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50f17fa640 1 -- 172.21.15.83:0/1425012664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40be960 con 0x7f50f406eec0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.519+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f40ab670 con 0x7f50f406eec0 2024-05-17T22:02:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50f17fa640 1 -- 172.21.15.83:0/1425012664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50e40b43f0 con 0x7f50f406eec0 2024-05-17T22:02:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50f17fa640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 0x7f50d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50f17fa640 1 -- 172.21.15.83:0/1425012664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f50e4129630 con 0x7f50f406eec0 2024-05-17T22:02:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 0x7f50d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 0x7f50d4071910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f50f415ecd0 tx=0x7f50dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 Processor -- start 2024-05-17T22:02:04.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 -- start start 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cae4f80 0x563e2cc4ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2cc4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e2cae9a90 con 0x563e2cad1880 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca5d120 con 0x563e2cad2090 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563e2ca4ccc0 con 0x563e2cae4f80 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58634/0 (socket says 172.21.15.83:58634) 2024-05-17T22:02:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.523+0000 7f50eb7fe640 1 -- 172.21.15.83:0/2380181799 learned_addr learned my addr 172.21.15.83:0/2380181799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50f3fff640 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cae4f80 0x563e2cc4ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50f8af7640 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2cc4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 -- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cae4f80 msgr2=0x563e2cc4ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cae4f80 0x563e2cc4ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 -- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 msgr2=0x563e2cc4b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2cc4b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 -- 172.21.15.83:0/2380181799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2ca09110 con 0x563e2cad1880 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50dc011cc0 tx=0x7f50dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05c65126270c301 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50e9ffb640 1 -- 172.21.15.83:0/2380181799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc011cf0 con 0x563e2cad1880 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50e9ffb640 1 -- 172.21.15.83:0/2380181799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50dc026070 con 0x563e2cad1880 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f50e9ffb640 1 -- 172.21.15.83:0/2380181799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc021680 con 0x563e2cad1880 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- 172.21.15.83:0/2380181799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 msgr2=0x563e2cc4b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f50dc011cc0 tx=0x7f50dc014000 comp rx=0 tx=0).stop 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- 172.21.15.83:0/2380181799 shutdown_connections 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2cc4b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2cc4b360 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- 172.21.15.83:0/2380181799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cae4f80 0x563e2cc4ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- 172.21.15.83:0/2380181799 >> 172.21.15.83:0/2380181799 conn(0x563e2cb20010 msgr2=0x563e2cc005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- 172.21.15.83:0/2380181799 shutdown_connections 2024-05-17T22:02:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- 172.21.15.83:0/2380181799 wait complete. 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- start start 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 Processor -- start 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 0x563e2cc0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca08ec0 con 0x563e2cad2090 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca50d00 con 0x563e2cae4f80 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563e2c716680 con 0x563e2cad1880 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.527+0000 7f51017d4000 10 client.0 _ll_register_callbacks cb 0x563e2cb041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:04.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58636/0 (socket says 172.21.15.83:58636) 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 -- 172.21.15.83:0/3350279786 learned_addr learned my addr 172.21.15.83:0/3350279786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 0x563e2cc0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 -- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 msgr2=0x563e2cc07a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc07a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 -- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 msgr2=0x563e2cc0cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 0x563e2cc0cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 -- 172.21.15.83:0/3350279786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2ca0b890 con 0x563e2cad2090 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 0x563e2cc0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f50dc008010 tx=0x7f50dc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ed1331a95b2c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc021940 con 0x563e2cad2090 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f50dc021ae0 con 0x563e2cad2090 2024-05-17T22:02:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50dc026070 con 0x563e2cad2090 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f50dc03e050 con 0x563e2cad2090 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 10 client.6363 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x563e2ca0d650 con 0x563e2cad2090 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50dc026070 con 0x563e2cad2090 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f50dc041020 con 0x563e2cad2090 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 10 client.6363 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 10 client.6363 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 20 client.6363 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c6f3930 0x563e2cc51e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 4 client.6363 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e2c9e8c40 con 0x563e2c6f3930 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c6f3930 0x563e2cc51e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 10 client.6363 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c6f3930 0x563e2cc51e90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3069a4818db886e4 server_cookie=d0e780105976dd0b in_seq=0 out_seq=0 2024-05-17T22:02:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3350279786 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563e2c9e8c40 con 0x563e2c6f3930 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 10 client.6363 handle_command_reply: tid=0 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 client.6363 shutdown 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 msgr2=0x563e2cc07f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f50dc008010 tx=0x7f50dc0046d0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 shutdown_connections 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cae4f80 0x563e2cc0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c6f3930 0x563e2cc51e90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc07f70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 --2- 172.21.15.83:0/3350279786 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc07a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f50d2ffd640 0 client.6363 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 >> 172.21.15.83:0/3350279786 conn(0x563e2cb20010 msgr2=0x563e2cc005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 shutdown_connections 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 1 -- 172.21.15.83:0/3350279786 wait complete. 2024-05-17T22:02:04.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.531+0000 7f51017d4000 20 client.6363 trim_cache size 0 max 16384 2024-05-17T22:02:04.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 Processor -- start 2024-05-17T22:02:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 -- start start 2024-05-17T22:02:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cb20010 0x563e2cc61410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc61950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4205790 con 0x563e2cad2090 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f50f42048d0 con 0x563e2cb20010 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f50f41f9f00 con 0x563e2cad1880 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58650/0 (socket says 172.21.15.83:58650) 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 -- 172.21.15.83:0/3935002717 learned_addr learned my addr 172.21.15.83:0/3935002717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cb20010 0x563e2cc61410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc61950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 -- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 msgr2=0x563e2cc61950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc61950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 -- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cb20010 msgr2=0x563e2cc61410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cb20010 0x563e2cc61410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 -- 172.21.15.83:0/3935002717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e2c9ec8c0 con 0x563e2cad2090 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50ec001cf0 tx=0x7f50ec0021b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4d73871a8321f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50e97fa640 1 -- 172.21.15.83:0/3935002717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec010e10 con 0x563e2cad2090 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50e97fa640 1 -- 172.21.15.83:0/3935002717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec019070 con 0x563e2cad2090 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.551+0000 7f50e97fa640 1 -- 172.21.15.83:0/3935002717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003b80 con 0x563e2cad2090 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3935002717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 msgr2=0x563e2cc61e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50ec001cf0 tx=0x7f50ec0021b0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3935002717 shutdown_connections 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad2090 0x563e2cc61e90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cad1880 0x563e2cc61950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- 172.21.15.83:0/3935002717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cb20010 0x563e2cc61410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3935002717 >> 172.21.15.83:0/3935002717 conn(0x563e2cc04400 msgr2=0x563e2cc069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3935002717 shutdown_connections 2024-05-17T22:02:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3935002717 wait complete. 2024-05-17T22:02:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- start start 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 Processor -- start 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2ccbe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cb20010 0x563e2cc01bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e2ca55330 con 0x563e2cad1880 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x563e2c9f40f0 con 0x563e2cad2090 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x563e2c716680 con 0x563e2cb20010 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 10 client.0 _ll_register_callbacks cb 0x7f50dc011b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58666/0 (socket says 172.21.15.83:58666) 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 -- 172.21.15.83:0/3232945607 learned_addr learned my addr 172.21.15.83:0/3232945607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2ccbe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f8af7640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cb20010 0x563e2cc01bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 -- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cb20010 msgr2=0x563e2cc01bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cb20010 0x563e2cc01bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 -- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 msgr2=0x563e2ccbe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2ccbe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 -- 172.21.15.83:0/3232945607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2ca07fa0 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50f3fff640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f50e40f24c0 tx=0x7f50e40f5950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b2c5260e933804 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40c4040 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f50e40f2160 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ec019070 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f50e40fa1d0 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 10 client.6369 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x563e2c707220 con 0x563e2cad1880 2024-05-17T22:02:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec019070 con 0x563e2cad1880 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f50e4120bd0 con 0x563e2cad1880 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 10 client.6369 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 10 client.6369 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 20 client.6369 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c70cfe0 0x563e2cc02100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 4 client.6369 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.555+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e2ca5f070 con 0x563e2c70cfe0 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c70cfe0 0x563e2cc02100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50d2ffd640 10 client.6369 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50eb7fe640 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c70cfe0 0x563e2cc02100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e385fe8d56c01 server_cookie=9b81e9586df82823 in_seq=0 out_seq=0 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50d2ffd640 1 -- 172.21.15.83:0/3232945607 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563e2ca5f070 con 0x563e2c70cfe0 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50d2ffd640 10 client.6369 handle_command_reply: tid=0 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 client.6369 shutdown 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 msgr2=0x563e2ccbe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f50e40f24c0 tx=0x7f50e40f5950 comp rx=0 tx=0).stop 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 shutdown_connections 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x563e2cb20010 0x563e2cc01bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e2cad2090 0x563e2ccbe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x563e2c70cfe0 0x563e2cc02100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 --2- 172.21.15.83:0/3232945607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e2cad1880 0x563e2ccbe1f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50d2ffd640 0 client.6369 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 >> 172.21.15.83:0/3232945607 conn(0x563e2cc04400 msgr2=0x563e2cc4a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 shutdown_connections 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 1 -- 172.21.15.83:0/3232945607 wait complete. 2024-05-17T22:02:04.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f51017d4000 20 client.6369 trim_cache size 0 max 16384 2024-05-17T22:02:04.556 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:02:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:02:04.556 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 msgr2=0x7f50d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 0x7f50d4071910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f50f415ecd0 tx=0x7f50dc008040 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 msgr2=0x7f50f40aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50e40ad040 tx=0x7f50e401ada0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 shutdown_connections 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f50f41606f0 0x7f50f40cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f50d406f400 0x7f50d4071910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50f415be70 0x7f50f41fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 --2- 172.21.15.83:0/1425012664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f406eec0 0x7f50f40aa310 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 >> 172.21.15.83:0/1425012664 conn(0x7f50f406cd90 msgr2=0x7f50f40b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 shutdown_connections 2024-05-17T22:02:04.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.559+0000 7f50fa55a640 1 -- 172.21.15.83:0/1425012664 wait complete. 2024-05-17T22:02:04.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:02:04.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 Processor -- start 2024-05-17T22:02:04.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 -- start start 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb406cea0 0x7f5cb40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb409bcc0 0x7f5cb4167380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb409c0d0 con 0x7f5cb40b4910 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb40afc40 con 0x7f5cb406cea0 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cbb7a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb415e110 con 0x7f5cb409bcc0 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb406cea0 0x7f5cb40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb406cea0 0x7f5cb40b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58678/0 (socket says 172.21.15.83:58678) 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 -- 172.21.15.83:0/1893702401 learned_addr learned my addr 172.21.15.83:0/1893702401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb9d41640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb409bcc0 0x7f5cb4167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 -- 172.21.15.83:0/1893702401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb409bcc0 msgr2=0x7f5cb4167380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb409bcc0 0x7f5cb4167380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 -- 172.21.15.83:0/1893702401 >> v1:172.21.15.155:6789/0 conn(0x7f5cb40b4910 legacy=0x7f5cb406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:04.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.723+0000 7f5cb8d3f640 1 -- 172.21.15.83:0/1893702401 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb40b3a20 con 0x7f5cb406cea0 2024-05-17T22:02:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cb8d3f640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb406cea0 0x7f5cb40b3150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40afdc0 tx=0x7f5cb001c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4f3c6f86d6e025 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cabfff640 1 -- 172.21.15.83:0/1893702401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00b0070 con 0x7f5cb406cea0 2024-05-17T22:02:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cabfff640 1 -- 172.21.15.83:0/1893702401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb00ab600 con 0x7f5cb406cea0 2024-05-17T22:02:04.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cabfff640 1 -- 172.21.15.83:0/1893702401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00ab9d0 con 0x7f5cb406cea0 2024-05-17T22:02:04.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1893702401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb406cea0 msgr2=0x7f5cb40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb406cea0 0x7f5cb40b3150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40afdc0 tx=0x7f5cb001c2d0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1893702401 shutdown_connections 2024-05-17T22:02:04.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb409bcc0 0x7f5cb4167380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1893702401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb406cea0 0x7f5cb40b3150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1893702401 >> 172.21.15.83:0/1893702401 conn(0x7f5cb409b8b0 msgr2=0x7f5cb40b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1893702401 shutdown_connections 2024-05-17T22:02:04.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.727+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1893702401 wait complete. 2024-05-17T22:02:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 Processor -- start 2024-05-17T22:02:04.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- start start 2024-05-17T22:02:04.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cb40b4910 0x7f5cb414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 0x7f5cb41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb416a260 con 0x7f5cb409bcc0 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb416a0e0 con 0x7f5cb40b4910 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb416a3e0 con 0x7f5cb41545e0 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58690/0 (socket says 172.21.15.83:58690) 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 -- 172.21.15.83:0/1250473395 learned_addr learned my addr 172.21.15.83:0/1250473395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9d41640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 0x7f5cb41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb8d3f640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cb40b4910 0x7f5cb414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 -- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 msgr2=0x7f5cb41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 0x7f5cb41549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 -- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cb40b4910 msgr2=0x7f5cb414c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cb40b4910 0x7f5cb414c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 -- 172.21.15.83:0/1250473395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb4154f30 con 0x7f5cb409bcc0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9d41640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 0x7f5cb41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cb9540640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b470 tx=0x7f5ca400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4015030 con 0x7f5cb409bcc0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca4004020 con 0x7f5cb409bcc0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4004410 con 0x7f5cb409bcc0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb415d8e0 con 0x7f5cb409bcc0 2024-05-17T22:02:04.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb413c0e0 con 0x7f5cb409bcc0 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.731+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb406c790 con 0x7f5cb409bcc0 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.735+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ca4018070 con 0x7f5cb409bcc0 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.735+0000 7f5ca9ffb640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 0x7f5c98071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.735+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f5ca408ed60 con 0x7f5cb409bcc0 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.735+0000 7f5cb8d3f640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 0x7f5c98071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:04.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.735+0000 7f5cb8d3f640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 0x7f5c98071940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00ad000 tx=0x7f5cb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:04.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.739+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5cb4154f30 con 0x7f5cb409bcc0 2024-05-17T22:02:04.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb41564d0 con 0x7f5cb409bcc0 2024-05-17T22:02:04.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5ca9ffb640 1 -- 172.21.15.83:0/1250473395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5187 (secure 0 0 0) 0x7f5ca405bce0 con 0x7f5cb409bcc0 2024-05-17T22:02:04.923 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:04.923 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:01:29.304867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:04.923 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 50 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 msgr2=0x7f5c98071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 0x7f5c98071940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00ad000 tx=0x7f5cb0008040 comp rx=0 tx=0).stop 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 msgr2=0x7f5cb41f9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b470 tx=0x7f5ca400b930 comp rx=0 tx=0).stop 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 shutdown_connections 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cb41545e0 0x7f5cb41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f430 0x7f5c98071940 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cb40b4910 0x7f5cb414c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 --2- 172.21.15.83:0/1250473395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb409bcc0 0x7f5cb41f9a60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:04.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 >> 172.21.15.83:0/1250473395 conn(0x7f5cb409b8b0 msgr2=0x7f5cb40b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:04.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 shutdown_connections 2024-05-17T22:02:04.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:04.931+0000 7f5cbb7a4640 1 -- 172.21.15.83:0/1250473395 wait complete. 2024-05-17T22:02:04.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-17T22:02:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.103+0000 7fddadeee640 1 Processor -- start 2024-05-17T22:02:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.103+0000 7fddadeee640 1 -- start start 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fddadeee640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdda80b3d00 0x7fdda80d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fddadeee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda80d10d0 con 0x7fdda8143f20 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fddadeee640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda8070120 con 0x7fdda8073590 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fddadeee640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdda80b4110 con 0x7fdda80b3d00 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdda8143f20 0x7fdda8072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57614/0 (socket says 172.21.15.83:57614) 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda6ffd640 1 -- 172.21.15.83:0/1569420308 learned_addr learned my addr 172.21.15.83:0/1569420308 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:05.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1569420308 >> v2:172.21.15.155:3300/0 conn(0x7fdda80b3d00 0x7fdda80d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741158030 0 0) 0x7fdda80d10d0 con 0x7fdda8143f20 2024-05-17T22:02:05.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd90003530 con 0x7fdda8143f20 2024-05-17T22:02:05.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2189383632 0 0) 0x7fdd90003530 con 0x7fdda8143f20 2024-05-17T22:02:05.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 >> v2:172.21.15.155:3300/0 conn(0x7fdda80b3d00 msgr2=0x7fdda80d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 --2- 172.21.15.83:0/1569420308 >> v2:172.21.15.155:3300/0 conn(0x7fdda80b3d00 0x7fdda80d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 >> v1:172.21.15.155:6789/0 conn(0x7fdda8073590 legacy=0x7fdda81447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda8070400 con 0x7fdda8143f20 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fdd9c001700 con 0x7fdda8143f20 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 >> v1:172.21.15.83:6789/0 conn(0x7fdda8143f20 legacy=0x7fdda8072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 0x7fdd90003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd90004810 0x7fdd90004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90005260 0x7fdd9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda80b4110 con 0x7fdd90003a60 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9000c1f0 con 0x7fdd90004810 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9000c3f0 con 0x7fdd90005260 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 0x7fdd90003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd90004810 0x7fdd90004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda7fff640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90005260 0x7fdd9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 -- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90005260 msgr2=0x7fdd9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90005260 0x7fdd9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 -- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd90004810 msgr2=0x7fdd90004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd90004810 0x7fdd90004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 -- 172.21.15.83:0/1569420308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd90003530 con 0x7fdd90003a60 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda7fff640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90005260 0x7fdd9000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda7fff640 1 -- 172.21.15.83:0/1569420308 reap_dead start 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.107+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 0x7fdd90003ed0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009ae0 tx=0x7fdd9801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c8e66eb1fee466 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980b2070 con 0x7fdd90003a60 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8145800 con 0x7fdd90003a60 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fdda67fc640 1 -- 172.21.15.83:0/1569420308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980096b0 con 0x7fdd90003a60 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 msgr2=0x7fdd90003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 0x7fdd90003ed0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009ae0 tx=0x7fdd9801ca70 comp rx=0 tx=0).stop 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- 172.21.15.83:0/1569420308 shutdown_connections 2024-05-17T22:02:05.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 --2- 172.21.15.83:0/1569420308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd90003a60 0x7fdd90003ed0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- 172.21.15.83:0/1569420308 >> 172.21.15.83:0/1569420308 conn(0x7fdda806cd90 msgr2=0x7fdda80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- 172.21.15.83:0/1569420308 shutdown_connections 2024-05-17T22:02:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- 172.21.15.83:0/1569420308 wait complete. 2024-05-17T22:02:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 Processor -- start 2024-05-17T22:02:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.111+0000 7fddadeee640 1 -- start start 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90003a60 0x7fdda815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdda8161e00 0x7fdda81eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8073410 con 0x7fdda813ea90 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8073290 con 0x7fdda8161e00 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdda8070c00 con 0x7fdd90003a60 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58716/0 (socket says 172.21.15.83:58716) 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 -- 172.21.15.83:0/4144031542 learned_addr learned my addr 172.21.15.83:0/4144031542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda7fff640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdda8161e00 0x7fdda81eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda77fe640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90003a60 0x7fdda815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 -- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90003a60 msgr2=0x7fdda815fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90003a60 0x7fdda815fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 -- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdda8161e00 msgr2=0x7fdda81eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdda8161e00 0x7fdda81eb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 -- 172.21.15.83:0/4144031542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda813da30 con 0x7fdda813ea90 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00e7c0 tx=0x7fdd9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdd8ffff640 1 -- 172.21.15.83:0/4144031542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00c8d0 con 0x7fdda813ea90 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdd8ffff640 1 -- 172.21.15.83:0/4144031542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c018070 con 0x7fdda813ea90 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fdd8ffff640 1 -- 172.21.15.83:0/4144031542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00ccd0 con 0x7fdda813ea90 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda81602a0 con 0x7fdda813ea90 2024-05-17T22:02:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.115+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda813dfb0 con 0x7fdda813ea90 2024-05-17T22:02:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8ffff640 1 -- 172.21.15.83:0/4144031542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdd9c01d460 con 0x7fdda813ea90 2024-05-17T22:02:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8ffff640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 0x7fdd94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8ffff640 1 -- 172.21.15.83:0/4144031542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fdd9c08f190 con 0x7fdda813ea90 2024-05-17T22:02:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda77fe640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 0x7fdd94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 Processor -- start 2024-05-17T22:02:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda77fe640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 0x7fdd94071940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd980af000 tx=0x7fdd980046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 -- start start 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c092367020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092201030 0x55c092367aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c092205fa0 con 0x55c092217460 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c09217a2e0 con 0x55c092221e00 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c092169e80 con 0x55c092201030 2024-05-17T22:02:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58724/0 (socket says 172.21.15.83:58724) 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 -- 172.21.15.83:0/367482250 learned_addr learned my addr 172.21.15.83:0/367482250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 -- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092201030 msgr2=0x55c092367aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda7fff640 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092201030 0x55c092367aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda77fe640 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c092367020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092201030 0x55c092367aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 -- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 msgr2=0x55c092367020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c092367020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 -- 172.21.15.83:0/367482250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c092214b80 con 0x55c092217460 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c08d5d0 tx=0x7fdd9c058100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea38217470c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8d7fa640 1 -- 172.21.15.83:0/367482250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c05b490 con 0x55c092217460 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8d7fa640 1 -- 172.21.15.83:0/367482250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c05c080 con 0x55c092217460 2024-05-17T22:02:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.119+0000 7fdd8d7fa640 1 -- 172.21.15.83:0/367482250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c05b980 con 0x55c092217460 2024-05-17T22:02:05.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/367482250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 msgr2=0x55c092367560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c08d5d0 tx=0x7fdd9c058100 comp rx=0 tx=0).stop 2024-05-17T22:02:05.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/367482250 shutdown_connections 2024-05-17T22:02:05.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092201030 0x55c092367aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092217460 0x55c092367560 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- 172.21.15.83:0/367482250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c092367020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/367482250 >> 172.21.15.83:0/367482250 conn(0x55c091e2b760 msgr2=0x55c092324270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/367482250 shutdown_connections 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/367482250 wait complete. 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- start start 2024-05-17T22:02:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 Processor -- start 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c09236e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c09236ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c092125eb0 con 0x55c092201030 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c09216dec0 con 0x55c092221e00 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c091e33820 con 0x55c092217460 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 10 client.0 _ll_register_callbacks cb 0x55c092309d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58730/0 (socket says 172.21.15.83:58730) 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 -- 172.21.15.83:0/2339332453 learned_addr learned my addr 172.21.15.83:0/2339332453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c09236e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda7fff640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c09236ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 -- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 msgr2=0x55c09236e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c09236e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 -- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 msgr2=0x55c09236ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c09236ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 -- 172.21.15.83:0/2339332453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0921b2ef0 con 0x55c092201030 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c09236e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda7fff640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c09236ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdda77fe640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdd9801cf50 tx=0x7fdd98019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c04c96e6c7bf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980c7060 con 0x55c092201030 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fdd98003af0 con 0x55c092201030 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9c05c080 con 0x55c092201030 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdd980cd050 con 0x55c092201030 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 10 client.6387 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55c0921a7910 con 0x55c092201030 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd9c05c080 con 0x55c092201030 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fdd980d0020 con 0x55c092201030 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 10 client.6387 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 10 client.6387 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 20 client.6387 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.123+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091e10ad0 0x55c092378680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 4 client.6387 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c092105cf0 con 0x55c091e10ad0 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091e10ad0 0x55c092378680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdd867fc640 10 client.6387 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091e10ad0 0x55c092378680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb37b8878195292 server_cookie=32ac52dcd438c8d in_seq=0 out_seq=0 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdd867fc640 1 -- 172.21.15.83:0/2339332453 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c092105cf0 con 0x55c091e10ad0 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdd867fc640 10 client.6387 handle_command_reply: tid=0 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 client.6387 shutdown 2024-05-17T22:02:05.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 msgr2=0x55c09236e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdd9801cf50 tx=0x7fdd98019810 comp rx=0 tx=0).stop 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 shutdown_connections 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c09236ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c09236e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091e10ad0 0x55c092378680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 --2- 172.21.15.83:0/2339332453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c09236e2a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fdd867fc640 0 client.6387 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 >> 172.21.15.83:0/2339332453 conn(0x55c091e2b760 msgr2=0x55c09231c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 shutdown_connections 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 1 -- 172.21.15.83:0/2339332453 wait complete. 2024-05-17T22:02:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.127+0000 7fddb5168000 20 client.6387 trim_cache size 0 max 16384 2024-05-17T22:02:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 Processor -- start 2024-05-17T22:02:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 -- start start 2024-05-17T22:02:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c0923d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.139+0000 7fddb5168000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c092321fd0 con 0x55c092201030 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8204290 con 0x55c092221e00 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdda82032d0 con 0x55c092217460 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58738/0 (socket says 172.21.15.83:58738) 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 -- 172.21.15.83:0/1296106156 learned_addr learned my addr 172.21.15.83:0/1296106156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 -- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 msgr2=0x55c0923d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 -- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 msgr2=0x55c0923d5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c0923d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 -- 172.21.15.83:0/1296106156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c092127670 con 0x55c092201030 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdd9801cab0 tx=0x7fdd980bb070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98d06920d240bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdd8cff9640 1 -- 172.21.15.83:0/1296106156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980b2040 con 0x55c092201030 2024-05-17T22:02:05.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdd8cff9640 1 -- 172.21.15.83:0/1296106156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd980039f0 con 0x55c092201030 2024-05-17T22:02:05.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fdd8cff9640 1 -- 172.21.15.83:0/1296106156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003ba0 con 0x55c092201030 2024-05-17T22:02:05.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- 172.21.15.83:0/1296106156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 msgr2=0x55c0923d53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdd9801cab0 tx=0x7fdd980bb070 comp rx=0 tx=0).stop 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- 172.21.15.83:0/1296106156 shutdown_connections 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092221e00 0x55c0923d5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- 172.21.15.83:0/1296106156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092201030 0x55c0923d53a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- 172.21.15.83:0/1296106156 >> 172.21.15.83:0/1296106156 conn(0x55c091e2b760 msgr2=0x55c09231cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- 172.21.15.83:0/1296106156 shutdown_connections 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- 172.21.15.83:0/1296106156 wait complete. 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- start start 2024-05-17T22:02:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 Processor -- start 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092201030 0x55c0923d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8202c80 con 0x55c092221e00 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdda813c7a0 con 0x55c092201030 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c0921111a0 con 0x55c092217460 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.143+0000 7fddb5168000 10 client.0 _ll_register_callbacks cb 0x7fdd980b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58754/0 (socket says 172.21.15.83:58754) 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 -- 172.21.15.83:0/1724210548 learned_addr learned my addr 172.21.15.83:0/1724210548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda6ffd640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092201030 0x55c0923d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 -- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 msgr2=0x55c0923d9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 -- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092201030 msgr2=0x55c0923d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092201030 0x55c0923d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 -- 172.21.15.83:0/1724210548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c092126100 con 0x55c092221e00 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda7fff640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdda81472f0 tx=0x7fdda0011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe70294e7192f351 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0018020 con 0x55c092221e00 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fdda0003d90 con 0x55c092221e00 2024-05-17T22:02:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd980039f0 con 0x55c092221e00 2024-05-17T22:02:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdda0022050 con 0x55c092221e00 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 10 client.6393 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55c0921eed80 con 0x55c092221e00 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd980039f0 con 0x55c092221e00 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fdda0025020 con 0x55c092221e00 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 10 client.6393 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 10 client.6393 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 20 client.6393 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091dfeb30 0x55c092395150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 4 client.6393 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55c09217c230 con 0x55c091dfeb30 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091dfeb30 0x55c092395150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.147+0000 7fdd867fc640 10 client.6393 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fdda77fe640 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091dfeb30 0x55c092395150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634afc5412cd9a60 server_cookie=a202ab48326e0e2f in_seq=0 out_seq=0 2024-05-17T22:02:05.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fdd867fc640 1 -- 172.21.15.83:0/1724210548 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55c09217c230 con 0x55c091dfeb30 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fdd867fc640 10 client.6393 handle_command_reply: tid=0 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 client.6393 shutdown 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 msgr2=0x55c0923da160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdda81472f0 tx=0x7fdda0011970 comp rx=0 tx=0).stop 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 shutdown_connections 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55c091dfeb30 0x55c092395150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c092221e00 0x55c0923da160 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c092217460 0x55c0923d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fdd867fc640 0 client.6393 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 --2- 172.21.15.83:0/1724210548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c092201030 0x55c0923d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 >> 172.21.15.83:0/1724210548 conn(0x55c091e2b760 msgr2=0x55c09231d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 shutdown_connections 2024-05-17T22:02:05.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 1 -- 172.21.15.83:0/1724210548 wait complete. 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddb5168000 20 client.6393 trim_cache size 0 max 16384 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "dentry", 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 1423738998, 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627776, 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "frag": "*", 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "dname": "file_to_be_damaged", 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_id": "head", 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-17T22:02:05.145 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:05.146 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:02:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 msgr2=0x7fdd94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 0x7fdd94071940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd980af000 tx=0x7fdd980046d0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 msgr2=0x7fdda81618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00e7c0 tx=0x7fdd9c00ec80 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 shutdown_connections 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd9406f430 0x7fdd94071940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdda8161e00 0x7fdda81eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda813ea90 0x7fdda81618c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 --2- 172.21.15.83:0/4144031542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd90003a60 0x7fdda815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 >> 172.21.15.83:0/4144031542 conn(0x7fdda806cd90 msgr2=0x7fdda8146a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 shutdown_connections 2024-05-17T22:02:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.151+0000 7fddadeee640 1 -- 172.21.15.83:0/4144031542 wait complete. 2024-05-17T22:02:05.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:02:05.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 Processor -- start 2024-05-17T22:02:05.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 -- start start 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01ec15ffa0 0x7f01ec0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01ec0d1150 0x7f01ec06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec06e610 con 0x7f01ec06e200 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f01ec0d34e0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f2956640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f01ec0d3780 con 0x7f01ec0d1150 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01ebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01ec15ffa0 0x7f01ec0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01ebfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f01ec15ffa0 0x7f01ec0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58764/0 (socket says 172.21.15.83:58764) 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01f0ef3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f01ec06e200 0x7f01ec163a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57618/0 (socket says 172.21.15.83:57618) 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.299+0000 7f01ebfff640 1 -- 172.21.15.83:0/4274948253 learned_addr learned my addr 172.21.15.83:0/4274948253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01e3fff640 1 --2- 172.21.15.83:0/4274948253 >> v2:172.21.15.155:3300/0 conn(0x7f01ec0d1150 0x7f01ec06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01ebfff640 1 -- 172.21.15.83:0/4274948253 >> v2:172.21.15.155:3300/0 conn(0x7f01ec0d1150 msgr2=0x7f01ec06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01ebfff640 1 --2- 172.21.15.83:0/4274948253 >> v2:172.21.15.155:3300/0 conn(0x7f01ec0d1150 0x7f01ec06dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01ebfff640 1 -- 172.21.15.83:0/4274948253 >> v1:172.21.15.83:6789/0 conn(0x7f01ec06e200 legacy=0x7f01ec163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01ebfff640 1 -- 172.21.15.83:0/4274948253 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec0d3a60 con 0x7f01ec15ffa0 2024-05-17T22:02:05.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01eb7fe640 1 -- 172.21.15.83:0/4274948253 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303096046 0 0) 0x7f01e40014a0 con 0x7f01ec06e200 2024-05-17T22:02:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01ebfff640 1 --2- 172.21.15.83:0/4274948253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0d0c10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0b5260 tx=0x7f01dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15fc422a730a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01eb7fe640 1 -- 172.21.15.83:0/4274948253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0b0070 con 0x7f01ec15ffa0 2024-05-17T22:02:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01eb7fe640 1 -- 172.21.15.83:0/4274948253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01dc0ab600 con 0x7f01ec15ffa0 2024-05-17T22:02:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01eb7fe640 1 -- 172.21.15.83:0/4274948253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0ab9d0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 -- 172.21.15.83:0/4274948253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 msgr2=0x7f01ec0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 --2- 172.21.15.83:0/4274948253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0d0c10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0b5260 tx=0x7f01dc01c520 comp rx=0 tx=0).stop 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 -- 172.21.15.83:0/4274948253 shutdown_connections 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 --2- 172.21.15.83:0/4274948253 >> v2:172.21.15.155:3300/0 conn(0x7f01ec0d1150 0x7f01ec06dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 --2- 172.21.15.83:0/4274948253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0d0c10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 -- 172.21.15.83:0/4274948253 >> 172.21.15.83:0/4274948253 conn(0x7f01ec0d28f0 msgr2=0x7f01ec0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 -- 172.21.15.83:0/4274948253 shutdown_connections 2024-05-17T22:02:05.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.303+0000 7f01f2956640 1 -- 172.21.15.83:0/4274948253 wait complete. 2024-05-17T22:02:05.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 Processor -- start 2024-05-17T22:02:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- start start 2024-05-17T22:02:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01ec06e200 0x7f01ec147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01ec0d1150 0x7f01ec0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec16cd00 con 0x7f01ec15ffa0 2024-05-17T22:02:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec16cb80 con 0x7f01ec0d1150 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec16ce80 con 0x7f01ec06e200 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58774/0 (socket says 172.21.15.83:58774) 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 -- 172.21.15.83:0/3883388004 learned_addr learned my addr 172.21.15.83:0/3883388004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 -- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01ec06e200 msgr2=0x7f01ec147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01ec06e200 0x7f01ec147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 -- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01ec0d1150 msgr2=0x7f01ec0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01ec0d1150 0x7f01ec0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 -- 172.21.15.83:0/3883388004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec0d0040 con 0x7f01ec15ffa0 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f0ef3640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c760 tx=0x7f01e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400ce10 con 0x7f01ec15ffa0 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e400a640 con 0x7f01ec15ffa0 2024-05-17T22:02:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400aa30 con 0x7f01ec15ffa0 2024-05-17T22:02:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec0cefa0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.307+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec0cf560 con 0x7f01ec15ffa0 2024-05-17T22:02:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec06dcc0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01e400abd0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01e97fa640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 0x7f01cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f01e408fbd0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01ebfff640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 0x7f01cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.311+0000 7f01ebfff640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 0x7f01cc071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0ad000 tx=0x7f01dc004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.315+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f01ec06e610 con 0x7f01ec15ffa0 2024-05-17T22:02:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f01ec06df60 con 0x7f01ec15ffa0 2024-05-17T22:02:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01e97fa640 1 -- 172.21.15.83:0/3883388004 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f01e405cba0 con 0x7f01ec15ffa0 2024-05-17T22:02:05.499 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-17T22:02:05.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 msgr2=0x7f01cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 0x7f01cc071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0ad000 tx=0x7f01dc004740 comp rx=0 tx=0).stop 2024-05-17T22:02:05.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 msgr2=0x7f01ec0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c760 tx=0x7f01e400cc20 comp rx=0 tx=0).stop 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 shutdown_connections 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01ec15ffa0 0x7f01ec0cce60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f01cc06f400 0x7f01cc071910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01ec0d1150 0x7f01ec0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 --2- 172.21.15.83:0/3883388004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01ec06e200 0x7f01ec147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 >> 172.21.15.83:0/3883388004 conn(0x7f01ec0d28f0 msgr2=0x7f01ec16a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 shutdown_connections 2024-05-17T22:02:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.507+0000 7f01f2956640 1 -- 172.21.15.83:0/3883388004 wait complete. 2024-05-17T22:02:05.519 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-17T22:02:05.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:05.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-17T22:02:05.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.667+0000 7fc9514f6640 1 Processor -- start 2024-05-17T22:02:05.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.667+0000 7fc9514f6640 1 -- start start 2024-05-17T22:02:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc9514f6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc94c145fd0 0x7fc94c0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc9514f6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c0d1210 con 0x7fc94c0d03b0 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc9514f6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc94c0d0970 con 0x7fc94c160b00 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc9514f6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc94c0ce360 con 0x7fc94c145fd0 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc94c145fd0 0x7fc94c0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc94c0d03b0 0x7fc94c1603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57630/0 (socket says 172.21.15.83:57630) 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/2237995262 >> v2:172.21.15.83:3300/0 conn(0x7fc94c145fd0 0x7fc94c0cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58784/0 (socket says 172.21.15.83:58784) 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94affd640 1 -- 172.21.15.83:0/2237995262 learned_addr learned my addr 172.21.15.83:0/2237995262 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 -- 172.21.15.83:0/2237995262 >> v1:172.21.15.155:6789/0 conn(0x7fc94c160b00 legacy=0x7fc94c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 -- 172.21.15.83:0/2237995262 >> v1:172.21.15.83:6789/0 conn(0x7fc94c0d03b0 legacy=0x7fc94c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 -- 172.21.15.83:0/2237995262 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c0d0c50 con 0x7fc94c145fd0 2024-05-17T22:02:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/2237995262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c0cde20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc94400bfa0 tx=0x7fc94401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3fdda896e5aab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94a7fc640 1 -- 172.21.15.83:0/2237995262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94400b2d0 con 0x7fc94c145fd0 2024-05-17T22:02:05.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94a7fc640 1 -- 172.21.15.83:0/2237995262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9440a7930 con 0x7fc94c145fd0 2024-05-17T22:02:05.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.671+0000 7fc94a7fc640 1 -- 172.21.15.83:0/2237995262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440a7d00 con 0x7fc94c145fd0 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- 172.21.15.83:0/2237995262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 msgr2=0x7fc94c0cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 --2- 172.21.15.83:0/2237995262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c0cde20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc94400bfa0 tx=0x7fc94401eb70 comp rx=0 tx=0).stop 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- 172.21.15.83:0/2237995262 shutdown_connections 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 --2- 172.21.15.83:0/2237995262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c0cde20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- 172.21.15.83:0/2237995262 >> 172.21.15.83:0/2237995262 conn(0x7fc94c06cd90 msgr2=0x7fc94c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- 172.21.15.83:0/2237995262 shutdown_connections 2024-05-17T22:02:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- 172.21.15.83:0/2237995262 wait complete. 2024-05-17T22:02:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 Processor -- start 2024-05-17T22:02:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- start start 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 0x7fc94c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc94c13e7b0 0x7fc94c13ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c070540 con 0x7fc94c145fd0 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c0703c0 con 0x7fc94c0d03b0 2024-05-17T22:02:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9514f6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c0706c0 con 0x7fc94c13e7b0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58798/0 (socket says 172.21.15.83:58798) 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 -- 172.21.15.83:0/1950689610 learned_addr learned my addr 172.21.15.83:0/1950689610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 -- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc94c13e7b0 msgr2=0x7fc94c13ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc94c13e7b0 0x7fc94c13ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc94c13e7b0 0x7fc94c13ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 -- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 msgr2=0x7fc94c0b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc94affd640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 0x7fc94c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 0x7fc94c0b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 -- 172.21.15.83:0/1950689610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c13f100 con 0x7fc94c145fd0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc94affd640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 0x7fc94c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.675+0000 7fc9427fc640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc93400b3e0 tx=0x7fc93400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 -- 172.21.15.83:0/1950689610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934015020 con 0x7fc94c145fd0 2024-05-17T22:02:05.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 -- 172.21.15.83:0/1950689610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc934011850 con 0x7fc94c145fd0 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 -- 172.21.15.83:0/1950689610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934011c40 con 0x7fc94c145fd0 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c135210 con 0x7fc94c145fd0 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1359f0 con 0x7fc94c145fd0 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 -- 172.21.15.83:0/1950689610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc934018020 con 0x7fc94c145fd0 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 0x7fc92c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc943fff640 1 -- 172.21.15.83:0/1950689610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fc93408ed10 con 0x7fc94c145fd0 2024-05-17T22:02:05.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc94affd640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 0x7fc92c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc94affd640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 0x7fc92c071940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000f70 tx=0x7fc93c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc958770000 1 Processor -- start 2024-05-17T22:02:05.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.679+0000 7fc958770000 1 -- start start 2024-05-17T22:02:05.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f700a00c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f7008110a0 con 0x55f7008931f0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f700800c40 con 0x55f7008b0090 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f700804c80 con 0x55f7008a18d0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58814/0 (socket says 172.21.15.83:58814) 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 -- 172.21.15.83:0/483834967 learned_addr learned my addr 172.21.15.83:0/483834967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc9427fc640 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f700a00c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94affd640 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 -- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 msgr2=0x55f700a00c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f700a00c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 -- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 msgr2=0x55f700a006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 -- 172.21.15.83:0/483834967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7007ba5f0 con 0x55f7008931f0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9440b1040 tx=0x7fc944004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4443902c4e86e17 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc940ff9640 1 -- 172.21.15.83:0/483834967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94400b2d0 con 0x55f7008931f0 2024-05-17T22:02:05.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc940ff9640 1 -- 172.21.15.83:0/483834967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9440a7930 con 0x55f7008931f0 2024-05-17T22:02:05.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc940ff9640 1 -- 172.21.15.83:0/483834967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440a7ce0 con 0x55f7008931f0 2024-05-17T22:02:05.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- 172.21.15.83:0/483834967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 msgr2=0x55f700a01160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9440b1040 tx=0x7fc944004ac0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- 172.21.15.83:0/483834967 shutdown_connections 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a01160 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f700a00c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 --2- 172.21.15.83:0/483834967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- 172.21.15.83:0/483834967 >> 172.21.15.83:0/483834967 conn(0x55f7004c3020 msgr2=0x55f7009b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- 172.21.15.83:0/483834967 shutdown_connections 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- 172.21.15.83:0/483834967 wait complete. 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 -- start start 2024-05-17T22:02:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.683+0000 7fc958770000 1 Processor -- start 2024-05-17T22:02:05.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f7009bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f7009ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f7007bce90 con 0x55f7008931f0 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f7008092b0 con 0x55f7008b0090 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f7007a8000 con 0x55f7008a18d0 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 10 client.0 _ll_register_callbacks cb 0x55f7009a2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58820/0 (socket says 172.21.15.83:58820) 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 -- 172.21.15.83:0/2153125944 learned_addr learned my addr 172.21.15.83:0/2153125944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 -- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 msgr2=0x55f7009bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f7009bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 -- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 msgr2=0x55f7009ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f7009ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 -- 172.21.15.83:0/2153125944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7007bf7e0 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc94affd640 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc93c007650 tx=0x7fc93c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93cf2109b802ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c021630 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9440a7930 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fc93c020b10 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc93c005e50 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 10 client.6417 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55f7008d11a0 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9440a7930 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc93c020420 con 0x55f7008931f0 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 10 client.6417 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 10 client.6417 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:05.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 20 client.6417 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f7004a7bd0 0x55f700a07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 4 client.6417 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f70079cb50 con 0x55f7004a7bd0 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc9427fc640 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f7004a7bd0 0x55f700a07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 10 client.6417 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc9427fc640 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f7004a7bd0 0x55f700a07960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6284b9ee65b5310 server_cookie=8d18be7872a1149b in_seq=0 out_seq=0 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 1 -- 172.21.15.83:0/2153125944 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f70079cb50 con 0x55f7004a7bd0 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 10 client.6417 handle_command_reply: tid=0 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 client.6417 shutdown 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 msgr2=0x55f7009bc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc93c007650 tx=0x7fc93c000bd0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 shutdown_connections 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f7009ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008a18d0 0x55f7009bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f7004a7bd0 0x55f700a07960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 --2- 172.21.15.83:0/2153125944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f7009bc9e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc929ffb640 0 client.6417 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 >> 172.21.15.83:0/2153125944 conn(0x55f7004c3020 msgr2=0x55f7009b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 shutdown_connections 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 1 -- 172.21.15.83:0/2153125944 wait complete. 2024-05-17T22:02:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.687+0000 7fc958770000 20 client.6417 trim_cache size 0 max 16384 2024-05-17T22:02:05.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 Processor -- start 2024-05-17T22:02:05.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- start start 2024-05-17T22:02:05.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008a18d0 0x55f700a184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008b0090 0x55f7009b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c200550 con 0x55f7008931f0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c13b4d0 con 0x55f7008a18d0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc94c0b1990 con 0x55f7008b0090 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58832/0 (socket says 172.21.15.83:58832) 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 -- 172.21.15.83:0/282167392 learned_addr learned my addr 172.21.15.83:0/282167392 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 -- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008b0090 msgr2=0x55f7009b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94b7fe640 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008b0090 0x55f7009b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc9427fc640 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008a18d0 0x55f700a184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008b0090 0x55f7009b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 -- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008a18d0 msgr2=0x55f700a184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008a18d0 0x55f700a184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 -- 172.21.15.83:0/282167392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7008889c0 con 0x55f7008931f0 2024-05-17T22:02:05.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc94affd640 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0214b0 tx=0x7fc93c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eff43b51571e19a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc92bfff640 1 -- 172.21.15.83:0/282167392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c02f040 con 0x55f7008931f0 2024-05-17T22:02:05.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc92bfff640 1 -- 172.21.15.83:0/282167392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c005e20 con 0x55f7008931f0 2024-05-17T22:02:05.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc92bfff640 1 -- 172.21.15.83:0/282167392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c020bc0 con 0x55f7008931f0 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- 172.21.15.83:0/282167392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 msgr2=0x55f700a17fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0214b0 tx=0x7fc93c01d000 comp rx=0 tx=0).stop 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- 172.21.15.83:0/282167392 shutdown_connections 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008b0090 0x55f7009b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008a18d0 0x55f700a184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 --2- 172.21.15.83:0/282167392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008931f0 0x55f700a17fb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- 172.21.15.83:0/282167392 >> 172.21.15.83:0/282167392 conn(0x55f7004c3020 msgr2=0x55f700a178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- 172.21.15.83:0/282167392 shutdown_connections 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.707+0000 7fc958770000 1 -- 172.21.15.83:0/282167392 wait complete. 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- start start 2024-05-17T22:02:05.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 Processor -- start 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008931f0 0x55f700a73c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a74690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f70080fe70 con 0x55f7008a18d0 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f700787860 con 0x55f7008b0090 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f7007cfd60 con 0x55f7008931f0 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 10 client.0 _ll_register_callbacks cb 0x7fc93c0013c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58846/0 (socket says 172.21.15.83:58846) 2024-05-17T22:02:05.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 -- 172.21.15.83:0/691968749 learned_addr learned my addr 172.21.15.83:0/691968749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 -- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008931f0 msgr2=0x55f700a73c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008931f0 0x55f700a73c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 -- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 msgr2=0x55f700a74690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a74690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 -- 172.21.15.83:0/691968749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7007c0640 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc9427fc640 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc934057a50 tx=0x7fc934057320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36af215f6cdf12d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93405a6c0 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fc93405a860 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93c005e20 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc934074bc0 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 10 client.6423 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55f7007ca640 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c005e20 con 0x55f7008a18d0 2024-05-17T22:02:05.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fc934062b00 con 0x55f7008a18d0 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 10 client.6423 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 10 client.6423 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 20 client.6423 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f700495c30 0x55f700a2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 4 client.6423 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55f7007a1070 con 0x55f700495c30 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc94affd640 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f700495c30 0x55f700a2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc929ffb640 10 client.6423 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.711+0000 7fc94affd640 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f700495c30 0x55f700a2fd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9434e439e81752a server_cookie=47a351e9c8217a13 in_seq=0 out_seq=0 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc929ffb640 1 -- 172.21.15.83:0/691968749 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f7007a1070 con 0x55f700495c30 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc929ffb640 10 client.6423 handle_command_reply: tid=0 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 client.6423 shutdown 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 msgr2=0x55f700a74150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc934057a50 tx=0x7fc934057320 comp rx=0 tx=0).stop 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 shutdown_connections 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f7008b0090 0x55f700a74690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f700495c30 0x55f700a2fd70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f7008a18d0 0x55f700a74150 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 --2- 172.21.15.83:0/691968749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f7008931f0 0x55f700a73c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc929ffb640 0 client.6423 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 >> 172.21.15.83:0/691968749 conn(0x55f7004c3020 msgr2=0x55f700a18030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 shutdown_connections 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 1 -- 172.21.15.83:0/691968749 wait complete. 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc958770000 20 client.6423 trim_cache size 0 max 16384 2024-05-17T22:02:05.710 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:05.711 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:02:05.711 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "37d1cbf0-f428-41e9-bdab-b0e0962720f1", 2024-05-17T22:02:05.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:02:05.711 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 msgr2=0x7fc92c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 0x7fc92c071940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc93c000f70 tx=0x7fc93c009040 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 msgr2=0x7fc94c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc93400b3e0 tx=0x7fc93400b8a0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 shutdown_connections 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc94c13e7b0 0x7fc94c13ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c145fd0 0x7fc94c15be20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc92c06f430 0x7fc92c071940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 --2- 172.21.15.83:0/1950689610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc94c0d03b0 0x7fc94c0b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 >> 172.21.15.83:0/1950689610 conn(0x7fc94c06cd90 msgr2=0x7fc94c15df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 shutdown_connections 2024-05-17T22:02:05.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.715+0000 7fc9514f6640 1 -- 172.21.15.83:0/1950689610 wait complete. 2024-05-17T22:02:05.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:05.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-17T22:02:05.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 Processor -- start 2024-05-17T22:02:05.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 -- start start 2024-05-17T22:02:05.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 0x7f9ba40d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ba409ba00 0x7f9ba409be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9ba40ce130 0x7f9ba40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba40d1a40 con 0x7f9ba409ba00 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba409d040 con 0x7f9ba40ce130 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9baa051640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba409c350 con 0x7f9ba40b88b0 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ba409ba00 0x7f9ba409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9ba409ba00 0x7f9ba409be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58854/0 (socket says 172.21.15.83:58854) 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3243481352 learned_addr learned my addr 172.21.15.83:0/3243481352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 0x7f9ba40d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/3243481352 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40ce130 0x7f9ba40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3243481352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 msgr2=0x7f9ba40d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 0x7f9ba40d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3243481352 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40ce130 msgr2=0x7f9ba40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3243481352 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40ce130 0x7f9ba40ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3243481352 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba409c710 con 0x7f9ba409ba00 2024-05-17T22:02:05.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.871+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 0x7f9ba40d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba409ba00 0x7f9ba409be10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009870 tx=0x7f9b9801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d14776d81083fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9ba27fc640 1 -- 172.21.15.83:0/3243481352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980b0070 con 0x7f9ba409ba00 2024-05-17T22:02:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9ba27fc640 1 -- 172.21.15.83:0/3243481352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980ab600 con 0x7f9ba409ba00 2024-05-17T22:02:05.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9ba27fc640 1 -- 172.21.15.83:0/3243481352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980ab9d0 con 0x7f9ba409ba00 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 -- 172.21.15.83:0/3243481352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba409ba00 msgr2=0x7f9ba409be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba409ba00 0x7f9ba409be10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009870 tx=0x7f9b9801c400 comp rx=0 tx=0).stop 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 -- 172.21.15.83:0/3243481352 shutdown_connections 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 --2- 172.21.15.83:0/3243481352 >> v2:172.21.15.155:3300/0 conn(0x7f9ba40ce130 0x7f9ba40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba409ba00 0x7f9ba409be10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 --2- 172.21.15.83:0/3243481352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba40b88b0 0x7f9ba40d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 -- 172.21.15.83:0/3243481352 >> 172.21.15.83:0/3243481352 conn(0x7f9ba40d3620 msgr2=0x7f9ba40d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 -- 172.21.15.83:0/3243481352 shutdown_connections 2024-05-17T22:02:05.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.875+0000 7f9baa051640 1 -- 172.21.15.83:0/3243481352 wait complete. 2024-05-17T22:02:05.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 Processor -- start 2024-05-17T22:02:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- start start 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba409ba00 0x7f9ba415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ba40b88b0 0x7f9ba40b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40b2f40 con 0x7f9ba40ce130 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40b2dc0 con 0x7f9ba40b88b0 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba40cee90 con 0x7f9ba409ba00 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58858/0 (socket says 172.21.15.83:58858) 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 -- 172.21.15.83:0/1599286937 learned_addr learned my addr 172.21.15.83:0/1599286937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba409ba00 0x7f9ba415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ba40b88b0 0x7f9ba40b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 -- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba409ba00 msgr2=0x7f9ba415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba409ba00 0x7f9ba415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 -- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ba40b88b0 msgr2=0x7f9ba40b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ba40b88b0 0x7f9ba40b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 -- 172.21.15.83:0/1599286937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba40b1450 con 0x7f9ba40ce130 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d550 tx=0x7f9b9400da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9b8bfff640 1 -- 172.21.15.83:0/1599286937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940125f0 con 0x7f9ba40ce130 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9b8bfff640 1 -- 172.21.15.83:0/1599286937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94019070 con 0x7f9ba40ce130 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba40a0e60 con 0x7f9ba40ce130 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9b8bfff640 1 -- 172.21.15.83:0/1599286937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940129d0 con 0x7f9ba40ce130 2024-05-17T22:02:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.879+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba40a1420 con 0x7f9ba40ce130 2024-05-17T22:02:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9b8bfff640 1 -- 172.21.15.83:0/1599286937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b94012b70 con 0x7f9ba40ce130 2024-05-17T22:02:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9b8bfff640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9b8bfff640 1 -- 172.21.15.83:0/1599286937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f9b9408f690 con 0x7f9ba40ce130 2024-05-17T22:02:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004430 tx=0x7f9b9c003a00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9bb12cb000 1 Processor -- start 2024-05-17T22:02:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.883+0000 7f9bb12cb000 1 -- start start 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b22360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379a9090 0x55f337b228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3379c0a90 con 0x55f3379bbf80 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f337934120 con 0x55f3379a8880 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f337923cc0 con 0x55f3379a9090 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58860/0 (socket says 172.21.15.83:58860) 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 -- 172.21.15.83:0/947197391 learned_addr learned my addr 172.21.15.83:0/947197391 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379a9090 0x55f337b228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 -- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379a9090 msgr2=0x55f337b228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379a9090 0x55f337b228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 -- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 msgr2=0x55f337b22360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b22360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b22360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 -- 172.21.15.83:0/947197391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3378e0110 con 0x55f3379bbf80 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009f30 tx=0x7f9b9c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203d5decb8f6131 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9b897fa640 1 -- 172.21.15.83:0/947197391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c026070 con 0x55f3379bbf80 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9b897fa640 1 -- 172.21.15.83:0/947197391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0147e0 con 0x55f3379bbf80 2024-05-17T22:02:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9b897fa640 1 -- 172.21.15.83:0/947197391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014b90 con 0x55f3379bbf80 2024-05-17T22:02:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- 172.21.15.83:0/947197391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 msgr2=0x55f337b21e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009f30 tx=0x7f9b9c012000 comp rx=0 tx=0).stop 2024-05-17T22:02:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- 172.21.15.83:0/947197391 shutdown_connections 2024-05-17T22:02:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379a9090 0x55f337b228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b22360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/947197391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379bbf80 0x55f337b21e20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- 172.21.15.83:0/947197391 >> 172.21.15.83:0/947197391 conn(0x55f3379f7010 msgr2=0x55f337ad75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- 172.21.15.83:0/947197391 shutdown_connections 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- 172.21.15.83:0/947197391 wait complete. 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 -- start start 2024-05-17T22:02:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.887+0000 7f9bb12cb000 1 Processor -- start 2024-05-17T22:02:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a9090 0x55f337adde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ade3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3378dfec0 con 0x55f3379a8880 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f337927d00 con 0x55f3379a9090 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f3375ed680 con 0x55f3379bbf80 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 10 client.0 _ll_register_callbacks cb 0x55f3379db1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58874/0 (socket says 172.21.15.83:58874) 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 -- 172.21.15.83:0/18043090 learned_addr learned my addr 172.21.15.83:0/18043090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ade3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a9090 0x55f337adde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 -- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 msgr2=0x55f337ade3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ade3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 -- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a9090 msgr2=0x55f337adde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a9090 0x55f337adde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 -- 172.21.15.83:0/18043090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3378e2890 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ade3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c024040 tx=0x7f9b9c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec286ee0f6436b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c026040 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f9b9c02c420 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b9c0147e0 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9b9c039050 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 10 client.6435 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55f3378e4650 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0147e0 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f9b9c03c020 con 0x55f3379a8880 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 10 client.6435 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 10 client.6435 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 20 client.6435 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375ca930 0x55f337ae4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 4 client.6435 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3378bfc40 con 0x55f3375ca930 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375ca930 0x55f337ae4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375ca930 0x55f337ae4410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a5045d30f08d77 server_cookie=c9f47445f82ccde1 in_seq=0 out_seq=0 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 10 client.6435 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 1 -- 172.21.15.83:0/18043090 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3378bfc40 con 0x55f3375ca930 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 10 client.6435 handle_command_reply: tid=0 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 client.6435 shutdown 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 msgr2=0x55f337b217f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c024040 tx=0x7f9b9c004ac0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 shutdown_connections 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ade3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a9090 0x55f337adde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375ca930 0x55f337ae4410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/18043090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a8880 0x55f337b217f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9b827fc640 0 client.6435 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 >> 172.21.15.83:0/18043090 conn(0x55f3379f7010 msgr2=0x55f337ad75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 shutdown_connections 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 1 -- 172.21.15.83:0/18043090 wait complete. 2024-05-17T22:02:05.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.891+0000 7f9bb12cb000 20 client.6435 trim_cache size 0 max 16384 2024-05-17T22:02:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 Processor -- start 2024-05-17T22:02:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- start start 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ad6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4203a10 con 0x55f3379a9090 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4202a30 con 0x55f3379a8880 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba40cb2d0 con 0x55f3379bbf80 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58876/0 (socket says 172.21.15.83:58876) 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/2265203070 learned_addr learned my addr 172.21.15.83:0/2265203070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 msgr2=0x55f337ad6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ad6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ad6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 msgr2=0x55f337b38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b38200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/2265203070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3378c38c0 con 0x55f3379a9090 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b9801c440 tx=0x7f9b980a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cb7128afb71d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9b88ff9640 1 -- 172.21.15.83:0/2265203070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980bc070 con 0x55f3379a9090 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9b88ff9640 1 -- 172.21.15.83:0/2265203070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980ab960 con 0x55f3379a9090 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9b88ff9640 1 -- 172.21.15.83:0/2265203070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980b0040 con 0x55f3379a9090 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- 172.21.15.83:0/2265203070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 msgr2=0x55f337b38740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9b9801c440 tx=0x7f9b980a6000 comp rx=0 tx=0).stop 2024-05-17T22:02:05.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- 172.21.15.83:0/2265203070 shutdown_connections 2024-05-17T22:02:05.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337ad6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b38740 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/2265203070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337b38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.911+0000 7f9bb12cb000 1 -- 172.21.15.83:0/2265203070 >> 172.21.15.83:0/2265203070 conn(0x55f3379f7010 msgr2=0x55f337add7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- 172.21.15.83:0/2265203070 shutdown_connections 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- 172.21.15.83:0/2265203070 wait complete. 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- start start 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 Processor -- start 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337ad9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337b510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3378eaf10 con 0x55f3379a9090 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40cb920 con 0x55f3379a8880 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f337adf470 con 0x55f3379bbf80 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 10 client.0 _ll_register_callbacks cb 0x7f9b9c024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:05.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58882/0 (socket says 172.21.15.83:58882) 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3843901277 learned_addr learned my addr 172.21.15.83:0/3843901277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba3fff640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337b510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337ad9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 msgr2=0x55f337b510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337b510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 msgr2=0x55f337ad9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337ad9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 -- 172.21.15.83:0/3843901277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3378defa0 con 0x55f3379a9090 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba2ffd640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9b9801c230 tx=0x7f9b9801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6d7d1f1cb0bf47 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980bc070 con 0x55f3379a9090 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b980ab960 con 0x55f3379a9090 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f9b980abd10 con 0x55f3379a9090 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9b980ca050 con 0x55f3379a9090 2024-05-17T22:02:05.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 10 client.6441 fetch_fsmap learned FSMap version 50 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55f3375de220 con 0x55f3379a9090 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980ab960 con 0x55f3379a9090 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f9b980cd020 con 0x55f3379a9090 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 10 client.6441 fetch_fsmap finished waiting for FSMap version 50 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 10 client.6441 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 20 client.6441 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375b8990 0x55f337b515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 4 client.6441 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f337936070 con 0x55f3375b8990 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375b8990 0x55f337b515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.915+0000 7f9b827fc640 10 client.6441 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9ba37fe640 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375b8990 0x55f337b515e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35e8c233b0e7889 server_cookie=88f1b01342f9bb8 in_seq=0 out_seq=0 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9b827fc640 1 -- 172.21.15.83:0/3843901277 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55f337936070 con 0x55f3375b8990 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9b827fc640 10 client.6441 handle_command_reply: tid=0 2024-05-17T22:02:05.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 client.6441 shutdown 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 msgr2=0x55f337b50b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9b9801c230 tx=0x7f9b9801ac40 comp rx=0 tx=0).stop 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 shutdown_connections 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f3379bbf80 0x55f337b510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55f3375b8990 0x55f337b515e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3379a9090 0x55f337b50b60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 --2- 172.21.15.83:0/3843901277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3379a8880 0x55f337ad9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 >> 172.21.15.83:0/3843901277 conn(0x55f3379f7010 msgr2=0x55f337addf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9b827fc640 0 client.6441 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 shutdown_connections 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 1 -- 172.21.15.83:0/3843901277 wait complete. 2024-05-17T22:02:05.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9bb12cb000 20 client.6441 trim_cache size 0 max 16384 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "37d1cbf0-f428-41e9-bdab-b0e0962720f1": { 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/subdir", 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "37d1cbf0-f428-41e9-bdab-b0e0962720f1", 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 msgr2=0x7f9b90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c004430 tx=0x7f9b9c003a00 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 msgr2=0x7f9ba40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d550 tx=0x7f9b9400da10 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 shutdown_connections 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ba40ce130 0x7f9ba40b4610 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b9006f400 0x7f9b90071910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ba40b88b0 0x7f9ba40b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 --2- 172.21.15.83:0/1599286937 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ba409ba00 0x7f9ba415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:05.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 >> 172.21.15.83:0/1599286937 conn(0x7f9ba40d3620 msgr2=0x7f9ba40d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 shutdown_connections 2024-05-17T22:02:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:05.919+0000 7f9baa051640 1 -- 172.21.15.83:0/1599286937 wait complete. 2024-05-17T22:02:05.929 INFO:tasks.cephfs.filesystem:scrub status for tag:37d1cbf0-f428-41e9-bdab-b0e0962720f1 - {'path': '/subdir', 'tag': '37d1cbf0-f428-41e9-bdab-b0e0962720f1', 'options': 'recursive,repair'} 2024-05-17T22:02:30.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:02:30.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:02:35.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:35.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-17T22:02:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 Processor -- start 2024-05-17T22:02:36.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 -- start start 2024-05-17T22:02:36.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f940415d700 0x7f94041673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040b5640 con 0x7f94040cdf40 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040b58e0 con 0x7f94040cf830 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f940a284640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9404070870 con 0x7f940415d700 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f93fb7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f94040cdf40 0x7f940415cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56524/0 (socket says 172.21.15.83:56524) 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f93fb7fe640 1 -- 172.21.15.83:0/3676861931 learned_addr learned my addr 172.21.15.83:0/3676861931 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9408821640 1 --2- 172.21.15.83:0/3676861931 >> v2:172.21.15.155:3300/0 conn(0x7f940415d700 0x7f94041673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750691881 0 0) 0x7f94040b5640 con 0x7f94040cdf40 2024-05-17T22:02:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93f0003530 con 0x7f94040cdf40 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 989841651 0 0) 0x7f93f0003530 con 0x7f94040cdf40 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 >> v2:172.21.15.155:3300/0 conn(0x7f940415d700 msgr2=0x7f94041673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 --2- 172.21.15.83:0/3676861931 >> v2:172.21.15.155:3300/0 conn(0x7f940415d700 0x7f94041673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 >> v1:172.21.15.155:6789/0 conn(0x7f94040cf830 legacy=0x7f94040b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940406f060 con 0x7f94040cdf40 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f93ec001720 con 0x7f94040cdf40 2024-05-17T22:02:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 >> v1:172.21.15.83:6789/0 conn(0x7f94040cdf40 legacy=0x7f940415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 0x7f93f0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f0004810 0x7f93f0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f00051d0 0x7f93f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93f000c140 con 0x7f93f0003a10 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93f000c2f0 con 0x7f93f0004810 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93f000c4f0 con 0x7f93f00051d0 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 0x7f93f0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f0004810 0x7f93f0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9408821640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f00051d0 0x7f93f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 -- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f00051d0 msgr2=0x7f93f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f00051d0 0x7f93f000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 -- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f0004810 msgr2=0x7f93f0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93f0004810 0x7f93f0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9408821640 1 -- 172.21.15.83:0/3676861931 reap_dead start 2024-05-17T22:02:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.103+0000 7f9403fff640 1 -- 172.21.15.83:0/3676861931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f0003530 con 0x7f93f0003a10 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f9403fff640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 0x7f93f0003e40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001fc0 tx=0x7f93f401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba6bc22907bddbd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4001b60 con 0x7f93f0003a10 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f40a95f0 con 0x7f93f0003a10 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f94037fe640 1 -- 172.21.15.83:0/3676861931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a9a20 con 0x7f93f0003a10 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 -- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 msgr2=0x7f93f0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 0x7f93f0003e40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001fc0 tx=0x7f93f401c880 comp rx=0 tx=0).stop 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 -- 172.21.15.83:0/3676861931 shutdown_connections 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 --2- 172.21.15.83:0/3676861931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f0003a10 0x7f93f0003e40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 -- 172.21.15.83:0/3676861931 >> 172.21.15.83:0/3676861931 conn(0x7f940406cd90 msgr2=0x7f94040b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 -- 172.21.15.83:0/3676861931 shutdown_connections 2024-05-17T22:02:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.107+0000 7f940a284640 1 -- 172.21.15.83:0/3676861931 wait complete. 2024-05-17T22:02:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 Processor -- start 2024-05-17T22:02:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- start start 2024-05-17T22:02:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f0003a10 0x7f9404156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94041510b0 0x7f94041514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f940406f9c0 con 0x7f9404145520 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f940406f840 con 0x7f94041510b0 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f940406fb40 con 0x7f93f0003a10 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50192/0 (socket says 172.21.15.83:50192) 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 -- 172.21.15.83:0/3138402441 learned_addr learned my addr 172.21.15.83:0/3138402441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9403fff640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f0003a10 0x7f9404156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94041510b0 0x7f94041514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 -- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f0003a10 msgr2=0x7f9404156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f0003a10 0x7f9404156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 -- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94041510b0 msgr2=0x7f94041514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94041510b0 0x7f94041514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 -- 172.21.15.83:0/3138402441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9404159a60 con 0x7f9404145520 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f9408821640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f94040d1460 tx=0x7f93fc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f94017fa640 1 -- 172.21.15.83:0/3138402441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00f040 con 0x7f9404145520 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f94017fa640 1 -- 172.21.15.83:0/3138402441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc00e840 con 0x7f9404145520 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9404144a40 con 0x7f9404145520 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f94017fa640 1 -- 172.21.15.83:0/3138402441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00ec30 con 0x7f9404145520 2024-05-17T22:02:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.111+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9404151a00 con 0x7f9404145520 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94017fa640 1 -- 172.21.15.83:0/3138402441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93fc0253f0 con 0x7f9404145520 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94017fa640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 0x7f93e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94017fa640 1 -- 172.21.15.83:0/3138402441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f93fc08f560 con 0x7f9404145520 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f9403fff640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 0x7f93e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f9403fff640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 0x7f93e4071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001f90 tx=0x7f93f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 Processor -- start 2024-05-17T22:02:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 -- start start 2024-05-17T22:02:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3bde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f244880 0x55e19f3be360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e19f25ca90 con 0x55e19f245090 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e19f1d0120 con 0x55e19f244880 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e19f1bfcc0 con 0x55e19f257f80 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f9408821640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f9408821640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50204/0 (socket says 172.21.15.83:50204) 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.115+0000 7f9408821640 1 -- 172.21.15.83:0/2480593507 learned_addr learned my addr 172.21.15.83:0/2480593507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9403fff640 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f244880 0x55e19f3be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 -- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 msgr2=0x55e19f3bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3bde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 -- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f244880 msgr2=0x55e19f3be360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f244880 0x55e19f3be360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 -- 172.21.15.83:0/2480593507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19f17c110 con 0x55e19f245090 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9408821640 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0583d0 tx=0x7f93fc05ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe15f98c3110dee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f93f9ffb640 1 -- 172.21.15.83:0/2480593507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc064bb0 con 0x55e19f245090 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f93f9ffb640 1 -- 172.21.15.83:0/2480593507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc058070 con 0x55e19f245090 2024-05-17T22:02:36.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f93f9ffb640 1 -- 172.21.15.83:0/2480593507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc064920 con 0x55e19f245090 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- 172.21.15.83:0/2480593507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 msgr2=0x55e19f3be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0583d0 tx=0x7f93fc05ba30 comp rx=0 tx=0).stop 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- 172.21.15.83:0/2480593507 shutdown_connections 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f245090 0x55e19f3be8a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f244880 0x55e19f3be360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- 172.21.15.83:0/2480593507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- 172.21.15.83:0/2480593507 >> 172.21.15.83:0/2480593507 conn(0x55e19f293010 msgr2=0x55e19f3735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- 172.21.15.83:0/2480593507 shutdown_connections 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- 172.21.15.83:0/2480593507 wait complete. 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- start start 2024-05-17T22:02:36.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 Processor -- start 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f37af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f37fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e19f17bec0 con 0x55e19f244880 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e19f1c3d00 con 0x55e19f245090 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e19ee89680 con 0x55e19f257f80 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f94114fe000 10 client.0 _ll_register_callbacks cb 0x55e19f2771c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:36.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50218/0 (socket says 172.21.15.83:50218) 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.119+0000 7f9403fff640 1 -- 172.21.15.83:0/3918784952 learned_addr learned my addr 172.21.15.83:0/3918784952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9408821640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f37fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f37af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 -- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 msgr2=0x55e19f37fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f37fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 -- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 msgr2=0x55e19f37af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f37af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 -- 172.21.15.83:0/3918784952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19f17e890 con 0x55e19f244880 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f9403fff640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93f401c6b0 tx=0x7f93f401b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96870256993b5673 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40a9d00 con 0x55e19f244880 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f93f40bc960 con 0x55e19f244880 2024-05-17T22:02:36.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93fc058070 con 0x55e19f244880 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f93f40d4050 con 0x55e19f244880 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 10 client.6453 fetch_fsmap learned FSMap version 51 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55e19f180650 con 0x55e19f244880 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93fc058070 con 0x55e19f244880 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f93f40d7020 con 0x55e19f244880 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 10 client.6453 fetch_fsmap finished waiting for FSMap version 51 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 10 client.6453 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 20 client.6453 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee66930 0x55e19f3c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 4 client.6453 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e19f15bc40 con 0x55e19ee66930 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee66930 0x55e19f3c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 10 client.6453 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee66930 0x55e19f3c4e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc116edfc8ccd05 server_cookie=cbfd02a76f81dca6 in_seq=0 out_seq=0 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3918784952 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e19f15bc40 con 0x55e19ee66930 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 10 client.6453 handle_command_reply: tid=0 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 client.6453 shutdown 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 msgr2=0x55e19f37aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93f401c6b0 tx=0x7f93f401b0c0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 shutdown_connections 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f37fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f37af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee66930 0x55e19f3c4e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 --2- 172.21.15.83:0/3918784952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f37aa20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f93e2ffd640 0 client.6453 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 >> 172.21.15.83:0/3918784952 conn(0x55e19f293010 msgr2=0x55e19f3735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 shutdown_connections 2024-05-17T22:02:36.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 1 -- 172.21.15.83:0/3918784952 wait complete. 2024-05-17T22:02:36.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.123+0000 7f94114fe000 20 client.6453 trim_cache size 0 max 16384 2024-05-17T22:02:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 Processor -- start 2024-05-17T22:02:36.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 -- start start 2024-05-17T22:02:36.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f245090 0x55e19f379f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f257f80 0x55e19f37a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9404201b10 con 0x55e19f244880 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9404200af0 con 0x55e19f257f80 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9404159110 con 0x55e19f245090 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50222/0 (socket says 172.21.15.83:50222) 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 -- 172.21.15.83:0/529540545 learned_addr learned my addr 172.21.15.83:0/529540545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 -- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f245090 msgr2=0x55e19f379f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f245090 0x55e19f379f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 -- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f257f80 msgr2=0x55e19f37a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f257f80 0x55e19f37a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 -- 172.21.15.83:0/529540545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19f15f8c0 con 0x55e19f244880 2024-05-17T22:02:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f9403fff640 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000f90 tx=0x7f93f4001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bb8e28c1f77164 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f93f97fa640 1 -- 172.21.15.83:0/529540545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40bc7e0 con 0x55e19f244880 2024-05-17T22:02:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f93f97fa640 1 -- 172.21.15.83:0/529540545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f40cc070 con 0x55e19f244880 2024-05-17T22:02:36.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.143+0000 7f93f97fa640 1 -- 172.21.15.83:0/529540545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40c7400 con 0x55e19f244880 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/529540545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 msgr2=0x55e19f3799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f93f4000f90 tx=0x7f93f4001700 comp rx=0 tx=0).stop 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/529540545 shutdown_connections 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f257f80 0x55e19f37a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f245090 0x55e19f379f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- 172.21.15.83:0/529540545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f3799d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/529540545 >> 172.21.15.83:0/529540545 conn(0x55e19f293010 msgr2=0x55e19f3d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/529540545 shutdown_connections 2024-05-17T22:02:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/529540545 wait complete. 2024-05-17T22:02:36.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- start start 2024-05-17T22:02:36.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 Processor -- start 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f375770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e19f1670f0 con 0x55e19f244880 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9404200480 con 0x55e19f245090 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e19f186f10 con 0x55e19f257f80 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 10 client.0 _ll_register_callbacks cb 0x7f93f0003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50234/0 (socket says 172.21.15.83:50234) 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 -- 172.21.15.83:0/3454974446 learned_addr learned my addr 172.21.15.83:0/3454974446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f375770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 -- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 msgr2=0x55e19f3edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 -- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 msgr2=0x55e19f375770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f375770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 -- 172.21.15.83:0/3454974446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19f17afa0 con 0x55e19f244880 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f375770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f9403fff640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f93f401c6b0 tx=0x7f93f40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5f4dff22796bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40c77f0 con 0x55e19f244880 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f93f40e3b40 con 0x55e19f244880 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f40cc070 con 0x55e19f244880 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f93f40e3da0 con 0x55e19f244880 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 10 client.6459 fetch_fsmap learned FSMap version 51 2024-05-17T22:02:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.147+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55e19ee7a220 con 0x55e19f244880 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f40cc070 con 0x55e19f244880 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f93f40d7040 con 0x55e19f244880 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 10 client.6459 fetch_fsmap finished waiting for FSMap version 51 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 10 client.6459 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 20 client.6459 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee54990 0x55e19f3ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 4 client.6459 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e19f1d2070 con 0x55e19ee54990 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee54990 0x55e19f3ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 10 client.6459 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93fb7fe640 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee54990 0x55e19f3ee710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76038c396dc9b66b server_cookie=f516cfcc04f14281 in_seq=0 out_seq=0 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 1 -- 172.21.15.83:0/3454974446 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e19f1d2070 con 0x55e19ee54990 2024-05-17T22:02:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 10 client.6459 handle_command_reply: tid=0 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 client.6459 shutdown 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 msgr2=0x55e19f375230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f93f401c6b0 tx=0x7f93f40a7000 comp rx=0 tx=0).stop 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 shutdown_connections 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e19f257f80 0x55e19f3edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e19f245090 0x55e19f375770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55e19ee54990 0x55e19f3ee710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 --2- 172.21.15.83:0/3454974446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e19f244880 0x55e19f375230 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 >> 172.21.15.83:0/3454974446 conn(0x55e19f293010 msgr2=0x55e19f3d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f93e2ffd640 0 client.6459 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 shutdown_connections 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 1 -- 172.21.15.83:0/3454974446 wait complete. 2024-05-17T22:02:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f94114fe000 20 client.6459 trim_cache size 0 max 16384 2024-05-17T22:02:36.148 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:36.148 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:02:36.148 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:02:36.148 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 msgr2=0x7f93e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 0x7f93e4071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93f4001f90 tx=0x7f93f4009040 comp rx=0 tx=0).stop 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 msgr2=0x7f9404145930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f94040d1460 tx=0x7f93fc010b70 comp rx=0 tx=0).stop 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 shutdown_connections 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9404145520 0x7f9404145930 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f93e406f400 0x7f93e4071910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94041510b0 0x7f94041514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 --2- 172.21.15.83:0/3138402441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93f0003a10 0x7f9404156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 >> 172.21.15.83:0/3138402441 conn(0x7f940406cd90 msgr2=0x7f94040b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 shutdown_connections 2024-05-17T22:02:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.151+0000 7f940a284640 1 -- 172.21.15.83:0/3138402441 wait complete. 2024-05-17T22:02:36.163 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:02:36.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:36.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.311+0000 7f9f8c647640 1 Processor -- start 2024-05-17T22:02:36.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.311+0000 7f9f8c647640 1 -- start start 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8c647640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8c647640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f840cf1f0 0x7f9f840cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8c647640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f840cfb40 con 0x7f9f8415c8f0 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8c647640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9f840b5d00 con 0x7f9f840cf1f0 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8c647640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f840b5f50 con 0x7f9f840d1080 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f840cf1f0 0x7f9f840cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8a3e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9f840cf1f0 0x7f9f840cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:51610/0 (socket says 172.21.15.83:51610) 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 -- 172.21.15.83:0/3919144643 learned_addr learned my addr 172.21.15.83:0/3919144643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f8abe4640 1 --1- 172.21.15.83:0/3919144643 >> v1:172.21.15.155:6789/0 conn(0x7f9f8415c8f0 0x7f9f8415cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:42650/0 (socket says 172.21.15.83:42650) 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 -- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 msgr2=0x7f9f840d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 --2- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 -- 172.21.15.83:0/3919144643 >> v1:172.21.15.155:6789/0 conn(0x7f9f8415c8f0 legacy=0x7f9f8415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.315+0000 7f9f89be2640 1 -- 172.21.15.83:0/3919144643 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f840b6230 con 0x7f9f840cf1f0 2024-05-17T22:02:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f89be2640 1 --2- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f840cf1f0 0x7f9f840cf600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009870 tx=0x7f9f7401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3f93ef51146579e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f893e1640 1 -- 172.21.15.83:0/3919144643 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740b1070 con 0x7f9f840cf1f0 2024-05-17T22:02:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f893e1640 1 -- 172.21.15.83:0/3919144643 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f740ac620 con 0x7f9f840cf1f0 2024-05-17T22:02:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f893e1640 1 -- 172.21.15.83:0/3919144643 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740ac9f0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 -- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f840cf1f0 msgr2=0x7f9f840cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 --2- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f840cf1f0 0x7f9f840cf600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009870 tx=0x7f9f7401c4d0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 -- 172.21.15.83:0/3919144643 shutdown_connections 2024-05-17T22:02:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 --2- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f840cf1f0 0x7f9f840cf600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 --2- 172.21.15.83:0/3919144643 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 -- 172.21.15.83:0/3919144643 >> 172.21.15.83:0/3919144643 conn(0x7f9f840742e0 msgr2=0x7f9f8406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 -- 172.21.15.83:0/3919144643 shutdown_connections 2024-05-17T22:02:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.319+0000 7f9f8c647640 1 -- 172.21.15.83:0/3919144643 wait complete. 2024-05-17T22:02:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 Processor -- start 2024-05-17T22:02:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- start start 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8415c8f0 0x7f9f840b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f840b7800 con 0x7f9f840cf1f0 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84071bf0 con 0x7f9f8415c8f0 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f840b7980 con 0x7f9f840d1080 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50250/0 (socket says 172.21.15.83:50250) 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 -- 172.21.15.83:0/1871554488 learned_addr learned my addr 172.21.15.83:0/1871554488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8abe4640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8415c8f0 0x7f9f840b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f89be2640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 -- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 msgr2=0x7f9f840b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 -- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8415c8f0 msgr2=0x7f9f840b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8415c8f0 0x7f9f840b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 -- 172.21.15.83:0/1871554488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f840b16d0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8a3e3640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e870 tx=0x7f9f8000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80017070 con 0x7f9f840cf1f0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8000c940 con 0x7f9f840cf1f0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f840b19b0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000ccf0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f840b1f70 con 0x7f9f840cf1f0 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.323+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8414b2a0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.327+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9f8001a070 con 0x7f9f840cf1f0 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.327+0000 7f9f7affd640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 0x7f9f68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.327+0000 7f9f89be2640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 0x7f9f68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.327+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f9f8008f260 con 0x7f9f840cf1f0 2024-05-17T22:02:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.327+0000 7f9f89be2640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 0x7f9f68071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9f740ae000 tx=0x7f9f74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.331+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9f8414b2a0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f840cf600 con 0x7f9f840cf1f0 2024-05-17T22:02:36.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f7affd640 1 -- 172.21.15.83:0/1871554488 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5188 (secure 0 0 0) 0x7f9f8005c0c0 con 0x7f9f840cf1f0 2024-05-17T22:02:36.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:36.515 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:02:09.307460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:36.515 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-05-17T22:02:36.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 msgr2=0x7f9f68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 0x7f9f68071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9f740ae000 tx=0x7f9f74008040 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 msgr2=0x7f9f84149990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e870 tx=0x7f9f8000ec70 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 shutdown_connections 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f6806f400 0x7f9f68071910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f8415c8f0 0x7f9f840b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f840d1080 0x7f9f840b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 --2- 172.21.15.83:0/1871554488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f840cf1f0 0x7f9f84149990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.523+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 >> 172.21.15.83:0/1871554488 conn(0x7f9f840742e0 msgr2=0x7f9f84157120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.527+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 shutdown_connections 2024-05-17T22:02:36.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.527+0000 7f9f8c647640 1 -- 172.21.15.83:0/1871554488 wait complete. 2024-05-17T22:02:36.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-17T22:02:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.679+0000 7faf03c8e640 1 Processor -- start 2024-05-17T22:02:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.679+0000 7faf03c8e640 1 -- start start 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf03c8e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faefc0b4810 0x7faefc0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf03c8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 0x7faefc0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf03c8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faefc0d4170 con 0x7faefc0d0290 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf03c8e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7faefc0d4450 con 0x7faefc0b4810 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf03c8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faefc0d4730 con 0x7faefc0d2000 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf0222b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 0x7faefc0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faefc0b4810 0x7faefc0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf0222b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 0x7faefc0d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:42508/0 (socket says 172.21.15.83:42508) 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faefc0b4810 0x7faefc0d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:51628/0 (socket says 172.21.15.83:51628) 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf0222b640 1 -- 172.21.15.83:0/3956205069 learned_addr learned my addr 172.21.15.83:0/3956205069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 -- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 msgr2=0x7faefc0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 --2- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 0x7faefc0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 -- 172.21.15.83:0/3956205069 >> v1:172.21.15.155:6789/0 conn(0x7faefc0d0290 legacy=0x7faefc15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf00a28640 1 -- 172.21.15.83:0/3956205069 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2857587015 0 0) 0x7faefc0d4170 con 0x7faefc0d0290 2024-05-17T22:02:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.683+0000 7faf01a2a640 1 -- 172.21.15.83:0/3956205069 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0d4a10 con 0x7faefc0b4810 2024-05-17T22:02:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf01a2a640 1 --2- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc0d5210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faef80097c0 tx=0x7faef801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18bf24a6f702684f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf00a28640 1 -- 172.21.15.83:0/3956205069 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80aa620 con 0x7faefc0b4810 2024-05-17T22:02:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf00a28640 1 -- 172.21.15.83:0/3956205069 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80af070 con 0x7faefc0b4810 2024-05-17T22:02:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf00a28640 1 -- 172.21.15.83:0/3956205069 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80aa9e0 con 0x7faefc0b4810 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 -- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 msgr2=0x7faefc0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 --2- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc0d5210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faef80097c0 tx=0x7faef801c4a0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 -- 172.21.15.83:0/3956205069 shutdown_connections 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 --2- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d2000 0x7faefc0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 --2- 172.21.15.83:0/3956205069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc0d5210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 -- 172.21.15.83:0/3956205069 >> 172.21.15.83:0/3956205069 conn(0x7faefc06e010 msgr2=0x7faefc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 -- 172.21.15.83:0/3956205069 shutdown_connections 2024-05-17T22:02:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.687+0000 7faf03c8e640 1 -- 172.21.15.83:0/3956205069 wait complete. 2024-05-17T22:02:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 Processor -- start 2024-05-17T22:02:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- start start 2024-05-17T22:02:36.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d0290 0x7faefc0b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0b9670 con 0x7faefc0d2000 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0b94f0 con 0x7faefc0b4810 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faefc0b97f0 con 0x7faefc0d0290 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50260/0 (socket says 172.21.15.83:50260) 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 -- 172.21.15.83:0/1067919630 learned_addr learned my addr 172.21.15.83:0/1067919630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf01229640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d0290 0x7faefc0b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 -- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d0290 msgr2=0x7faefc0b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d0290 0x7faefc0b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 -- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 msgr2=0x7faefc152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 -- 172.21.15.83:0/1067919630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0b7130 con 0x7faefc0d2000 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf0222b640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faef400efc0 tx=0x7faef400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faef27fc640 1 -- 172.21.15.83:0/1067919630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4009e60 con 0x7faefc0d2000 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faef27fc640 1 -- 172.21.15.83:0/1067919630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef400ccc0 con 0x7faefc0d2000 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faef27fc640 1 -- 172.21.15.83:0/1067919630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4003be0 con 0x7faefc0d2000 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc0b7410 con 0x7faefc0d2000 2024-05-17T22:02:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.691+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc0b79d0 con 0x7faefc0d2000 2024-05-17T22:02:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faef27fc640 1 -- 172.21.15.83:0/1067919630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faef4012050 con 0x7faefc0d2000 2024-05-17T22:02:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faef27fc640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 0x7faee0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faef27fc640 1 -- 172.21.15.83:0/1067919630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7faef408f710 con 0x7faefc0d2000 2024-05-17T22:02:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 0x7faee0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 0x7faee0071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faef80ae000 tx=0x7faef80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faf0af08000 1 Processor -- start 2024-05-17T22:02:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.695+0000 7faf0af08000 1 -- start start 2024-05-17T22:02:36.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d137e00 0x55714d27d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d117030 0x55714d27daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55714d11bfa0 con 0x55714d12d460 2024-05-17T22:02:36.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55714d0902e0 con 0x55714d117030 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55714d07fe80 con 0x55714d137e00 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50272/0 (socket says 172.21.15.83:50272) 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 -- 172.21.15.83:0/3711009697 learned_addr learned my addr 172.21.15.83:0/3711009697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01a2a640 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d137e00 0x55714d27d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0222b640 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d117030 0x55714d27daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 -- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d137e00 msgr2=0x55714d27d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d137e00 0x55714d27d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 -- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d117030 msgr2=0x55714d27daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d117030 0x55714d27daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 -- 172.21.15.83:0/3711009697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55714d12ab80 con 0x55714d12d460 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf01229640 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7faeec001550 tx=0x7faeec001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac47a4af75fe2723 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faedf7fe640 1 -- 172.21.15.83:0/3711009697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec019070 con 0x55714d12d460 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faedf7fe640 1 -- 172.21.15.83:0/3711009697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec0050d0 con 0x55714d12d460 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faedf7fe640 1 -- 172.21.15.83:0/3711009697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec00e740 con 0x55714d12d460 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- 172.21.15.83:0/3711009697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 msgr2=0x55714d27d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7faeec001550 tx=0x7faeec001a10 comp rx=0 tx=0).stop 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- 172.21.15.83:0/3711009697 shutdown_connections 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d117030 0x55714d27daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d27d560 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- 172.21.15.83:0/3711009697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d137e00 0x55714d27d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- 172.21.15.83:0/3711009697 >> 172.21.15.83:0/3711009697 conn(0x55714cd41760 msgr2=0x55714d23a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- 172.21.15.83:0/3711009697 shutdown_connections 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- 172.21.15.83:0/3711009697 wait complete. 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- start start 2024-05-17T22:02:36.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 Processor -- start 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 0x55714d2847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55714d03beb0 con 0x55714d137e00 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55714d083ec0 con 0x55714d12d460 2024-05-17T22:02:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55714cd49820 con 0x55714d117030 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0af08000 10 client.0 _ll_register_callbacks cb 0x55714d21fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.699+0000 7faf0222b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50284/0 (socket says 172.21.15.83:50284) 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 -- 172.21.15.83:0/1558504167 learned_addr learned my addr 172.21.15.83:0/1558504167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf01229640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 0x55714d2847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 -- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 msgr2=0x55714d2842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 -- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 msgr2=0x55714d2847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 0x55714d2847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 -- 172.21.15.83:0/1558504167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55714d0c8ef0 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf01229640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 0x55714d2847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0222b640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faef400c990 tx=0x7faef405b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1826528e9b014c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef405b360 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7faef4057870 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeec0050d0 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7faef405bf00 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 10 client.6471 fetch_fsmap learned FSMap version 51 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55714d0bd910 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec0050d0 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7faef4057a70 con 0x55714d137e00 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 10 client.6471 fetch_fsmap finished waiting for FSMap version 51 2024-05-17T22:02:36.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 10 client.6471 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 20 client.6471 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd26ad0 0x55714d28e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 4 client.6471 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55714d01bcf0 con 0x55714cd26ad0 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd26ad0 0x55714d28e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 10 client.6471 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf01a2a640 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd26ad0 0x55714d28e680 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38301a80dbacb99 server_cookie=6bea38040c5ff418 in_seq=0 out_seq=0 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 1 -- 172.21.15.83:0/1558504167 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55714d01bcf0 con 0x55714cd26ad0 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 10 client.6471 handle_command_reply: tid=0 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 client.6471 shutdown 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 msgr2=0x55714d284d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faef400c990 tx=0x7faef405b940 comp rx=0 tx=0).stop 2024-05-17T22:02:36.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 shutdown_connections 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd26ad0 0x55714d28e680 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d137e00 0x55714d284d20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d12d460 0x55714d2847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faedcff9640 0 client.6471 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 --2- 172.21.15.83:0/1558504167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 >> 172.21.15.83:0/1558504167 conn(0x55714cd41760 msgr2=0x55714d2324b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 shutdown_connections 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 1 -- 172.21.15.83:0/1558504167 wait complete. 2024-05-17T22:02:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.703+0000 7faf0af08000 20 client.6471 trim_cache size 0 max 16384 2024-05-17T22:02:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.715+0000 7faf0af08000 1 Processor -- start 2024-05-17T22:02:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.715+0000 7faf0af08000 1 -- start start 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faefc205550 con 0x55714d12d460 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faefc2045d0 con 0x55714d137e00 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faefc143fc0 con 0x55714d117030 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50288/0 (socket says 172.21.15.83:50288) 2024-05-17T22:02:36.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 -- 172.21.15.83:0/1757872930 learned_addr learned my addr 172.21.15.83:0/1757872930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 -- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 msgr2=0x55714d2eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 -- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 msgr2=0x55714d2ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 -- 172.21.15.83:0/1757872930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55714d03d670 con 0x55714d12d460 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf01229640 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faeec001a50 tx=0x7faeec00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e276d14a4380a20 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faedeffd640 1 -- 172.21.15.83:0/1757872930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec019040 con 0x55714d12d460 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faedeffd640 1 -- 172.21.15.83:0/1757872930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec023070 con 0x55714d12d460 2024-05-17T22:02:36.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faedeffd640 1 -- 172.21.15.83:0/1757872930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0052f0 con 0x55714d12d460 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- 172.21.15.83:0/1757872930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 msgr2=0x55714d2eb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faeec001a50 tx=0x7faeec00a000 comp rx=0 tx=0).stop 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- 172.21.15.83:0/1757872930 shutdown_connections 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d2eb8d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 --2- 172.21.15.83:0/1757872930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d2eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- 172.21.15.83:0/1757872930 >> 172.21.15.83:0/1757872930 conn(0x55714cd41760 msgr2=0x55714d27b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- 172.21.15.83:0/1757872930 shutdown_connections 2024-05-17T22:02:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- 172.21.15.83:0/1757872930 wait complete. 2024-05-17T22:02:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 -- start start 2024-05-17T22:02:36.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.719+0000 7faf0af08000 1 Processor -- start 2024-05-17T22:02:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d234ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55714d13f260 con 0x55714d12d460 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faefc15ce30 con 0x55714d137e00 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55714d01b850 con 0x55714d117030 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 10 client.0 _ll_register_callbacks cb 0x7faef4009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50294/0 (socket says 172.21.15.83:50294) 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 -- 172.21.15.83:0/679672694 learned_addr learned my addr 172.21.15.83:0/679672694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 -- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 msgr2=0x55714d234ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d234ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 -- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 msgr2=0x55714d2ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 -- 172.21.15.83:0/679672694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55714d03c100 con 0x55714d12d460 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01229640 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faeec020040 tx=0x7faeec000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d1e91f58f9f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec019040 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7faeec0053a0 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeec023070 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7faeec034050 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 10 client.6477 fetch_fsmap learned FSMap version 51 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55714d104d80 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec023070 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7faeec037020 con 0x55714d12d460 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 10 client.6477 fetch_fsmap finished waiting for FSMap version 51 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 10 client.6477 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 20 client.6477 populate_metadata read hostname 'smithi083' 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd14b30 0x55714d2abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 4 client.6477 mds_command: new command op to 6234 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 --> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55714d092230 con 0x55714cd14b30 2024-05-17T22:02:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01a2a640 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd14b30 0x55714d2abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 10 client.6477 ms_handle_connect on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf01a2a640 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd14b30 0x55714d2abe90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65927634ced4b91b server_cookie=bce70aaad5416563 in_seq=0 out_seq=0 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 1 -- 172.21.15.83:0/679672694 <== mds.0 v2:172.21.15.83:6836/1116165089 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55714d092230 con 0x55714cd14b30 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 10 client.6477 handle_command_reply: tid=0 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 client.6477 shutdown 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 msgr2=0x55714d234ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faeec020040 tx=0x7faeec000ee0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 shutdown_connections 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55714d137e00 0x55714d2ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:6836/1116165089,v1:172.21.15.83:6837/1116165089] conn(0x55714cd14b30 0x55714d2abe90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55714d12d460 0x55714d234ff0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faedcff9640 0 client.6477 ms_handle_reset on v2:172.21.15.83:6836/1116165089 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 --2- 172.21.15.83:0/679672694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55714d117030 0x55714d234ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 >> 172.21.15.83:0/679672694 conn(0x55714cd41760 msgr2=0x55714d27bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 shutdown_connections 2024-05-17T22:02:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 1 -- 172.21.15.83:0/679672694 wait complete. 2024-05-17T22:02:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf0af08000 20 client.6477 trim_cache size 0 max 16384 2024-05-17T22:02:36.721 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:36.721 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 msgr2=0x7faee0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 0x7faee0071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faef80ae000 tx=0x7faef80a7000 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 msgr2=0x7faefc154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faef400efc0 tx=0x7faef400c490 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 shutdown_connections 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faefc0d2000 0x7faefc154560 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faefc0d0290 0x7faefc0b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faee006f400 0x7faee0071910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 --2- 172.21.15.83:0/1067919630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faefc0b4810 0x7faefc152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 >> 172.21.15.83:0/1067919630 conn(0x7faefc06e010 msgr2=0x7faefc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.723+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 shutdown_connections 2024-05-17T22:02:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.727+0000 7faf03c8e640 1 -- 172.21.15.83:0/1067919630 wait complete. 2024-05-17T22:02:36.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:02:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.879+0000 7f145b1da640 1 Processor -- start 2024-05-17T22:02:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.879+0000 7f145b1da640 1 -- start start 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f145b1da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f145415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f145b1da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f145415e6b0 0x7f145406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f145b1da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14541465d0 con 0x7f1454070510 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f145b1da640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f145406ef30 con 0x7f145415e6b0 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f145b1da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f145406f210 con 0x7f14541461c0 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f145415e6b0 0x7f145406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f1459777640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1454070510 0x7f14541605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56530/0 (socket says 172.21.15.83:56530) 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f1459777640 1 -- 172.21.15.83:0/3374920526 learned_addr learned my addr 172.21.15.83:0/3374920526 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f1458f76640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f145415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144bfff640 1 -- 172.21.15.83:0/3374920526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 msgr2=0x7f145415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144bfff640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f145415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144bfff640 1 -- 172.21.15.83:0/3374920526 >> v1:172.21.15.83:6789/0 conn(0x7f1454070510 legacy=0x7f14541605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144b7fe640 1 -- 172.21.15.83:0/3374920526 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705564561 0 0) 0x7f14541465d0 con 0x7f1454070510 2024-05-17T22:02:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.883+0000 7f144bfff640 1 -- 172.21.15.83:0/3374920526 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1454070920 con 0x7f145415e6b0 2024-05-17T22:02:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f144bfff640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145415e6b0 0x7f145406ffd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f14440097c0 tx=0x7f144401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37716cf511b8bc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f144b7fe640 1 -- 172.21.15.83:0/3374920526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440a7070 con 0x7f145415e6b0 2024-05-17T22:02:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f144b7fe640 1 -- 172.21.15.83:0/3374920526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14440b0070 con 0x7f145415e6b0 2024-05-17T22:02:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f144b7fe640 1 -- 172.21.15.83:0/3374920526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440ab890 con 0x7f145415e6b0 2024-05-17T22:02:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- 172.21.15.83:0/3374920526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145415e6b0 msgr2=0x7f145406ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145415e6b0 0x7f145406ffd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f14440097c0 tx=0x7f144401c470 comp rx=0 tx=0).stop 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- 172.21.15.83:0/3374920526 shutdown_connections 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145415e6b0 0x7f145406ffd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 --2- 172.21.15.83:0/3374920526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f145415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- 172.21.15.83:0/3374920526 >> 172.21.15.83:0/3374920526 conn(0x7f145406cd90 msgr2=0x7f14540b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- 172.21.15.83:0/3374920526 shutdown_connections 2024-05-17T22:02:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- 172.21.15.83:0/3374920526 wait complete. 2024-05-17T22:02:36.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 Processor -- start 2024-05-17T22:02:36.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.887+0000 7f145b1da640 1 -- start start 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f14540d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145415e6b0 0x7f14540d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1454160e30 con 0x7f1454070510 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1454160cb0 con 0x7f145415e6b0 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f14540d10a0 con 0x7f14541461c0 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50324/0 (socket says 172.21.15.83:50324) 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 -- 172.21.15.83:0/207836012 learned_addr learned my addr 172.21.15.83:0/207836012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1459777640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145415e6b0 0x7f14540d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 -- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 msgr2=0x7f14540d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f14540d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 -- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145415e6b0 msgr2=0x7f14540d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145415e6b0 0x7f14540d1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 -- 172.21.15.83:0/207836012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145413bd50 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f1458f76640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f145000d5d0 tx=0x7f145000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450004240 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500043e0 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145413c030 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450003840 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145413c5f0 con 0x7f1454070510 2024-05-17T22:02:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145415e170 con 0x7f1454070510 2024-05-17T22:02:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14500039e0 con 0x7f1454070510 2024-05-17T22:02:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 0x7f1440071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f144bfff640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 0x7f1440071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f1450090270 con 0x7f1454070510 2024-05-17T22:02:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.891+0000 7f144bfff640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 0x7f1440071910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14440af000 tx=0x7f1444008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:36.895+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f145005d030 con 0x7f1454070510 2024-05-17T22:02:37.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.079+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f145413cd30 con 0x7f1454070510 2024-05-17T22:02:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.079+0000 7f14497fa640 1 -- 172.21.15.83:0/207836012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f145005c930 con 0x7f1454070510 2024-05-17T22:02:37.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:37.071 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T22:02:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 msgr2=0x7f1440071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 0x7f1440071910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14440af000 tx=0x7f1444008040 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 msgr2=0x7f145415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f145000d5d0 tx=0x7f145000da90 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 shutdown_connections 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144006f400 0x7f1440071910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145415e6b0 0x7f14540d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14541461c0 0x7f14540d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 --2- 172.21.15.83:0/207836012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1454070510 0x7f145415bcd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 >> 172.21.15.83:0/207836012 conn(0x7f145406cd90 msgr2=0x7f1454075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 shutdown_connections 2024-05-17T22:02:37.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.083+0000 7f145b1da640 1 -- 172.21.15.83:0/207836012 wait complete. 2024-05-17T22:02:37.092 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-17T22:02:37.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:02:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 Processor -- start 2024-05-17T22:02:37.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 -- start start 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f050015df90 0x7f05000d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f050006ec90 0x7f05000d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050009be60 con 0x7f05000d2220 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f05000ccc90 con 0x7f050015df90 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f05065c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f050009c140 con 0x7f050006ec90 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f04fffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f050015df90 0x7f05000d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f04fffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f050015df90 0x7f05000d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47370/0 (socket says 172.21.15.83:47370) 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f04fffff640 1 -- 172.21.15.83:0/1487990905 learned_addr learned my addr 172.21.15.83:0/1487990905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:37.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.251+0000 7f04ff7fe640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f050006ec90 0x7f05000d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04fffff640 1 -- 172.21.15.83:0/1487990905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f050006ec90 msgr2=0x7f05000d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04fffff640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f050006ec90 0x7f05000d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04fffff640 1 -- 172.21.15.83:0/1487990905 >> v1:172.21.15.155:6789/0 conn(0x7f05000d2220 legacy=0x7f05000cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:37.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04fffff640 1 -- 172.21.15.83:0/1487990905 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05000af6f0 con 0x7f050015df90 2024-05-17T22:02:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04fffff640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f05000d17a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009870 tx=0x7f04f801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7613228fece167 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04feffd640 1 -- 172.21.15.83:0/1487990905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80b1070 con 0x7f050015df90 2024-05-17T22:02:37.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04feffd640 1 -- 172.21.15.83:0/1487990905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f80ac620 con 0x7f050015df90 2024-05-17T22:02:37.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f04feffd640 1 -- 172.21.15.83:0/1487990905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80ac9f0 con 0x7f050015df90 2024-05-17T22:02:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 -- 172.21.15.83:0/1487990905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 msgr2=0x7f05000d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f05000d17a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009870 tx=0x7f04f801c440 comp rx=0 tx=0).stop 2024-05-17T22:02:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 -- 172.21.15.83:0/1487990905 shutdown_connections 2024-05-17T22:02:37.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f050006ec90 0x7f05000d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 --2- 172.21.15.83:0/1487990905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f05000d17a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 -- 172.21.15.83:0/1487990905 >> 172.21.15.83:0/1487990905 conn(0x7f050009b8b0 msgr2=0x7f05000b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 -- 172.21.15.83:0/1487990905 shutdown_connections 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.255+0000 7f05065c0640 1 -- 172.21.15.83:0/1487990905 wait complete. 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 Processor -- start 2024-05-17T22:02:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 -- start start 2024-05-17T22:02:37.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f050006ec90 0x7f0500142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05000d2220 0x7f0500154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05000b1360 con 0x7f050015df90 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f05000b11e0 con 0x7f050006ec90 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f05065c0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f050016a980 con 0x7f05000d2220 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47372/0 (socket says 172.21.15.83:47372) 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 -- 172.21.15.83:0/3473188810 learned_addr learned my addr 172.21.15.83:0/3473188810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f04fffff640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f050006ec90 0x7f0500142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f04ff7fe640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05000d2220 0x7f0500154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 -- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05000d2220 msgr2=0x7f0500154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05000d2220 0x7f0500154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 -- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f050006ec90 msgr2=0x7f0500142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f050006ec90 0x7f0500142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 -- 172.21.15.83:0/3473188810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05000b24b0 con 0x7f050015df90 2024-05-17T22:02:37.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.259+0000 7f0504b5d640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b9e0 tx=0x7f04f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0003b80 con 0x7f050015df90 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0003d20 con 0x7f050015df90 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cb60 con 0x7f050015df90 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05000b2790 con 0x7f050015df90 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05000b2d50 con 0x7f050015df90 2024-05-17T22:02:37.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05000cef60 con 0x7f050015df90 2024-05-17T22:02:37.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04f0006d10 con 0x7f050015df90 2024-05-17T22:02:37.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:37.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f04f008edf0 con 0x7f050015df90 2024-05-17T22:02:37.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fffff640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:37.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.263+0000 7f04fffff640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 0x7f04e0071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ae000 tx=0x7f04f80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:37.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.267+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f04f005b600 con 0x7f050015df90 2024-05-17T22:02:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.455+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f05000717e0 con 0x7f050015df90 2024-05-17T22:02:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f04fcff9640 1 -- 172.21.15.83:0/3473188810 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f04f005f900 con 0x7f050015df90 2024-05-17T22:02:37.450 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:02:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 msgr2=0x7f04e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 0x7f04e0071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ae000 tx=0x7f04f80a7000 comp rx=0 tx=0).stop 2024-05-17T22:02:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 msgr2=0x7f0500155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b9e0 tx=0x7f04f000bea0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 shutdown_connections 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050015df90 0x7f0500155290 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05000d2220 0x7f0500154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 --2- 172.21.15.83:0/3473188810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f050006ec90 0x7f0500142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 >> 172.21.15.83:0/3473188810 conn(0x7f050009b8b0 msgr2=0x7f05000b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 shutdown_connections 2024-05-17T22:02:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37.459+0000 7f05065c0640 1 -- 172.21.15.83:0/3473188810 wait complete. 2024-05-17T22:02:37.470 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:02:37.471 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:02:37.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:37.471 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:02:37.478 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42187sec preferred_lft 42187sec 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.479 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:02:37.480 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:02:37.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:37.481 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:02:37.529 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:02:37.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:37.531 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:02:37.581 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:02:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:02:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:02:37.582 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:02:37.582 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:02:37.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:02:37.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:02:37.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:37.696 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:02:37.741 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:02:37.741 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:02:37.742 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:02:37.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:37.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:02:37.797 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:02:37.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:02:37.797 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:37.845 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:02:37.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:02:37.846 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:37.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:02:37.951 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:02:37.999 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:02:37.999 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:02:37.999 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:02:37.999 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:02:37.999 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:02:38.000 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:38.001 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:38.002 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:38.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:38.002 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:02:38.047 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:02:38.047 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:02:38.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:02:38.103 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:02:38.150 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:02:38.151 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:38.152 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:38.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:38.153 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:02:38.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:02:38.207+0000 7f65fedb9580 -1 init, newargv = 0x563849337080 newargc=15 2024-05-17T22:02:38.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[42313]: starting ceph client 2024-05-17T22:02:38.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[42313]: starting fuse 2024-05-17T22:02:39.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:02:39.318 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:02:39.366 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:02:39.367 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:02:39.368 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:02:39.369 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:39.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:39.370 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:02:39.417 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:02:39.417 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:02:39.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:02:39.417 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:02:39.417 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:02:39.418 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:02:39.419 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:02:39.543 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.42313.asok 2024-05-17T22:02:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:02:39.615 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42313.asok 2024-05-17T22:02:39.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:39.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42313.asok status 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "42313", 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:39.798 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6498, 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6498 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1705537094 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:02:39.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1705537094 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6498 192.168.0.1:0/1705537094", 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1705537094", 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 51, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 70, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:02:39.800 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:39.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:39.814 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:02:39.814 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:39.819 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:02:39.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:39.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:39.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:39.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:39.871 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-17T22:02:39.922 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:02:39.922 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-17T22:02:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 Processor -- start 2024-05-17T22:02:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 -- start start 2024-05-17T22:02:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 0x56376bf0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf99460 0x56376bf953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56376be2a1b0 con 0x56376bf25100 2024-05-17T22:02:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56376be85ef0 con 0x56376bf24ac0 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56376bedaed0 con 0x56376bf99460 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47388/0 (socket says 172.21.15.83:47388) 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 -- 172.21.15.83:0/604106287 learned_addr learned my addr 172.21.15.83:0/604106287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f67791640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf99460 0x56376bf953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f66f90640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 0x56376bf0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 -- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf99460 msgr2=0x56376bf953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf99460 0x56376bf953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 -- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 msgr2=0x56376bf0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 0x56376bf0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 -- 172.21.15.83:0/604106287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376be04c10 con 0x56376bf25100 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f66f90640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 0x56376bf0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.039+0000 7f8f6678f640 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001ec0 tx=0x7f8f5c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f09ef8bce75ad06 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6578d640 1 -- 172.21.15.83:0/604106287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00d510 con 0x56376bf25100 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6578d640 1 -- 172.21.15.83:0/604106287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c008070 con 0x56376bf25100 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6578d640 1 -- 172.21.15.83:0/604106287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00d910 con 0x56376bf25100 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/604106287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 msgr2=0x56376bf94e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001ec0 tx=0x7f8f5c00b6b0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/604106287 shutdown_connections 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf99460 0x56376bf953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bf94e90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/604106287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf24ac0 0x56376bf0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/604106287 >> 172.21.15.83:0/604106287 conn(0x56376bec6e40 msgr2=0x56376bedec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/604106287 shutdown_connections 2024-05-17T22:02:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/604106287 wait complete. 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 Processor -- start 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- start start 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf24ac0 0x56376bf97a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bfadf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf99460 0x56376bfae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56376bf95cb0 con 0x56376bf25100 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56376bf95a10 con 0x56376bf99460 2024-05-17T22:02:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6d4ae080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56376bf95f50 con 0x56376bf24ac0 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bfadf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bfadf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47390/0 (socket says 172.21.15.83:47390) 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 -- 172.21.15.83:0/3877949647 learned_addr learned my addr 172.21.15.83:0/3877949647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f66f90640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf24ac0 0x56376bf97a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f67791640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf99460 0x56376bfae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 -- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf24ac0 msgr2=0x56376bf97a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56376bf24ac0 0x56376bf97a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 -- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf99460 msgr2=0x56376bfae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56376bf99460 0x56376bfae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 -- 172.21.15.83:0/3877949647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376bfaf200 con 0x56376bf25100 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.043+0000 7f8f6678f640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bfadf90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000f70 tx=0x7f8f5c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 -- 172.21.15.83:0/3877949647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01b850 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 -- 172.21.15.83:0/3877949647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c020030 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 -- 172.21.15.83:0/3877949647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01bc30 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56376bfaf4a0 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376c042100 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 -- 172.21.15.83:0/3877949647 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8f5c017040 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f5406de10 0x7f8f54070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f4ffff640 1 -- 172.21.15.83:0/3877949647 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f8f5c090130 con 0x56376bf25100 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f66f90640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f5406de10 0x7f8f54070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x56376c042d50 0x56376c043160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) -- 0x56376c0436a0 con 0x56376c042d50 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f66f90640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f5406de10 0x7f8f54070320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f600ad000 tx=0x7f8f600a5070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f67791640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x56376c042d50 0x56376c043160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.047+0000 7f8f67791640 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x56376c042d50 0x56376c043160 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f67791640 1 -- 172.21.15.83:0/3877949647 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v70'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8f580097a0 con 0x56376c042d50 2024-05-17T22:02:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x56376c042d50 msgr2=0x56376c043160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x56376c042d50 0x56376c043160 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f5406de10 msgr2=0x7f8f54070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f5406de10 0x7f8f54070320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8f600ad000 tx=0x7f8f600a5070 comp rx=0 tx=0).stop 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 msgr2=0x56376bfadf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 --2- 172.21.15.83:0/3877949647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56376bf25100 0x56376bfadf90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c000f70 tx=0x7f8f5c000ee0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f67791640 1 -- 172.21.15.83:0/3877949647 reap_dead start 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 shutdown_connections 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 >> 172.21.15.83:0/3877949647 conn(0x56376bec6e40 msgr2=0x56376bf989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 shutdown_connections 2024-05-17T22:02:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.051+0000 7f8f6d4ae080 1 -- 172.21.15.83:0/3877949647 wait complete. 2024-05-17T22:02:40.049 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:02:40.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:02:40.098 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:02:40.154 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:02:40.154 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:02:40.154 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:02:40.155 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:40.160 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:02:40.160 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:40.211 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:02:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.423+0000 7faaa6d26640 1 Processor -- start 2024-05-17T22:02:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.423+0000 7faaa6d26640 1 -- start start 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa6d26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 0x7faaa015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa6d26640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faaa00b44c0 0x7faaa00b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa6d26640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa006f2d0 con 0x7faaa006eec0 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa6d26640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faaa015c3b0 con 0x7faaa00b44c0 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa6d26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faaa006f5a0 con 0x7faaa00b08a0 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa4ac2640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faaa006eec0 0x7faaa01601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59822/0 (socket says 172.21.15.83:59822) 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa4ac2640 1 -- 172.21.15.83:0/2207714425 learned_addr learned my addr 172.21.15.83:0/2207714425 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 --2- 172.21.15.83:0/2207714425 >> v2:172.21.15.83:3300/0 conn(0x7faaa00b44c0 0x7faaa00b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faa9ffff640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 0x7faaa015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faa9f7fe640 1 -- 172.21.15.83:0/2207714425 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020685723 0 0) 0x7faaa006f2d0 con 0x7faaa006eec0 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faa9f7fe640 1 -- 172.21.15.83:0/2207714425 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa90003530 con 0x7faaa006eec0 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 -- 172.21.15.83:0/2207714425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 msgr2=0x7faaa015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 0x7faaa015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 -- 172.21.15.83:0/2207714425 >> v1:172.21.15.83:6789/0 conn(0x7faaa006eec0 legacy=0x7faaa01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 -- 172.21.15.83:0/2207714425 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa006f880 con 0x7faaa00b44c0 2024-05-17T22:02:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faa9ffff640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 0x7faaa015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faaa52c3640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa00b48d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7faa98009d90 tx=0x7faa9801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6547f8411b0c415 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.427+0000 7faa9f7fe640 1 -- 172.21.15.83:0/2207714425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980b2070 con 0x7faaa00b44c0 2024-05-17T22:02:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faa9f7fe640 1 -- 172.21.15.83:0/2207714425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa980a58a0 con 0x7faaa00b44c0 2024-05-17T22:02:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faa9f7fe640 1 -- 172.21.15.83:0/2207714425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980a5c50 con 0x7faaa00b44c0 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- 172.21.15.83:0/2207714425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 msgr2=0x7faaa00b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa00b48d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7faa98009d90 tx=0x7faa9801c970 comp rx=0 tx=0).stop 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- 172.21.15.83:0/2207714425 shutdown_connections 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa00b48d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 --2- 172.21.15.83:0/2207714425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa00b08a0 0x7faaa015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- 172.21.15.83:0/2207714425 >> 172.21.15.83:0/2207714425 conn(0x7faaa006cd90 msgr2=0x7faaa00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- 172.21.15.83:0/2207714425 shutdown_connections 2024-05-17T22:02:40.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- 172.21.15.83:0/2207714425 wait complete. 2024-05-17T22:02:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 Processor -- start 2024-05-17T22:02:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.431+0000 7faaa6d26640 1 -- start start 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 0x7faaa0153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa00b08a0 0x7faaa0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0160a40 con 0x7faaa00b44c0 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01608c0 con 0x7faaa00b08a0 2024-05-17T22:02:40.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faaa0072e80 con 0x7faaa006eec0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47410/0 (socket says 172.21.15.83:47410) 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 -- 172.21.15.83:0/177527506 learned_addr learned my addr 172.21.15.83:0/177527506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa4ac2640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 0x7faaa0153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 -- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 msgr2=0x7faaa0153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9ffff640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa00b08a0 0x7faaa0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 0x7faaa0153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 -- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa00b08a0 msgr2=0x7faaa0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa00b08a0 0x7faaa0156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 -- 172.21.15.83:0/177527506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa0158cf0 con 0x7faaa00b44c0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa4ac2640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 0x7faaa0153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa52c3640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faa980b0040 tx=0x7faa98019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980b2050 con 0x7faaa00b44c0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa980b3040 con 0x7faaa00b44c0 2024-05-17T22:02:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa0158fd0 con 0x7faaa00b44c0 2024-05-17T22:02:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98003bb0 con 0x7faaa00b44c0 2024-05-17T22:02:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa0155c20 con 0x7faaa00b44c0 2024-05-17T22:02:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa015be70 con 0x7faaa00b44c0 2024-05-17T22:02:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa980c0020 con 0x7faaa00b44c0 2024-05-17T22:02:40.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 0x7faa84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7faa98129b30 con 0x7faaa00b44c0 2024-05-17T22:02:40.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.435+0000 7faaa4ac2640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 0x7faa84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.439+0000 7faaa4ac2640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 0x7faa84071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faa94001230 tx=0x7faa94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.439+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7faa98016560 con 0x7faaa00b44c0 2024-05-17T22:02:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.607+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faaa00b0f10 con 0x7faaa00b44c0 2024-05-17T22:02:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faa9d7fa640 1 -- 172.21.15.83:0/177527506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) ==== 80+0+122 (secure 0 0 0) 0x7faa980f6a20 con 0x7faaa00b44c0 2024-05-17T22:02:40.602 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:02:40.602 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/1850779527 2024-05-18T22:01:18.280208+0000 2024-05-17T22:02:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:02:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 msgr2=0x7faa84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 0x7faa84071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faa94001230 tx=0x7faa94009040 comp rx=0 tx=0).stop 2024-05-17T22:02:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 msgr2=0x7faaa0156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faa980b0040 tx=0x7faa98019710 comp rx=0 tx=0).stop 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 shutdown_connections 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaa00b44c0 0x7faaa0156960 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faa8406f400 0x7faa84071910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa00b08a0 0x7faaa0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 --2- 172.21.15.83:0/177527506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faaa006eec0 0x7faaa0153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 >> 172.21.15.83:0/177527506 conn(0x7faaa006cd90 msgr2=0x7faaa00b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 shutdown_connections 2024-05-17T22:02:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.611+0000 7faaa6d26640 1 -- 172.21.15.83:0/177527506 wait complete. 2024-05-17T22:02:40.621 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:02:40.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:40.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:02:40.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[42313]: fuse finished with error 0 and tester_r 0 2024-05-17T22:02:40.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:02:40.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:02:40.631 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:02:40.760 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:02:40.760 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:02:40.760 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:40.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:02:40.767 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:02:40.767 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:02:40.767 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:40.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:02:40.773 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:02:40.773 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:02:40.773 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:40.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:02:40.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.967+0000 7fe94fd46640 1 Processor -- start 2024-05-17T22:02:40.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 -- start start 2024-05-17T22:02:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe94815fe20 0x7fe948160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 0x7fe948072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe94815c830 con 0x7fe94815d2f0 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe9481608a0 con 0x7fe94815fe20 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94fd46640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe948160b80 con 0x7fe9480b39e0 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe94815fe20 0x7fe948160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe94815fe20 0x7fe948160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47418/0 (socket says 172.21.15.83:47418) 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 -- 172.21.15.83:0/1476433656 learned_addr learned my addr 172.21.15.83:0/1476433656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94e2e3640 1 --2- 172.21.15.83:0/1476433656 >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 0x7fe948072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 -- 172.21.15.83:0/1476433656 >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 msgr2=0x7fe948072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 --2- 172.21.15.83:0/1476433656 >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 0x7fe948072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 -- 172.21.15.83:0/1476433656 >> v1:172.21.15.155:6789/0 conn(0x7fe94815d2f0 legacy=0x7fe948159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94d2e1640 1 -- 172.21.15.83:0/1476433656 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94815caf0 con 0x7fe94815fe20 2024-05-17T22:02:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.971+0000 7fe94e2e3640 1 --2- 172.21.15.83:0/1476433656 >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 0x7fe948072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94d2e1640 1 --2- 172.21.15.83:0/1476433656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe948160230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe94806d630 tx=0x7fe93801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dba995af498e11 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94cae0640 1 -- 172.21.15.83:0/1476433656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9380b2070 con 0x7fe94815fe20 2024-05-17T22:02:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94cae0640 1 -- 172.21.15.83:0/1476433656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9380a9070 con 0x7fe94815fe20 2024-05-17T22:02:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94cae0640 1 -- 172.21.15.83:0/1476433656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9380ad880 con 0x7fe94815fe20 2024-05-17T22:02:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- 172.21.15.83:0/1476433656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 msgr2=0x7fe948160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 --2- 172.21.15.83:0/1476433656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe948160230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe94806d630 tx=0x7fe93801c490 comp rx=0 tx=0).stop 2024-05-17T22:02:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- 172.21.15.83:0/1476433656 shutdown_connections 2024-05-17T22:02:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 --2- 172.21.15.83:0/1476433656 >> v2:172.21.15.155:3300/0 conn(0x7fe9480b39e0 0x7fe948072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 --2- 172.21.15.83:0/1476433656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe948160230 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- 172.21.15.83:0/1476433656 >> 172.21.15.83:0/1476433656 conn(0x7fe94806cd90 msgr2=0x7fe9480b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- 172.21.15.83:0/1476433656 shutdown_connections 2024-05-17T22:02:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- 172.21.15.83:0/1476433656 wait complete. 2024-05-17T22:02:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 Processor -- start 2024-05-17T22:02:40.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.975+0000 7fe94fd46640 1 -- start start 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9480b39e0 0x7fe94814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94815d2f0 0x7fe94815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480b0f10 con 0x7fe94815fe20 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480b0d90 con 0x7fe94815d2f0 2024-05-17T22:02:40.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480b1090 con 0x7fe9480b39e0 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47424/0 (socket says 172.21.15.83:47424) 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 -- 172.21.15.83:0/2259654449 learned_addr learned my addr 172.21.15.83:0/2259654449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94d2e1640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94815d2f0 0x7fe94815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 -- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9480b39e0 msgr2=0x7fe94814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94dae2640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9480b39e0 0x7fe94814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9480b39e0 0x7fe94814cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 -- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94815d2f0 msgr2=0x7fe94815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94815d2f0 0x7fe94815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 -- 172.21.15.83:0/2259654449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94815f330 con 0x7fe94815fe20 2024-05-17T22:02:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94e2e3640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe94000cd80 tx=0x7fe94000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94000f680 con 0x7fe94815fe20 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94000f840 con 0x7fe94815fe20 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94815f610 con 0x7fe94815fe20 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94000fc30 con 0x7fe94815fe20 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481501f0 con 0x7fe94815fe20 2024-05-17T22:02:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94814fd00 con 0x7fe94815fe20 2024-05-17T22:02:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.979+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe9400213f0 con 0x7fe94815fe20 2024-05-17T22:02:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.983+0000 7fe93e7fc640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 0x7fe92c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.983+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fe9400900a0 con 0x7fe94815fe20 2024-05-17T22:02:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.983+0000 7fe94dae2640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 0x7fe92c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.983+0000 7fe94dae2640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 0x7fe92c071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe94400a750 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:40.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:40.987+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe940014030 con 0x7fe94815fe20 2024-05-17T22:02:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.167+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe948072e30 con 0x7fe94815fe20 2024-05-17T22:02:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.167+0000 7fe93e7fc640 1 -- 172.21.15.83:0/2259654449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5188 (secure 0 0 0) 0x7fe94005cf90 con 0x7fe94815fe20 2024-05-17T22:02:41.160 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:41.161 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:02:09.307460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 msgr2=0x7fe92c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 0x7fe92c071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe94400a750 tx=0x7fe944008040 comp rx=0 tx=0).stop 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 msgr2=0x7fe94815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe94000cd80 tx=0x7fe94000a650 comp rx=0 tx=0).stop 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 shutdown_connections 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe94815fe20 0x7fe94815e340 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe92c06f400 0x7fe92c071910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe94815d2f0 0x7fe94815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 --2- 172.21.15.83:0/2259654449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9480b39e0 0x7fe94814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 >> 172.21.15.83:0/2259654449 conn(0x7fe94806cd90 msgr2=0x7fe9480b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 shutdown_connections 2024-05-17T22:02:41.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.171+0000 7fe94fd46640 1 -- 172.21.15.83:0/2259654449 wait complete. 2024-05-17T22:02:41.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.323+0000 7fd838772640 1 Processor -- start 2024-05-17T22:02:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.323+0000 7fd838772640 1 -- start start 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd838772640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd838772640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd83015e6b0 0x7fd83006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd838772640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8301465d0 con 0x7fd830070510 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd838772640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd83006ef30 con 0x7fd83015e6b0 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd838772640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd83006f210 con 0x7fd8301461c0 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd835d0d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd83015e6b0 0x7fd83006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:45034/0 (socket says 172.21.15.83:45034) 2024-05-17T22:02:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 -- 172.21.15.83:0/3554550095 learned_addr learned my addr 172.21.15.83:0/3554550095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 -- 172.21.15.83:0/3554550095 >> v2:172.21.15.155:3300/0 conn(0x7fd83015e6b0 msgr2=0x7fd83006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 --2- 172.21.15.83:0/3554550095 >> v2:172.21.15.155:3300/0 conn(0x7fd83015e6b0 0x7fd83006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 -- 172.21.15.83:0/3554550095 >> v1:172.21.15.155:6789/0 conn(0x7fd830070510 legacy=0x7fd8301605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.327+0000 7fd83650e640 1 -- 172.21.15.83:0/3554550095 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830070920 con 0x7fd8301461c0 2024-05-17T22:02:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd83650e640 1 --2- 172.21.15.83:0/3554550095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd820009870 tx=0x7fd82001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b4dfb3b2c3279f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd83550c640 1 -- 172.21.15.83:0/3554550095 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200a95f0 con 0x7fd8301461c0 2024-05-17T22:02:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd83550c640 1 -- 172.21.15.83:0/3554550095 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200a9790 con 0x7fd8301461c0 2024-05-17T22:02:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd83550c640 1 -- 172.21.15.83:0/3554550095 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200a9b60 con 0x7fd8301461c0 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- 172.21.15.83:0/3554550095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 msgr2=0x7fd83015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 --2- 172.21.15.83:0/3554550095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd820009870 tx=0x7fd82001c530 comp rx=0 tx=0).stop 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- 172.21.15.83:0/3554550095 shutdown_connections 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 --2- 172.21.15.83:0/3554550095 >> v2:172.21.15.155:3300/0 conn(0x7fd83015e6b0 0x7fd83006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 --2- 172.21.15.83:0/3554550095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83015e170 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- 172.21.15.83:0/3554550095 >> 172.21.15.83:0/3554550095 conn(0x7fd83006cd90 msgr2=0x7fd8300b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- 172.21.15.83:0/3554550095 shutdown_connections 2024-05-17T22:02:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- 172.21.15.83:0/3554550095 wait complete. 2024-05-17T22:02:41.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 Processor -- start 2024-05-17T22:02:41.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.331+0000 7fd838772640 1 -- start start 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83013f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd83015e6b0 0x7fd83013f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd830160e30 con 0x7fd830070510 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd830160cb0 con 0x7fd83015e6b0 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd8300d10a0 con 0x7fd8301461c0 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47438/0 (socket says 172.21.15.83:47438) 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 -- 172.21.15.83:0/1627818270 learned_addr learned my addr 172.21.15.83:0/1627818270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd835d0d640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd836d0f640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd83015e6b0 0x7fd83013f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 -- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 msgr2=0x7fd83013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 -- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd83015e6b0 msgr2=0x7fd83013f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd83015e6b0 0x7fd83013f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 -- 172.21.15.83:0/1627818270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301409f0 con 0x7fd830070510 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd83650e640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd82001cdd0 tx=0x7fd8200192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200bb600 con 0x7fd830070510 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200bb7a0 con 0x7fd830070510 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd830140cd0 con 0x7fd830070510 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200bbb50 con 0x7fd830070510 2024-05-17T22:02:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd830141290 con 0x7fd830070510 2024-05-17T22:02:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83015e170 con 0x7fd830070510 2024-05-17T22:02:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd8200b13f0 con 0x7fd830070510 2024-05-17T22:02:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 0x7fd814071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fd8201297f0 con 0x7fd830070510 2024-05-17T22:02:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.335+0000 7fd835d0d640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 0x7fd814071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.339+0000 7fd835d0d640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 0x7fd814071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd8301448b0 tx=0x7fd82c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.343+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd8200c2070 con 0x7fd830070510 2024-05-17T22:02:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.523+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd830143fd0 con 0x7fd830070510 2024-05-17T22:02:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.523+0000 7fd826ffd640 1 -- 172.21.15.83:0/1627818270 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5188 (secure 0 0 0) 0x7fd8200ad020 con 0x7fd830070510 2024-05-17T22:02:41.515 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:41.515 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:02:09.307460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-05-17T22:02:41.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 msgr2=0x7fd814071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 0x7fd814071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd8301448b0 tx=0x7fd82c008040 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 msgr2=0x7fd830142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd82001cdd0 tx=0x7fd8200192d0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 shutdown_connections 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd81406f400 0x7fd814071910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd83015e6b0 0x7fd83013f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8301461c0 0x7fd83013f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 --2- 172.21.15.83:0/1627818270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd830070510 0x7fd830142c80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 >> 172.21.15.83:0/1627818270 conn(0x7fd83006cd90 msgr2=0x7fd8300b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 shutdown_connections 2024-05-17T22:02:41.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.527+0000 7fd838772640 1 -- 172.21.15.83:0/1627818270 wait complete. 2024-05-17T22:02:41.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:41.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 Processor -- start 2024-05-17T22:02:41.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 -- start start 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 0x7f75b80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b80d1150 0x7f75b806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b806e610 con 0x7f75b806e200 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75b80d34e0 con 0x7f75b80d1150 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75bd91c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f75b80d3780 con 0x7f75b815ffa0 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b80d1150 0x7f75b806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75b80d1150 0x7f75b806dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47450/0 (socket says 172.21.15.83:47450) 2024-05-17T22:02:41.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 -- 172.21.15.83:0/2503154845 learned_addr learned my addr 172.21.15.83:0/2503154845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b6ffd640 1 --2- 172.21.15.83:0/2503154845 >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 0x7f75b80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 -- 172.21.15.83:0/2503154845 >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 msgr2=0x7f75b80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 --2- 172.21.15.83:0/2503154845 >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 0x7f75b80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 -- 172.21.15.83:0/2503154845 >> v1:172.21.15.155:6789/0 conn(0x7f75b806e200 legacy=0x7f75b8163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b67fc640 1 -- 172.21.15.83:0/2503154845 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b80d3a60 con 0x7f75b80d1150 2024-05-17T22:02:41.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.691+0000 7f75b6ffd640 1 --2- 172.21.15.83:0/2503154845 >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 0x7f75b80d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75b67fc640 1 --2- 172.21.15.83:0/2503154845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80d1150 0x7f75b806dcc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f75b80b5260 tx=0x7f75ac01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e163646b06a1c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75b5ffb640 1 -- 172.21.15.83:0/2503154845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0b0070 con 0x7f75b80d1150 2024-05-17T22:02:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75b5ffb640 1 -- 172.21.15.83:0/2503154845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac0ab600 con 0x7f75b80d1150 2024-05-17T22:02:41.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75b5ffb640 1 -- 172.21.15.83:0/2503154845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0ab9d0 con 0x7f75b80d1150 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- 172.21.15.83:0/2503154845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80d1150 msgr2=0x7f75b806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 --2- 172.21.15.83:0/2503154845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80d1150 0x7f75b806dcc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f75b80b5260 tx=0x7f75ac01c340 comp rx=0 tx=0).stop 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- 172.21.15.83:0/2503154845 shutdown_connections 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 --2- 172.21.15.83:0/2503154845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80d1150 0x7f75b806dcc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 --2- 172.21.15.83:0/2503154845 >> v2:172.21.15.155:3300/0 conn(0x7f75b815ffa0 0x7f75b80d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- 172.21.15.83:0/2503154845 >> 172.21.15.83:0/2503154845 conn(0x7f75b80d28f0 msgr2=0x7f75b80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- 172.21.15.83:0/2503154845 shutdown_connections 2024-05-17T22:02:41.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- 172.21.15.83:0/2503154845 wait complete. 2024-05-17T22:02:41.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 Processor -- start 2024-05-17T22:02:41.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.695+0000 7f75bd91c640 1 -- start start 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b806e200 0x7f75b8146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b815ae00 0x7f75b80cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75b816cd00 con 0x7f75b815ffa0 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75b816cb80 con 0x7f75b815ae00 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75b816ce80 con 0x7f75b806e200 2024-05-17T22:02:41.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47464/0 (socket says 172.21.15.83:47464) 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 -- 172.21.15.83:0/3493207560 learned_addr learned my addr 172.21.15.83:0/3493207560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b6ffd640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b806e200 0x7f75b8146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b77fe640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b815ae00 0x7f75b80cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 -- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b806e200 msgr2=0x7f75b8146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b806e200 0x7f75b8146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 -- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b815ae00 msgr2=0x7f75b80cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b815ae00 0x7f75b80cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 -- 172.21.15.83:0/3493207560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b806e9a0 con 0x7f75b815ffa0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b67fc640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75ac01cbe0 tx=0x7f75ac0190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0bb070 con 0x7f75b815ffa0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac004030 con 0x7f75b815ffa0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b815f620 con 0x7f75b815ffa0 2024-05-17T22:02:41.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0b0040 con 0x7f75b815ffa0 2024-05-17T22:02:41.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b80ccda0 con 0x7f75b815ffa0 2024-05-17T22:02:41.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b806dcc0 con 0x7f75b815ffa0 2024-05-17T22:02:41.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75ac003620 con 0x7f75b815ffa0 2024-05-17T22:02:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 0x7f75a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f75ac128d70 con 0x7f75b815ffa0 2024-05-17T22:02:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.699+0000 7f75b6ffd640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 0x7f75a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:41.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.703+0000 7f75b6ffd640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 0x7f75a4071910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006fd0 tx=0x7f75a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:41.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.707+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f75b806e610 con 0x7f75b815ffa0 2024-05-17T22:02:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.855+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b80ccf70 con 0x7f75b815ffa0 2024-05-17T22:02:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.855+0000 7f759ffff640 1 -- 172.21.15.83:0/3493207560 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16871 (secure 0 0 0) 0x7f75ac0f5b50 con 0x7f75b815ffa0 2024-05-17T22:02:41.849 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:41.849 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:01:18.280270+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:00:58.412570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-17T22:00:59.672732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1850779527":"2024-05-18T22:01:18.280208+0000","172.21.15.83:6836/1850779527":"2024-05-18T22:01:18.280208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 msgr2=0x7f75a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 0x7f75a4071910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006fd0 tx=0x7f75a8008040 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 msgr2=0x7f75b8159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f75ac01cbe0 tx=0x7f75ac0190e0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 shutdown_connections 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f75a406f400 0x7f75a4071910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b815ae00 0x7f75b80cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b815ffa0 0x7f75b8159fc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 --2- 172.21.15.83:0/3493207560 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b806e200 0x7f75b8146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 >> 172.21.15.83:0/3493207560 conn(0x7f75b80d28f0 msgr2=0x7f75b80bb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 shutdown_connections 2024-05-17T22:02:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:41.859+0000 7f75bd91c640 1 -- 172.21.15.83:0/3493207560 wait complete. 2024-05-17T22:02:41.869 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:02:41.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 Processor -- start 2024-05-17T22:02:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 -- start start 2024-05-17T22:02:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 0x7f04180b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04180b5f30 con 0x7f041806f960 2024-05-17T22:02:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041806d410 con 0x7f041806c790 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041f984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f041806e020 con 0x7f04180b55e0 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041df21640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f041806f960 0x7f041806fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59824/0 (socket says 172.21.15.83:59824) 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041df21640 1 -- 172.21.15.83:0/1812283213 learned_addr learned my addr 172.21.15.83:0/1812283213 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041d720640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 0x7f04180b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278578216 0 0) 0x7f04080014a0 con 0x7f041806f960 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0400003530 con 0x7f041806f960 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2041955621 0 0) 0x7f0400003530 con 0x7f041806f960 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 msgr2=0x7f04180b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 0x7f04180b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 >> v1:172.21.15.155:6789/0 conn(0x7f041806c790 legacy=0x7f041806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041806d6f0 con 0x7f041806f960 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f0408001c50 con 0x7f041806f960 2024-05-17T22:02:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 >> v1:172.21.15.83:6789/0 conn(0x7f041806f960 legacy=0x7f041806fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400003a40 0x7f0400003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004810 0x7f0400004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 0x7f04000052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f041806e020 con 0x7f0400003a40 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f040000bb90 con 0x7f0400004810 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f040000bd90 con 0x7f04180b55e0 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.011+0000 7f041d720640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400003a40 0x7f0400003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041cf1f640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004810 0x7f0400004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 -- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 msgr2=0x7f04000052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04180b55e0 0x7f04000052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 -- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004810 msgr2=0x7f0400004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004810 0x7f0400004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 -- 172.21.15.83:0/1812283213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0400003530 con 0x7f0400003a40 2024-05-17T22:02:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041cf1f640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004810 0x7f0400004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041d720640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400003a40 0x7f0400003eb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ae0 tx=0x7f041001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627570d88eb31552 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100096b0 con 0x7f0400003a40 2024-05-17T22:02:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04100a6490 con 0x7f0400003a40 2024-05-17T22:02:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f0417fff640 1 -- 172.21.15.83:0/1812283213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418070c90 con 0x7f0400003a40 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041f984640 1 -- 172.21.15.83:0/1812283213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400003a40 msgr2=0x7f0400003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041f984640 1 --2- 172.21.15.83:0/1812283213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400003a40 0x7f0400003eb0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ae0 tx=0x7f041001cab0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041df21640 1 -- 172.21.15.83:0/1812283213 reap_dead start 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041f984640 1 -- 172.21.15.83:0/1812283213 shutdown_connections 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041f984640 1 -- 172.21.15.83:0/1812283213 >> 172.21.15.83:0/1812283213 conn(0x7f04180d1ee0 msgr2=0x7f04180d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.015+0000 7f041f984640 1 -- 172.21.15.83:0/1812283213 shutdown_connections 2024-05-17T22:02:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 -- 172.21.15.83:0/1812283213 wait complete. 2024-05-17T22:02:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 Processor -- start 2024-05-17T22:02:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 -- start start 2024-05-17T22:02:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 0x7f0418143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0418159010 0x7f0418159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f041816b8c0 con 0x7f0418143ae0 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0418161120 con 0x7f0418159010 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041f984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f041816ba40 con 0x7f0400003a40 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47474/0 (socket says 172.21.15.83:47474) 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 -- 172.21.15.83:0/3197690175 learned_addr learned my addr 172.21.15.83:0/3197690175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041d720640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 0x7f0418143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041df21640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0418159010 0x7f0418159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 -- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 msgr2=0x7f0418143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 0x7f0418143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 -- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0418159010 msgr2=0x7f0418159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0418159010 0x7f0418159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 -- 172.21.15.83:0/3197690175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418157a30 con 0x7f0418143ae0 2024-05-17T22:02:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041d720640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 0x7f0418143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f041cf1f640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f04181647f0 tx=0x7f040c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.019+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c005280 con 0x7f0418143ae0 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c005420 con 0x7f0418143ae0 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418157d10 con 0x7f0418143ae0 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c006a20 con 0x7f0418143ae0 2024-05-17T22:02:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181582d0 con 0x7f0418143ae0 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041806cba0 con 0x7f0418143ae0 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f040c01a020 con 0x7f0418143ae0 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f0415ffb640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 0x7f0404071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f040c08ea50 con 0x7f0418143ae0 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f041d720640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 0x7f0404071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.023+0000 7f041d720640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 0x7f0404071910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04100ad000 tx=0x7f04100046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.027+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f040c05b940 con 0x7f0418143ae0 2024-05-17T22:02:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.211+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f041806cff0 con 0x7f0418143ae0 2024-05-17T22:02:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.211+0000 7f0415ffb640 1 -- 172.21.15.83:0/3197690175 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5188 (secure 0 0 0) 0x7f040c05b240 con 0x7f0418143ae0 2024-05-17T22:02:42.203 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:42.204 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:02:09.307460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 msgr2=0x7f0404071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 0x7f0404071910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04100ad000 tx=0x7f04100046d0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 msgr2=0x7f0418158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f04181647f0 tx=0x7f040c00d9d0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 shutdown_connections 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040406f400 0x7f0404071910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0418159010 0x7f0418159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0418143ae0 0x7f0418158ad0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 --2- 172.21.15.83:0/3197690175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003a40 0x7f0418143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 >> 172.21.15.83:0/3197690175 conn(0x7f04180d1ee0 msgr2=0x7f0418161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 shutdown_connections 2024-05-17T22:02:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.215+0000 7f041f984640 1 -- 172.21.15.83:0/3197690175 wait complete. 2024-05-17T22:02:42.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 Processor -- start 2024-05-17T22:02:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 -- start start 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f0164400 0x7f42f00d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f42f00c5530 0x7f42f00c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00b27b0 0x7f42f00b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f42f00c5e80 con 0x7f42f0164400 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f42f00d0e90 con 0x7f42f00c5530 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42f5626640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f42f00d1170 con 0x7f42f00b27b0 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f0164400 0x7f42f00d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f42f0164400 0x7f42f00d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47486/0 (socket says 172.21.15.83:47486) 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 -- 172.21.15.83:0/1485551660 learned_addr learned my addr 172.21.15.83:0/1485551660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42ef7fe640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00b27b0 0x7f42f00b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42ee7fc640 1 --2- 172.21.15.83:0/1485551660 >> v2:172.21.15.155:3300/0 conn(0x7f42f00c5530 0x7f42f00c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 -- 172.21.15.83:0/1485551660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00b27b0 msgr2=0x7f42f00b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00b27b0 0x7f42f00b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 -- 172.21.15.83:0/1485551660 >> v2:172.21.15.155:3300/0 conn(0x7f42f00c5530 msgr2=0x7f42f00c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 --2- 172.21.15.83:0/1485551660 >> v2:172.21.15.155:3300/0 conn(0x7f42f00c5530 0x7f42f00c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.367+0000 7f42eeffd640 1 -- 172.21.15.83:0/1485551660 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f00d1450 con 0x7f42f0164400 2024-05-17T22:02:42.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42eeffd640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00d1c50 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f42f00bccf0 tx=0x7f42e801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a91def7d343b898 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42edffb640 1 -- 172.21.15.83:0/1485551660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e80a5780 con 0x7f42f0164400 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42edffb640 1 -- 172.21.15.83:0/1485551660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e80a5920 con 0x7f42f0164400 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42edffb640 1 -- 172.21.15.83:0/1485551660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e80a5cf0 con 0x7f42f0164400 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 -- 172.21.15.83:0/1485551660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 msgr2=0x7f42f00d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00d1c50 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f42f00bccf0 tx=0x7f42e801c9f0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 -- 172.21.15.83:0/1485551660 shutdown_connections 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00b27b0 0x7f42f00b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 --2- 172.21.15.83:0/1485551660 >> v2:172.21.15.155:3300/0 conn(0x7f42f00c5530 0x7f42f00c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 --2- 172.21.15.83:0/1485551660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00d1c50 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 -- 172.21.15.83:0/1485551660 >> 172.21.15.83:0/1485551660 conn(0x7f42f00b4090 msgr2=0x7f42f00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 -- 172.21.15.83:0/1485551660 shutdown_connections 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.371+0000 7f42f5626640 1 -- 172.21.15.83:0/1485551660 wait complete. 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 Processor -- start 2024-05-17T22:02:42.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- start start 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f00b27b0 0x7f42f01672f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00c5530 0x7f42f00b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42f00aff10 con 0x7f42f0164400 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42f00afd90 con 0x7f42f00b27b0 2024-05-17T22:02:42.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f42f0154a10 con 0x7f42f00c5530 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47492/0 (socket says 172.21.15.83:47492) 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 -- 172.21.15.83:0/4157659625 learned_addr learned my addr 172.21.15.83:0/4157659625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42eeffd640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f00b27b0 0x7f42f01672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ee7fc640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00c5530 0x7f42f00b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 -- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00c5530 msgr2=0x7f42f00b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00c5530 0x7f42f00b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 -- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f00b27b0 msgr2=0x7f42f01672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f00b27b0 0x7f42f01672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 -- 172.21.15.83:0/4157659625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f0160210 con 0x7f42f0164400 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42ef7fe640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c4d0 tx=0x7f42e000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0015070 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e0004240 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f01604f0 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0003620 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f00c65b0 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.375+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f00b5fd0 con 0x7f42f0164400 2024-05-17T22:02:42.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.379+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f42e00037c0 con 0x7f42f0164400 2024-05-17T22:02:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.379+0000 7f42e77fe640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 0x7f42d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.379+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7f42e008f260 con 0x7f42f0164400 2024-05-17T22:02:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.379+0000 7f42eeffd640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 0x7f42d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.379+0000 7f42eeffd640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 0x7f42d0071910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f42e80ae000 tx=0x7f42e80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.383+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f42e005c2c0 con 0x7f42f0164400 2024-05-17T22:02:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42f00c6810 con 0x7f42f0164400 2024-05-17T22:02:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42e77fe640 1 -- 172.21.15.83:0/4157659625 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5188 (secure 0 0 0) 0x7f42e00642b0 con 0x7f42f0164400 2024-05-17T22:02:42.570 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:42.570 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:01:00.724476+0000","modified":"2024-05-17T22:02:09.307460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6234":{"gid":6234,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":16,"addr":"172.21.15.83:6837/1116165089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1116165089},{"type":"v1","addr":"172.21.15.83:6837","nonce":1116165089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6234,"qdb_cluster":[6234]},"id":5}]} 2024-05-17T22:02:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-05-17T22:02:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 msgr2=0x7f42d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 0x7f42d0071910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f42e80ae000 tx=0x7f42e80a7070 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 msgr2=0x7f42f00b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f42e000c4d0 tx=0x7f42e000c990 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 shutdown_connections 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42f0164400 0x7f42f00b6c00 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42f00c5530 0x7f42f00b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f42d006f400 0x7f42d0071910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 --2- 172.21.15.83:0/4157659625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f00b27b0 0x7f42f01672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.575+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 >> 172.21.15.83:0/4157659625 conn(0x7f42f00b4090 msgr2=0x7f42f006c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.579+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 shutdown_connections 2024-05-17T22:02:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.579+0000 7f42f5626640 1 -- 172.21.15.83:0/4157659625 wait complete. 2024-05-17T22:02:42.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:42.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.727+0000 7fecc50f3640 1 Processor -- start 2024-05-17T22:02:42.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.727+0000 7fecc50f3640 1 -- start start 2024-05-17T22:02:42.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecc50f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 0x7fecc015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecc50f3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecc015e6b0 0x7fecc006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecc50f3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc01465d0 con 0x7fecc0070510 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecc50f3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fecc006ef30 con 0x7fecc015e6b0 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecc50f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fecc006f210 con 0x7fecc01461c0 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecc015e6b0 0x7fecc006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbf59e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fecc0070510 0x7fecc01605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59836/0 (socket says 172.21.15.83:59836) 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecc015e6b0 0x7fecc006ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47508/0 (socket says 172.21.15.83:47508) 2024-05-17T22:02:42.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbf59e640 1 -- 172.21.15.83:0/286293449 learned_addr learned my addr 172.21.15.83:0/286293449 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbed9d640 1 --2- 172.21.15.83:0/286293449 >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 0x7fecc015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 -- 172.21.15.83:0/286293449 >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 msgr2=0x7fecc015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 --2- 172.21.15.83:0/286293449 >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 0x7fecc015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 -- 172.21.15.83:0/286293449 >> v1:172.21.15.83:6789/0 conn(0x7fecc0070510 legacy=0x7fecc01605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 -- 172.21.15.83:0/286293449 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc0070920 con 0x7fecc015e6b0 2024-05-17T22:02:42.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbed9d640 1 --2- 172.21.15.83:0/286293449 >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 0x7fecc015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.731+0000 7fecbe59c640 1 --2- 172.21.15.83:0/286293449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc006ffd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fecc006d5a0 tx=0x7fecb401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f80bc349ba8140 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecbdd9b640 1 -- 172.21.15.83:0/286293449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40ab650 con 0x7fecc015e6b0 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecbdd9b640 1 -- 172.21.15.83:0/286293449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb40b0070 con 0x7fecc015e6b0 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecbdd9b640 1 -- 172.21.15.83:0/286293449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40aba10 con 0x7fecc015e6b0 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- 172.21.15.83:0/286293449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 msgr2=0x7fecc006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- 172.21.15.83:0/286293449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc006ffd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fecc006d5a0 tx=0x7fecb401c450 comp rx=0 tx=0).stop 2024-05-17T22:02:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- 172.21.15.83:0/286293449 shutdown_connections 2024-05-17T22:02:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- 172.21.15.83:0/286293449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc006ffd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- 172.21.15.83:0/286293449 >> v2:172.21.15.155:3300/0 conn(0x7fecc01461c0 0x7fecc015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- 172.21.15.83:0/286293449 >> 172.21.15.83:0/286293449 conn(0x7fecc006cd90 msgr2=0x7fecc00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- 172.21.15.83:0/286293449 shutdown_connections 2024-05-17T22:02:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- 172.21.15.83:0/286293449 wait complete. 2024-05-17T22:02:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 Processor -- start 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- start start 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecc0070510 0x7fecc0158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 0x7fecc0144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0160e30 con 0x7fecc015e6b0 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0160cb0 con 0x7fecc0070510 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.735+0000 7fecc50f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fecc00d10a0 con 0x7fecc01461c0 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47520/0 (socket says 172.21.15.83:47520) 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 -- 172.21.15.83:0/2186926470 learned_addr learned my addr 172.21.15.83:0/2186926470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbe59c640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 0x7fecc0144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbed9d640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecc0070510 0x7fecc0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 -- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 msgr2=0x7fecc0144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 0x7fecc0144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 -- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecc0070510 msgr2=0x7fecc0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecc0070510 0x7fecc0158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 -- 172.21.15.83:0/2186926470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc00b2710 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbe59c640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 0x7fecc0144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbf59e640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fecb800caa0 tx=0x7fecb800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8017070 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb800ac20 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc00b29f0 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8012400 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc00b2fb0 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc015e170 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fecb801a070 con 0x7fecc015e6b0 2024-05-17T22:02:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 0x7fecac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbed9d640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 0x7fecac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7fecb808f090 con 0x7fecc015e6b0 2024-05-17T22:02:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.739+0000 7fecbed9d640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 0x7fecac071910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006fd0 tx=0x7fecb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.743+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fecb805b800 con 0x7fecc015e6b0 2024-05-17T22:02:42.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.891+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecc015a9d0 con 0x7fecc015e6b0 2024-05-17T22:02:42.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7feca77fe640 1 -- 172.21.15.83:0/2186926470 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16871 (secure 0 0 0) 0x7fecb805fb00 con 0x7fecc015e6b0 2024-05-17T22:02:42.886 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:42.887 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:01:18.280270+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:00:58.412570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-17T22:00:59.672732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1850779527":"2024-05-18T22:01:18.280208+0000","172.21.15.83:6836/1850779527":"2024-05-18T22:01:18.280208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:42.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 msgr2=0x7fecac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 0x7fecac071910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006fd0 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 msgr2=0x7fecc0144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fecb800caa0 tx=0x7fecb800cf60 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 shutdown_connections 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc015e6b0 0x7fecc0144ab0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fecc01461c0 0x7fecc0144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fecac06f400 0x7fecac071910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 --2- 172.21.15.83:0/2186926470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecc0070510 0x7fecc0158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 >> 172.21.15.83:0/2186926470 conn(0x7fecc006cd90 msgr2=0x7fecc00d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 shutdown_connections 2024-05-17T22:02:42.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:42.895+0000 7fecc50f3640 1 -- 172.21.15.83:0/2186926470 wait complete. 2024-05-17T22:02:42.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:02:43.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.051+0000 7ff2e1bdf640 1 Processor -- start 2024-05-17T22:02:43.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.051+0000 7ff2e1bdf640 1 -- start start 2024-05-17T22:02:43.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2e1bdf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2e1bdf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2e1bdf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2dc0b5640 con 0x7ff2dc0cf830 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2e1bdf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc070870 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2e1bdf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc0b58e0 con 0x7ff2dc15d530 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2dbfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:45072/0 (socket says 172.21.15.83:45072) 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2dbfff640 1 -- 172.21.15.83:0/3430989004 learned_addr learned my addr 172.21.15.83:0/3430989004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:43.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/3430989004 >> v2:172.21.15.155:3300/0 conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:57658/0 (socket says 172.21.15.83:57658) 2024-05-17T22:02:43.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 -- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 msgr2=0x7ff2dc1663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:43.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 -- 172.21.15.83:0/3430989004 >> v1:172.21.15.155:6789/0 conn(0x7ff2dc0cf830 legacy=0x7ff2dc0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 -- 172.21.15.83:0/3430989004 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc06f060 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2dbfff640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:02:43.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097c0 tx=0x7ff2c401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db4f412af34548e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2daffd640 1 -- 172.21.15.83:0/3430989004 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b0070 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.055+0000 7ff2daffd640 1 -- 172.21.15.83:0/3430989004 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c40ab600 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2daffd640 1 -- 172.21.15.83:0/3430989004 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40ab9d0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cdf40 msgr2=0x7ff2dc15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097c0 tx=0x7ff2c401c4a0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/3430989004 shutdown_connections 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc1663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/3430989004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc15cff0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/3430989004 >> 172.21.15.83:0/3430989004 conn(0x7ff2dc06cd90 msgr2=0x7ff2dc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:43.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/3430989004 shutdown_connections 2024-05-17T22:02:43.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/3430989004 wait complete. 2024-05-17T22:02:43.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 Processor -- start 2024-05-17T22:02:43.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.059+0000 7ff2e1bdf640 1 -- start start 2024-05-17T22:02:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 0x7ff2dc14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc073a20 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0738a0 con 0x7ff2dc0cf830 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc073ba0 con 0x7ff2dc15d530 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47532/0 (socket says 172.21.15.83:47532) 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 -- 172.21.15.83:0/1605828756 learned_addr learned my addr 172.21.15.83:0/1605828756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2dbfff640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 0x7ff2dc14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 -- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 msgr2=0x7ff2dc152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 -- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 msgr2=0x7ff2dc14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 0x7ff2dc14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 -- 172.21.15.83:0/1605828756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc15e900 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 0x7ff2dc14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2db7fe640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c420 tx=0x7ff2cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0052d0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2cc005490 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc15ebe0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc007240 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc164d80 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc15ee00 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2cc0073e0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.063+0000 7ff2d8ff9640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.067+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5853+0+0 (secure 0 0 0) 0x7ff2cc08e6e0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.067+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.067+0000 7ff2d2ffd640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 0x7ff2bc071940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40ad000 tx=0x7ff2c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.071+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff2cc05b660 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.251+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff2dc15cff0 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.243 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:02:43.251+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:02:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.255+0000 7ff2d8ff9640 1 -- 172.21.15.83:0/1605828756 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7ff2cc001090 con 0x7ff2dc0cdf40 2024-05-17T22:02:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:02:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 msgr2=0x7ff2bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 0x7ff2bc071940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40ad000 tx=0x7ff2c4008040 comp rx=0 tx=0).stop 2024-05-17T22:02:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 msgr2=0x7ff2dc164740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00c420 tx=0x7ff2cc00c8e0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 shutdown_connections 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2dc15d530 0x7ff2dc152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff2bc06f430 0x7ff2bc071940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2dc0cf830 0x7ff2dc14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 --2- 172.21.15.83:0/1605828756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2dc0cdf40 0x7ff2dc164740 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 >> 172.21.15.83:0/1605828756 conn(0x7ff2dc06cd90 msgr2=0x7ff2dc0d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 shutdown_connections 2024-05-17T22:02:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.259+0000 7ff2e1bdf640 1 -- 172.21.15.83:0/1605828756 wait complete. 2024-05-17T22:02:43.259 INFO:tasks.ceph.mds.a.smithi083.stderr: -4396> 2024-05-17T22:01:24.295+0000 7f7d9e130640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:02:43.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:02:43.324 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:02:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 Processor -- start 2024-05-17T22:02:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 -- start start 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9a80b5e10 0x7fa9a80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80b3a30 con 0x7fa9a806d170 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a80b3d10 con 0x7fa9a80b5e10 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9b0121640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80b5610 con 0x7fa9a80ce920 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9ad6bc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9ae6be640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9a80b5e10 0x7fa9a80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9ad6bc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:45110/0 (socket says 172.21.15.83:45110) 2024-05-17T22:02:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.439+0000 7fa9ad6bc640 1 -- 172.21.15.83:0/4206191176 learned_addr learned my addr 172.21.15.83:0/4206191176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ae6be640 1 -- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 msgr2=0x7fa9a80b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ae6be640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ae6be640 1 -- 172.21.15.83:0/4206191176 >> v1:172.21.15.155:6789/0 conn(0x7fa9a806d170 legacy=0x7fa9a80ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ae6be640 1 -- 172.21.15.83:0/4206191176 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a80b58f0 con 0x7fa9a80b5e10 2024-05-17T22:02:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ad6bc640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:02:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9ae6be640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80b6220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009d90 tx=0x7fa9a001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e109cbe7102fdebe server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9acebb640 1 -- 172.21.15.83:0/4206191176 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00b2070 con 0x7fa9a80b5e10 2024-05-17T22:02:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9acebb640 1 -- 172.21.15.83:0/4206191176 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a00a5830 con 0x7fa9a80b5e10 2024-05-17T22:02:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9acebb640 1 -- 172.21.15.83:0/4206191176 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00a5c00 con 0x7fa9a80b5e10 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 -- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 msgr2=0x7fa9a80b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80b6220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009d90 tx=0x7fa9a001c970 comp rx=0 tx=0).stop 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 -- 172.21.15.83:0/4206191176 shutdown_connections 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80b6220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 --2- 172.21.15.83:0/4206191176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a80b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 -- 172.21.15.83:0/4206191176 >> 172.21.15.83:0/4206191176 conn(0x7fa9a80b4170 msgr2=0x7fa9a80b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 -- 172.21.15.83:0/4206191176 shutdown_connections 2024-05-17T22:02:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.443+0000 7fa9b0121640 1 -- 172.21.15.83:0/4206191176 wait complete. 2024-05-17T22:02:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 Processor -- start 2024-05-17T22:02:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 -- start start 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a816b0c0 con 0x7fa9a806d170 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a815df80 con 0x7fa9a80b5e10 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9b0121640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a816b240 con 0x7fa9a80ce920 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47556/0 (socket says 172.21.15.83:47556) 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 -- 172.21.15.83:0/1875729673 learned_addr learned my addr 172.21.15.83:0/1875729673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9ad6bc640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9ae6be640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 -- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 msgr2=0x7fa9a81582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a81582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 -- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 msgr2=0x7fa9a80ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 -- 172.21.15.83:0/1875729673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a80b0cc0 con 0x7fa9a806d170 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9ae6be640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a81582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.447+0000 7fa9adebd640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c3f0 tx=0x7fa9a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4017070 con 0x7fa9a806d170 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a40052d0 con 0x7fa9a806d170 2024-05-17T22:02:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a80cabb0 con 0x7fa9a806d170 2024-05-17T22:02:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4003820 con 0x7fa9a806d170 2024-05-17T22:02:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a80b1260 con 0x7fa9a806d170 2024-05-17T22:02:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a80b6220 con 0x7fa9a806d170 2024-05-17T22:02:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9a401a070 con 0x7fa9a806d170 2024-05-17T22:02:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 0x7fa98c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5939+0+0 (secure 0 0 0) 0x7fa9a408ee00 con 0x7fa9a806d170 2024-05-17T22:02:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa9ad6bc640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 0x7fa98c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.451+0000 7fa9ad6bc640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 0x7fa98c071940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9a815b0d0 tx=0x7fa998008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.455+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa9a4021070 con 0x7fa9a806d170 2024-05-17T22:02:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:43.651+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa9a81f7890 con 0x7fa9a806d170 2024-05-17T22:02:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.275+0000 7fa99e7fc640 1 -- 172.21.15.83:0/1875729673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7fa9a4005480 con 0x7fa9a806d170 2024-05-17T22:02:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 msgr2=0x7fa98c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 0x7fa98c071940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9a815b0d0 tx=0x7fa998008040 comp rx=0 tx=0).stop 2024-05-17T22:02:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 msgr2=0x7fa9a81f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c3f0 tx=0x7fa9a400c8b0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 shutdown_connections 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9a80ce920 0x7fa9a81582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa98c06f430 0x7fa98c071940 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9a80b5e10 0x7fa9a80ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 --2- 172.21.15.83:0/1875729673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9a806d170 0x7fa9a81f7350 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 >> 172.21.15.83:0/1875729673 conn(0x7fa9a80b4170 msgr2=0x7fa9a80b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 shutdown_connections 2024-05-17T22:02:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.279+0000 7fa9b0121640 1 -- 172.21.15.83:0/1875729673 wait complete. 2024-05-17T22:02:44.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:02:44.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.455+0000 7fc17db2e640 1 Processor -- start 2024-05-17T22:02:44.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.455+0000 7fc17db2e640 1 -- start start 2024-05-17T22:02:44.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc17db2e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 0x7fc1780b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc17db2e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 0x7fc1780b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc17db2e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1780d0820 con 0x7fc178160420 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc17db2e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc1780b4a90 con 0x7fc1780b4f20 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc17db2e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc1780d0b00 con 0x7fc1781570e0 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc177fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 0x7fc1780b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc1777fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 0x7fc1780b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc177fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 0x7fc1780b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:57726/0 (socket says 172.21.15.83:57726) 2024-05-17T22:02:44.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc177fff640 1 -- 172.21.15.83:0/1898115419 learned_addr learned my addr 172.21.15.83:0/1898115419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc1777fe640 1 -- 172.21.15.83:0/1898115419 >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 msgr2=0x7fc1780b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1898115419 >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 0x7fc1780b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc1777fe640 1 -- 172.21.15.83:0/1898115419 >> v1:172.21.15.155:6789/0 conn(0x7fc178160420 legacy=0x7fc178160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.459+0000 7fc1777fe640 1 -- 172.21.15.83:0/1898115419 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1780d0de0 con 0x7fc1781570e0 2024-05-17T22:02:44.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1898115419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 0x7fc1780b3a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1680097c0 tx=0x7fc16801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0379342a06efeca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:44.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc176ffd640 1 -- 172.21.15.83:0/1898115419 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680b2070 con 0x7fc1781570e0 2024-05-17T22:02:44.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc176ffd640 1 -- 172.21.15.83:0/1898115419 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1680ad640 con 0x7fc1781570e0 2024-05-17T22:02:44.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc176ffd640 1 -- 172.21.15.83:0/1898115419 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680ada10 con 0x7fc1781570e0 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- 172.21.15.83:0/1898115419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 msgr2=0x7fc1780b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1898115419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 0x7fc1780b3a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1680097c0 tx=0x7fc16801c470 comp rx=0 tx=0).stop 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- 172.21.15.83:0/1898115419 shutdown_connections 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1898115419 >> v2:172.21.15.155:3300/0 conn(0x7fc1780b4f20 0x7fc1780b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1898115419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1781570e0 0x7fc1780b3a60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- 172.21.15.83:0/1898115419 >> 172.21.15.83:0/1898115419 conn(0x7fc17806cd90 msgr2=0x7fc1780b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- 172.21.15.83:0/1898115419 shutdown_connections 2024-05-17T22:02:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- 172.21.15.83:0/1898115419 wait complete. 2024-05-17T22:02:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 Processor -- start 2024-05-17T22:02:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.463+0000 7fc17db2e640 1 -- start start 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 0x7fc17813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 0x7fc17813d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1781695b0 con 0x7fc17815ed70 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1780d05c0 con 0x7fc1780b4f20 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc178169730 con 0x7fc178160420 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47570/0 (socket says 172.21.15.83:47570) 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 -- 172.21.15.83:0/1013119104 learned_addr learned my addr 172.21.15.83:0/1013119104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc16ffff640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 0x7fc17813d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 0x7fc17813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 -- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 msgr2=0x7fc17813d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 0x7fc17813d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 -- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 msgr2=0x7fc17813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 0x7fc17813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 -- 172.21.15.83:0/1013119104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17815f6c0 con 0x7fc17815ed70 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 0x7fc17813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc16ffff640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 0x7fc17813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc177fff640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc17000efc0 tx=0x7fc17000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000cd20 con 0x7fc17815ed70 2024-05-17T22:02:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17813ded0 con 0x7fc17815ed70 2024-05-17T22:02:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1700039a0 con 0x7fc17815ed70 2024-05-17T22:02:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003d90 con 0x7fc17815ed70 2024-05-17T22:02:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17813e490 con 0x7fc17815ed70 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1780b3a60 con 0x7fc17815ed70 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc170012050 con 0x7fc17815ed70 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 0x7fc158071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5939+0+0 (secure 0 0 0) 0x7fc17008f940 con 0x7fc17815ed70 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.467+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 0x7fc158071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.471+0000 7fc1777fe640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 0x7fc158071910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1680af000 tx=0x7fc168008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.471+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc17815f6c0 con 0x7fc17815ed70 2024-05-17T22:02:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:44.627+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1780764e0 con 0x7fc17815ed70 2024-05-17T22:02:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.291+0000 7fc174ff9640 1 -- 172.21.15.83:0/1013119104 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7fc170010040 con 0x7fc17815ed70 2024-05-17T22:02:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:02:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.299+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 msgr2=0x7fc158071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.299+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 0x7fc158071910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1680af000 tx=0x7fc168008040 comp rx=0 tx=0).stop 2024-05-17T22:02:45.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.299+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 msgr2=0x7fc17815f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.299+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc17000efc0 tx=0x7fc17000c490 comp rx=0 tx=0).stop 2024-05-17T22:02:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 shutdown_connections 2024-05-17T22:02:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc17815ed70 0x7fc17815f180 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc178160420 0x7fc17813d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc15806f400 0x7fc158071910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 --2- 172.21.15.83:0/1013119104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1780b4f20 0x7fc17813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 >> 172.21.15.83:0/1013119104 conn(0x7fc17806cd90 msgr2=0x7fc1781550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 shutdown_connections 2024-05-17T22:02:45.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.303+0000 7fc17db2e640 1 -- 172.21.15.83:0/1013119104 wait complete. 2024-05-17T22:02:45.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:02:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 Processor -- start 2024-05-17T22:02:45.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 -- start start 2024-05-17T22:02:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c1570e0 0x7f116c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f116c160420 0x7f116c160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f116c160d70 con 0x7f116c15cde0 2024-05-17T22:02:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f116c0b4a90 con 0x7f116c160420 2024-05-17T22:02:45.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.491+0000 7f1172cb1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f116c0d0820 con 0x7f116c1570e0 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f116c160420 0x7f116c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f1170a4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c1570e0 0x7f116c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f116c160420 0x7f116c160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:57748/0 (socket says 172.21.15.83:57748) 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 -- 172.21.15.83:0/2720417700 learned_addr learned my addr 172.21.15.83:0/2720417700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f117124e640 1 --1- 172.21.15.83:0/2720417700 >> v1:172.21.15.155:6789/0 conn(0x7f116c15cde0 0x7f116c15d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:35210/0 (socket says 172.21.15.83:35210) 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 -- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c1570e0 msgr2=0x7f116c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 --2- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c1570e0 0x7f116c0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 -- 172.21.15.83:0/2720417700 >> v1:172.21.15.155:6789/0 conn(0x7f116c15cde0 legacy=0x7f116c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 -- 172.21.15.83:0/2720417700 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c0d0b00 con 0x7f116c160420 2024-05-17T22:02:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116bfff640 1 --2- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c160830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1160009870 tx=0x7f116001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ec702effc34055 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:45.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116b7fe640 1 -- 172.21.15.83:0/2720417700 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600a95f0 con 0x7f116c160420 2024-05-17T22:02:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116b7fe640 1 -- 172.21.15.83:0/2720417700 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11600a9790 con 0x7f116c160420 2024-05-17T22:02:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.495+0000 7f116b7fe640 1 -- 172.21.15.83:0/2720417700 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600a9b60 con 0x7f116c160420 2024-05-17T22:02:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 msgr2=0x7f116c160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c160830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1160009870 tx=0x7f116001c530 comp rx=0 tx=0).stop 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- 172.21.15.83:0/2720417700 shutdown_connections 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c160830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- 172.21.15.83:0/2720417700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c1570e0 0x7f116c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- 172.21.15.83:0/2720417700 >> 172.21.15.83:0/2720417700 conn(0x7f116c06cd90 msgr2=0x7f116c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- 172.21.15.83:0/2720417700 shutdown_connections 2024-05-17T22:02:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- 172.21.15.83:0/2720417700 wait complete. 2024-05-17T22:02:45.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 Processor -- start 2024-05-17T22:02:45.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- start start 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c15cde0 0x7f116c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0d03b0 con 0x7f116c1570e0 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f116c15dd10 con 0x7f116c160420 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1172cb1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f116c0d0530 con 0x7f116c15cde0 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1170a4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1170a4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47576/0 (socket says 172.21.15.83:47576) 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.499+0000 7f1170a4d640 1 -- 172.21.15.83:0/1436083880 learned_addr learned my addr 172.21.15.83:0/1436083880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f117124e640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f116bfff640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c15cde0 0x7f116c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 -- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c15cde0 msgr2=0x7f116c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c15cde0 0x7f116c1589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 -- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 msgr2=0x7f116c1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 -- 172.21.15.83:0/1436083880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c0b12b0 con 0x7f116c1570e0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1170a4d640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e8a0 tx=0x7f115c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c017070 con 0x7f116c1570e0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115c00c940 con 0x7f116c1570e0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c0b1550 con 0x7f116c1570e0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c00ccf0 con 0x7f116c1570e0 2024-05-17T22:02:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c1536f0 con 0x7f116c1570e0 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f116c160a40 con 0x7f116c1570e0 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f115c01a070 con 0x7f116c1570e0 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 0x7f1158071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5501+0+0 (secure 0 0 0) 0x7f115c08f4d0 con 0x7f116c1570e0 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f116bfff640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 0x7f1158071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.503+0000 7f116bfff640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 0x7f1158071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11600ae000 tx=0x7f11600a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.507+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f115c05c490 con 0x7f116c1570e0 2024-05-17T22:02:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:45.670+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f116c070140 con 0x7f116c1570e0 2024-05-17T22:02:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.294+0000 7f11697fa640 1 -- 172.21.15.83:0/1436083880 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f115c05bd90 con 0x7f116c1570e0 2024-05-17T22:02:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T22:02:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 msgr2=0x7f1158071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 0x7f1158071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f11600ae000 tx=0x7f11600a7000 comp rx=0 tx=0).stop 2024-05-17T22:02:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 msgr2=0x7f116c151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f115c00e8a0 tx=0x7f115c00eca0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 shutdown_connections 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f115806f400 0x7f1158071910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c160420 0x7f116c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f116c15cde0 0x7f116c1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 --2- 172.21.15.83:0/1436083880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116c1570e0 0x7f116c151620 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 >> 172.21.15.83:0/1436083880 conn(0x7f116c06cd90 msgr2=0x7f116c15e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 shutdown_connections 2024-05-17T22:02:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.298+0000 7f1172cb1640 1 -- 172.21.15.83:0/1436083880 wait complete. 2024-05-17T22:02:46.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T22:02:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.490+0000 7f5638f13640 1 Processor -- start 2024-05-17T22:02:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.490+0000 7f5638f13640 1 -- start start 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5638f13640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340b3d00 0x7f56340d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5638f13640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5634143f20 0x7f5634072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5638f13640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340737d0 con 0x7f56340733c0 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5638f13640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f56340b4110 con 0x7f5634143f20 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5638f13640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5634070770 con 0x7f56340b3d00 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5634143f20 0x7f5634072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5634143f20 0x7f5634072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47590/0 (socket says 172.21.15.83:47590) 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 -- 172.21.15.83:0/2243160578 learned_addr learned my addr 172.21.15.83:0/2243160578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f563259c640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340b3d00 0x7f56340d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 -- 172.21.15.83:0/2243160578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340b3d00 msgr2=0x7f56340d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340b3d00 0x7f56340d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 -- 172.21.15.83:0/2243160578 >> v1:172.21.15.155:6789/0 conn(0x7f56340733c0 legacy=0x7f563406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.494+0000 7f5629d9b640 1 -- 172.21.15.83:0/2243160578 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340d10d0 con 0x7f5634143f20 2024-05-17T22:02:46.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5629d9b640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5634143f20 0x7f5634072e80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f561c009870 tx=0x7f561c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fb04244dae3840 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5631d9b640 1 -- 172.21.15.83:0/2243160578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0ab650 con 0x7f5634143f20 2024-05-17T22:02:46.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5631d9b640 1 -- 172.21.15.83:0/2243160578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c0b0070 con 0x7f5634143f20 2024-05-17T22:02:46.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5631d9b640 1 -- 172.21.15.83:0/2243160578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0aba10 con 0x7f5634143f20 2024-05-17T22:02:46.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- 172.21.15.83:0/2243160578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5634143f20 msgr2=0x7f5634072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5634143f20 0x7f5634072e80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f561c009870 tx=0x7f561c01c560 comp rx=0 tx=0).stop 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- 172.21.15.83:0/2243160578 shutdown_connections 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5634143f20 0x7f5634072e80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 --2- 172.21.15.83:0/2243160578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340b3d00 0x7f56340d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- 172.21.15.83:0/2243160578 >> 172.21.15.83:0/2243160578 conn(0x7f563406cd90 msgr2=0x7f56340b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- 172.21.15.83:0/2243160578 shutdown_connections 2024-05-17T22:02:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- 172.21.15.83:0/2243160578 wait complete. 2024-05-17T22:02:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 Processor -- start 2024-05-17T22:02:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.498+0000 7f5638f13640 1 -- start start 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340733c0 0x7f563415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5634143f20 0x7f563413b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5634070c00 con 0x7f56340b3d00 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5634144330 con 0x7f5634143f20 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f56340af830 con 0x7f56340733c0 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47592/0 (socket says 172.21.15.83:47592) 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 -- 172.21.15.83:0/3739316716 learned_addr learned my addr 172.21.15.83:0/3739316716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f563259c640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340733c0 0x7f563415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5632d9d640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5634143f20 0x7f563413b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 -- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340733c0 msgr2=0x7f563415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340733c0 0x7f563415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 -- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5634143f20 msgr2=0x7f563413b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5634143f20 0x7f563413b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 -- 172.21.15.83:0/3739316716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340b54d0 con 0x7f56340b3d00 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5629d9b640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ad040 tx=0x7f561c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0bf610 con 0x7f56340b3d00 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c0bf7d0 con 0x7f56340b3d00 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56340b6570 con 0x7f56340b3d00 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0bfbc0 con 0x7f56340b3d00 2024-05-17T22:02:46.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340b5a90 con 0x7f56340b3d00 2024-05-17T22:02:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56340d0a10 con 0x7f56340b3d00 2024-05-17T22:02:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f561c0bd020 con 0x7f56340b3d00 2024-05-17T22:02:46.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.502+0000 7f562b7fe640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 0x7f5614071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.506+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7f561c129920 con 0x7f56340b3d00 2024-05-17T22:02:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.506+0000 7f563259c640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 0x7f5614071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.506+0000 7f563259c640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 0x7f5614071910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f562400b930 tx=0x7f5624008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.510+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f561c016150 con 0x7f56340b3d00 2024-05-17T22:02:46.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.682+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f56340d0cb0 con 0x7f56340b3d00 2024-05-17T22:02:46.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.686+0000 7f562b7fe640 1 -- 172.21.15.83:0/3739316716 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f561c0f6ab0 con 0x7f56340b3d00 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 msgr2=0x7f5614071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 0x7f5614071910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f562400b930 tx=0x7f5624008040 comp rx=0 tx=0).stop 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 msgr2=0x7f563413b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ad040 tx=0x7f561c019300 comp rx=0 tx=0).stop 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 shutdown_connections 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f561406f400 0x7f5614071910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5634143f20 0x7f563413b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56340b3d00 0x7f563413b3a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 --2- 172.21.15.83:0/3739316716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56340733c0 0x7f563415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 >> 172.21.15.83:0/3739316716 conn(0x7f563406cd90 msgr2=0x7f563413d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 shutdown_connections 2024-05-17T22:02:46.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.690+0000 7f5638f13640 1 -- 172.21.15.83:0/3739316716 wait complete. 2024-05-17T22:02:46.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-17T22:02:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 Processor -- start 2024-05-17T22:02:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 -- start start 2024-05-17T22:02:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 0x7f84ac0d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ac06e610 con 0x7f84ac15ffa0 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f84ac0d34e0 con 0x7f84ac0d4260 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b2b7f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac0d3780 con 0x7f84ac06dcc0 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b111c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 0x7f84ac0d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:45166/0 (socket says 172.21.15.83:45166) 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 -- 172.21.15.83:0/3968713632 learned_addr learned my addr 172.21.15.83:0/3968713632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:46.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 -- 172.21.15.83:0/3968713632 >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 msgr2=0x7f84ac0d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 --2- 172.21.15.83:0/3968713632 >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 0x7f84ac0d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 -- 172.21.15.83:0/3968713632 >> v1:172.21.15.155:6789/0 conn(0x7f84ac15ffa0 legacy=0x7f84ac0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84a3fff640 1 -- 172.21.15.83:0/3968713632 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac0d3a60 con 0x7f84ac06dcc0 2024-05-17T22:02:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.858+0000 7f84b111c640 1 --2- 172.21.15.83:0/3968713632 >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 0x7f84ac0d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:02:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84a3fff640 1 --2- 172.21.15.83:0/3968713632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f849c009870 tx=0x7f849c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c23cf70d585f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84a37fe640 1 -- 172.21.15.83:0/3968713632 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0b1070 con 0x7f84ac06dcc0 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84a37fe640 1 -- 172.21.15.83:0/3968713632 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849c0a8050 con 0x7f84ac06dcc0 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84a37fe640 1 -- 172.21.15.83:0/3968713632 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0ac860 con 0x7f84ac06dcc0 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 -- 172.21.15.83:0/3968713632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 msgr2=0x7f84ac06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/3968713632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f849c009870 tx=0x7f849c01c560 comp rx=0 tx=0).stop 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 -- 172.21.15.83:0/3968713632 shutdown_connections 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/3968713632 >> v2:172.21.15.155:3300/0 conn(0x7f84ac0d4260 0x7f84ac0d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/3968713632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac06dcc0 0x7f84ac06e0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 -- 172.21.15.83:0/3968713632 >> 172.21.15.83:0/3968713632 conn(0x7f84ac0d28f0 msgr2=0x7f84ac0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 -- 172.21.15.83:0/3968713632 shutdown_connections 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.862+0000 7f84b2b7f640 1 -- 172.21.15.83:0/3968713632 wait complete. 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 Processor -- start 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- start start 2024-05-17T22:02:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0d4260 0x7f84ac1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 0x7f84ac1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac0d17e0 con 0x7f84ac06dcc0 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84ac0d1660 con 0x7f84ac0d4260 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f84ac16cb80 con 0x7f84ac15ffa0 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47604/0 (socket says 172.21.15.83:47604) 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 -- 172.21.15.83:0/2878410987 learned_addr learned my addr 172.21.15.83:0/2878410987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b111c640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 0x7f84ac1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84a3fff640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0d4260 0x7f84ac1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 -- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 msgr2=0x7f84ac1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 0x7f84ac1588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 -- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0d4260 msgr2=0x7f84ac1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0d4260 0x7f84ac1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 -- 172.21.15.83:0/2878410987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ac15adc0 con 0x7f84ac06dcc0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b111c640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 0x7f84ac1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b091b640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c790 tx=0x7f84a800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8003b80 con 0x7f84ac06dcc0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a8003d40 con 0x7f84ac06dcc0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ac15f2a0 con 0x7f84ac06dcc0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a800aa90 con 0x7f84ac06dcc0 2024-05-17T22:02:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.866+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ac15b0a0 con 0x7f84ac06dcc0 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f84a800ac30 con 0x7f84ac06dcc0 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ac06e0d0 con 0x7f84ac06dcc0 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84a17fa640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 0x7f8498071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7f84a808ee60 con 0x7f84ac06dcc0 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84a3fff640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 0x7f8498071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.870+0000 7f84a3fff640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 0x7f8498071940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ae000 tx=0x7f849c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:46.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:46.874+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f84ac06e610 con 0x7f84ac06dcc0 2024-05-17T22:02:47.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.094+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f84ac0d0060 con 0x7f84ac06dcc0 2024-05-17T22:02:47.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.318+0000 7f84a17fa640 1 -- 172.21.15.83:0/2878410987 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v259) ==== 166+0+0 (secure 0 0 0) 0x7f84a805c080 con 0x7f84ac06dcc0 2024-05-17T22:02:47.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 msgr2=0x7f8498071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 0x7f8498071940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ae000 tx=0x7f849c008040 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 msgr2=0x7f84ac151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c790 tx=0x7f84a800cc50 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 shutdown_connections 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84ac15ffa0 0x7f84ac1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f849806f430 0x7f8498071940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0d4260 0x7f84ac1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 --2- 172.21.15.83:0/2878410987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ac06dcc0 0x7f84ac151da0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 >> 172.21.15.83:0/2878410987 conn(0x7f84ac0d28f0 msgr2=0x7f84ac0bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 shutdown_connections 2024-05-17T22:02:47.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.322+0000 7f84b2b7f640 1 -- 172.21.15.83:0/2878410987 wait complete. 2024-05-17T22:02:47.340 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-17T22:02:47.340 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-17T22:02:47.340 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T22:02:47.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:47.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 Processor -- start 2024-05-17T22:02:47.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 -- start start 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f566c144080 0x7f566c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f566c073370 0x7f566c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c073780 con 0x7f566c0d1040 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f566c15e790 con 0x7f566c073370 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f56717e8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f566c15ea40 con 0x7f566c144080 2024-05-17T22:02:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f566c073370 0x7f566c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f566c0d1040 0x7f566c15e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59582/0 (socket says 172.21.15.83:59582) 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566affd640 1 -- 172.21.15.83:0/2697434788 learned_addr learned my addr 172.21.15.83:0/2697434788 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566a7fc640 1 --2- 172.21.15.83:0/2697434788 >> v2:172.21.15.155:3300/0 conn(0x7f566c144080 0x7f566c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566b7fe640 1 -- 172.21.15.83:0/2697434788 >> v2:172.21.15.155:3300/0 conn(0x7f566c144080 msgr2=0x7f566c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566b7fe640 1 --2- 172.21.15.83:0/2697434788 >> v2:172.21.15.155:3300/0 conn(0x7f566c144080 0x7f566c072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566b7fe640 1 -- 172.21.15.83:0/2697434788 >> v1:172.21.15.83:6789/0 conn(0x7f566c0d1040 legacy=0x7f566c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f566b7fe640 1 -- 172.21.15.83:0/2697434788 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c15ed20 con 0x7f566c073370 2024-05-17T22:02:47.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.502+0000 7f5669ffb640 1 -- 172.21.15.83:0/2697434788 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243093749 0 0) 0x7f5660001120 con 0x7f566c0d1040 2024-05-17T22:02:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f566b7fe640 1 --2- 172.21.15.83:0/2697434788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c073370 0x7f566c06f730 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f566c0b6b10 tx=0x7f566401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f7359e79e95957 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f5669ffb640 1 -- 172.21.15.83:0/2697434788 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640b2070 con 0x7f566c073370 2024-05-17T22:02:47.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f5669ffb640 1 -- 172.21.15.83:0/2697434788 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640a59c0 con 0x7f566c073370 2024-05-17T22:02:47.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f5669ffb640 1 -- 172.21.15.83:0/2697434788 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640a5d90 con 0x7f566c073370 2024-05-17T22:02:47.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 -- 172.21.15.83:0/2697434788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c073370 msgr2=0x7f566c06f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 --2- 172.21.15.83:0/2697434788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c073370 0x7f566c06f730 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f566c0b6b10 tx=0x7f566401c900 comp rx=0 tx=0).stop 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 -- 172.21.15.83:0/2697434788 shutdown_connections 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 --2- 172.21.15.83:0/2697434788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c073370 0x7f566c06f730 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 --2- 172.21.15.83:0/2697434788 >> v2:172.21.15.155:3300/0 conn(0x7f566c144080 0x7f566c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 -- 172.21.15.83:0/2697434788 >> 172.21.15.83:0/2697434788 conn(0x7f566c06cd90 msgr2=0x7f566c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 -- 172.21.15.83:0/2697434788 shutdown_connections 2024-05-17T22:02:47.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.506+0000 7f56717e8640 1 -- 172.21.15.83:0/2697434788 wait complete. 2024-05-17T22:02:47.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 Processor -- start 2024-05-17T22:02:47.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- start start 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566c073370 0x7f566c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 0x7f566c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f566c070480 con 0x7f566c144080 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f566c144490 con 0x7f566c073370 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f566c070600 con 0x7f566c0d1040 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33800/0 (socket says 172.21.15.83:33800) 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 -- 172.21.15.83:0/2226308922 learned_addr learned my addr 172.21.15.83:0/2226308922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:47.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566affd640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566c073370 0x7f566c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566a7fc640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 0x7f566c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 -- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 msgr2=0x7f566c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 0x7f566c15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 -- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566c073370 msgr2=0x7f566c0ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566c073370 0x7f566c0ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 -- 172.21.15.83:0/2226308922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c0adf10 con 0x7f566c144080 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566a7fc640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 0x7f566c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f566b7fe640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5664019510 tx=0x7f56640196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640b2050 con 0x7f566c144080 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640be070 con 0x7f566c144080 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640ada70 con 0x7f566c144080 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c0ae1f0 con 0x7f566c144080 2024-05-17T22:02:47.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c0ae7b0 con 0x7f566c144080 2024-05-17T22:02:47.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.510+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c069c10 con 0x7f566c144080 2024-05-17T22:02:47.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.514+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56640adc10 con 0x7f566c144080 2024-05-17T22:02:47.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.514+0000 7f56537fe640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 0x7f5658071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.514+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7f5664129c80 con 0x7f566c144080 2024-05-17T22:02:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.514+0000 7f566affd640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 0x7f5658071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.514+0000 7f566affd640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 0x7f5658071940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5660000f70 tx=0x7f5660007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.518+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f56640c7020 con 0x7f566c144080 2024-05-17T22:02:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.678+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566c0709e0 con 0x7f566c144080 2024-05-17T22:02:47.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56537fe640 1 -- 172.21.15.83:0/2226308922 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13127 (secure 0 0 0) 0x7f5664016520 con 0x7f566c144080 2024-05-17T22:02:47.674 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:47.674 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":73,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:02:46.297056+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/1116165089":"2024-05-18T22:02:43.256497+0000","172.21.15.83:6836/1116165089":"2024-05-18T22:02:43.256497+0000","172.21.15.83:6837/1850779527":"2024-05-18T22:01:18.280208+0000","172.21.15.83:6836/1850779527":"2024-05-18T22:01:18.280208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 msgr2=0x7f5658071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 0x7f5658071940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5660000f70 tx=0x7f5660007c10 comp rx=0 tx=0).stop 2024-05-17T22:02:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 msgr2=0x7f566c15cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5664019510 tx=0x7f56640196a0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 shutdown_connections 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f566c144080 0x7f566c15cc40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f566c0d1040 0x7f566c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f565806f430 0x7f5658071940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 --2- 172.21.15.83:0/2226308922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f566c073370 0x7f566c0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 >> 172.21.15.83:0/2226308922 conn(0x7f566c06cd90 msgr2=0x7f566c1468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 shutdown_connections 2024-05-17T22:02:47.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.682+0000 7f56717e8640 1 -- 172.21.15.83:0/2226308922 wait complete. 2024-05-17T22:02:47.697 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T22:02:47.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:02:47.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.850+0000 7fa3a97ba640 1 Processor -- start 2024-05-17T22:02:47.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.850+0000 7fa3a97ba640 1 -- start start 2024-05-17T22:02:47.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a97ba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3a40b3d00 0x7fa3a40d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a97ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 0x7fa3a4072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a97ba640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a40737d0 con 0x7fa3a40733c0 2024-05-17T22:02:47.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a97ba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a40b4110 con 0x7fa3a40b3d00 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a97ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a4070770 con 0x7fa3a4143f20 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3a40b3d00 0x7fa3a40d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3a40b3d00 0x7fa3a40d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33806/0 (socket says 172.21.15.83:33806) 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 -- 172.21.15.83:0/3215009907 learned_addr learned my addr 172.21.15.83:0/3215009907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a27fc640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 0x7fa3a4072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 -- 172.21.15.83:0/3215009907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 msgr2=0x7fa3a4072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 0x7fa3a4072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 -- 172.21.15.83:0/3215009907 >> v1:172.21.15.155:6789/0 conn(0x7fa3a40733c0 legacy=0x7fa3a406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a2ffd640 1 -- 172.21.15.83:0/3215009907 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a40d10d0 con 0x7fa3a40b3d00 2024-05-17T22:02:47.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.854+0000 7fa3a27fc640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 0x7fa3a4072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:47.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a40b3d00 0x7fa3a40d0a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa394009870 tx=0x7fa39401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc24a5b06364a8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a1ffb640 1 -- 172.21.15.83:0/3215009907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940b1070 con 0x7fa3a40b3d00 2024-05-17T22:02:47.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a1ffb640 1 -- 172.21.15.83:0/3215009907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3940ac630 con 0x7fa3a40b3d00 2024-05-17T22:02:47.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a1ffb640 1 -- 172.21.15.83:0/3215009907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940a8050 con 0x7fa3a40b3d00 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 -- 172.21.15.83:0/3215009907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a40b3d00 msgr2=0x7fa3a40d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a40b3d00 0x7fa3a40d0a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa394009870 tx=0x7fa39401c560 comp rx=0 tx=0).stop 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 -- 172.21.15.83:0/3215009907 shutdown_connections 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a4143f20 0x7fa3a4072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/3215009907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a40b3d00 0x7fa3a40d0a10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 -- 172.21.15.83:0/3215009907 >> 172.21.15.83:0/3215009907 conn(0x7fa3a406cd90 msgr2=0x7fa3a40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 -- 172.21.15.83:0/3215009907 shutdown_connections 2024-05-17T22:02:47.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.858+0000 7fa3a97ba640 1 -- 172.21.15.83:0/3215009907 wait complete. 2024-05-17T22:02:47.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 Processor -- start 2024-05-17T22:02:47.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- start start 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 0x7fa3a4162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 0x7fa3a41f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4070c00 con 0x7fa3a4143f20 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4144330 con 0x7fa3a40b3d00 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a4161aa0 con 0x7fa3a40733c0 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33810/0 (socket says 172.21.15.83:33810) 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 -- 172.21.15.83:0/751021337 learned_addr learned my addr 172.21.15.83:0/751021337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:47.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 0x7fa3a4162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a27fc640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 0x7fa3a41f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 -- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 msgr2=0x7fa3a4162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 0x7fa3a4162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 -- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 msgr2=0x7fa3a41f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 0x7fa3a41f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 -- 172.21.15.83:0/751021337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a41615d0 con 0x7fa3a4143f20 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 0x7fa3a4162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a27fc640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 0x7fa3a41f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a37fe640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00b9e0 tx=0x7fa39c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003b80 con 0x7fa3a4143f20 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c003d20 con 0x7fa3a4143f20 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00cb00 con 0x7fa3a4143f20 2024-05-17T22:02:47.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41fb390 con 0x7fa3a4143f20 2024-05-17T22:02:47.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41fadd0 con 0x7fa3a4143f20 2024-05-17T22:02:47.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.862+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a40d0a10 con 0x7fa3a4143f20 2024-05-17T22:02:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.866+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa39c006d10 con 0x7fa3a4143f20 2024-05-17T22:02:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.866+0000 7fa38b7fe640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 0x7fa390071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.866+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 0x7fa390071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.866+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7fa39c08f320 con 0x7fa3a4143f20 2024-05-17T22:02:47.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.866+0000 7fa3a2ffd640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 0x7fa390071910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ae000 tx=0x7fa394008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:47.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:47.870+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa39c05c590 con 0x7fa3a4143f20 2024-05-17T22:02:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.030+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3a4168460 con 0x7fa3a4143f20 2024-05-17T22:02:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa38b7fe640 1 -- 172.21.15.83:0/751021337 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fa39c05be90 con 0x7fa3a4143f20 2024-05-17T22:02:48.025 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T22:02:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 msgr2=0x7fa390071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 0x7fa390071910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ae000 tx=0x7fa394008040 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 msgr2=0x7fa3a4143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00b9e0 tx=0x7fa39c00bea0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 shutdown_connections 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3a4143f20 0x7fa3a4143050 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa39006f400 0x7fa390071910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3a40b3d00 0x7fa3a41f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 --2- 172.21.15.83:0/751021337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3a40733c0 0x7fa3a4162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 >> 172.21.15.83:0/751021337 conn(0x7fa3a406cd90 msgr2=0x7fa3a4160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 shutdown_connections 2024-05-17T22:02:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.034+0000 7fa3a97ba640 1 -- 172.21.15.83:0/751021337 wait complete. 2024-05-17T22:02:48.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-17T22:02:48.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 Processor -- start 2024-05-17T22:02:48.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 -- start start 2024-05-17T22:02:48.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d0073540 0x7fe4d0168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d015e790 con 0x7fe4d00d1040 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d015ea40 con 0x7fe4d0144080 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4d5b52640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d015ed20 con 0x7fe4d0073540 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cf7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe4d00d1040 0x7fe4d015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59590/0 (socket says 172.21.15.83:59590) 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cf7fe640 1 -- 172.21.15.83:0/1334852363 learned_addr learned my addr 172.21.15.83:0/1334852363 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d0073540 0x7fe4d0168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670365149 0 0) 0x7fe4d015e790 con 0x7fe4d00d1040 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4bc003530 con 0x7fe4d00d1040 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 365855705 0 0) 0x7fe4bc003530 con 0x7fe4d00d1040 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d0073540 msgr2=0x7fe4d0168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4d0073540 0x7fe4d0168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 >> v1:172.21.15.155:6789/0 conn(0x7fe4d0144080 legacy=0x7fe4d0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d015f000 con 0x7fe4d00d1040 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe4c0001700 con 0x7fe4d00d1040 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 >> v1:172.21.15.83:6789/0 conn(0x7fe4d00d1040 legacy=0x7fe4d015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc003a40 0x7fe4bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4bc004810 0x7fe4bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4bc005160 0x7fe4bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d015ed20 con 0x7fe4bc005160 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc00c020 con 0x7fe4bc004810 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc00c220 con 0x7fe4bc003a40 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4bc005160 0x7fe4bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4c7fff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4bc004810 0x7fe4bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 -- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc003a40 msgr2=0x7fe4bc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc003a40 0x7fe4bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 -- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4bc004810 msgr2=0x7fe4bc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4bc004810 0x7fe4bc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.206+0000 7fe4cffff640 1 -- 172.21.15.83:0/1334852363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc003530 con 0x7fe4bc005160 2024-05-17T22:02:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4cffff640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4bc005160 0x7fe4bc00b9d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009f90 tx=0x7fe4c801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe639e3282ee7f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b2070 con 0x7fe4bc005160 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80a5c90 con 0x7fe4bc005160 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4ceffd640 1 -- 172.21.15.83:0/1334852363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8009c10 con 0x7fe4bc005160 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 -- 172.21.15.83:0/1334852363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4bc005160 msgr2=0x7fe4bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/1334852363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4bc005160 0x7fe4bc00b9d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009f90 tx=0x7fe4c801cc80 comp rx=0 tx=0).stop 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4cffff640 1 -- 172.21.15.83:0/1334852363 reap_dead start 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 -- 172.21.15.83:0/1334852363 shutdown_connections 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 -- 172.21.15.83:0/1334852363 >> 172.21.15.83:0/1334852363 conn(0x7fe4d006cd90 msgr2=0x7fe4d00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 -- 172.21.15.83:0/1334852363 shutdown_connections 2024-05-17T22:02:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.210+0000 7fe4d5b52640 1 -- 172.21.15.83:0/1334852363 wait complete. 2024-05-17T22:02:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 Processor -- start 2024-05-17T22:02:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- start start 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc005160 0x7fe4d00abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d01f6d10 0x7fe4d01f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0144490 con 0x7fe4d00acc80 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d006f210 con 0x7fe4d01f6d10 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d0073540 con 0x7fe4bc005160 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33834/0 (socket says 172.21.15.83:33834) 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 -- 172.21.15.83:0/679046803 learned_addr learned my addr 172.21.15.83:0/679046803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4cf7fe640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc005160 0x7fe4d00abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4cffff640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d01f6d10 0x7fe4d01f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 -- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc005160 msgr2=0x7fe4d00abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc005160 0x7fe4d00abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 -- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d01f6d10 msgr2=0x7fe4d01f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d01f6d10 0x7fe4d01f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 -- 172.21.15.83:0/679046803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d01f7660 con 0x7fe4d00acc80 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4c7fff640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e580 tx=0x7fe4b800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8019070 con 0x7fe4d00acc80 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b80052e0 con 0x7fe4d00acc80 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d00adc00 con 0x7fe4d00acc80 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8006a20 con 0x7fe4d00acc80 2024-05-17T22:02:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d00ae1c0 con 0x7fe4d00acc80 2024-05-17T22:02:48.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.214+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d00ad970 con 0x7fe4d00acc80 2024-05-17T22:02:48.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.218+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe4b801c070 con 0x7fe4d00acc80 2024-05-17T22:02:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.218+0000 7fe4ccff9640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 0x7fe4b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.218+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7fe4b808ee20 con 0x7fe4d00acc80 2024-05-17T22:02:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.218+0000 7fe4cf7fe640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 0x7fe4b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:48.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.218+0000 7fe4cf7fe640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 0x7fe4b0071910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0001870 tx=0x7fe4c00017a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.222+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe4d01f7660 con 0x7fe4d00acc80 2024-05-17T22:02:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:48.438+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fe4d0072e30 con 0x7fe4d00acc80 2024-05-17T22:02:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.326+0000 7fe4ccff9640 1 -- 172.21.15.83:0/679046803 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v261) ==== 170+0+0 (secure 0 0 0) 0x7fe4b805b960 con 0x7fe4d00acc80 2024-05-17T22:02:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 msgr2=0x7fe4b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 0x7fe4b0071910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0001870 tx=0x7fe4c00017a0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 msgr2=0x7fe4d00ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e580 tx=0x7fe4b800ea40 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 shutdown_connections 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe4b006f400 0x7fe4b0071910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4d01f6d10 0x7fe4d01f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d00acc80 0x7fe4d00ad300 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 --2- 172.21.15.83:0/679046803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4bc005160 0x7fe4d00abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 >> 172.21.15.83:0/679046803 conn(0x7fe4d006cd90 msgr2=0x7fe4d0144a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 shutdown_connections 2024-05-17T22:02:49.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.330+0000 7fe4d5b52640 1 -- 172.21.15.83:0/679046803 wait complete. 2024-05-17T22:02:49.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T22:02:49.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.506+0000 7f07bd489640 1 Processor -- start 2024-05-17T22:02:49.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 -- start start 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f07b80b4810 0x7f07b80d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d0290 0x7f07b815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07b80d2000 0x7f07b80d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f07b80d4170 con 0x7f07b80d2000 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f07b80d4450 con 0x7f07b80b4810 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07bd489640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07b80d4730 con 0x7f07b80d0290 2024-05-17T22:02:49.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07b80d2000 0x7f07b80d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07b80d2000 0x7f07b80d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33838/0 (socket says 172.21.15.83:33838) 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 -- 172.21.15.83:0/3499879845 learned_addr learned my addr 172.21.15.83:0/3499879845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b6ffd640 1 --2- 172.21.15.83:0/3499879845 >> v2:172.21.15.155:3300/0 conn(0x7f07b80b4810 0x7f07b80d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07affff640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d0290 0x7f07b815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 -- 172.21.15.83:0/3499879845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d0290 msgr2=0x7f07b815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d0290 0x7f07b815fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 -- 172.21.15.83:0/3499879845 >> v2:172.21.15.155:3300/0 conn(0x7f07b80b4810 msgr2=0x7f07b80d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 --2- 172.21.15.83:0/3499879845 >> v2:172.21.15.155:3300/0 conn(0x7f07b80b4810 0x7f07b80d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.510+0000 7f07b77fe640 1 -- 172.21.15.83:0/3499879845 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b80d4a10 con 0x7f07b80d2000 2024-05-17T22:02:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07b77fe640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80d2000 0x7f07b80d2410 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f07b808e620 tx=0x7f07b001d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3deb369a42ef9af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07b67fc640 1 -- 172.21.15.83:0/3499879845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a5890 con 0x7f07b80d2000 2024-05-17T22:02:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07b67fc640 1 -- 172.21.15.83:0/3499879845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b00a5a30 con 0x7f07b80d2000 2024-05-17T22:02:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07b67fc640 1 -- 172.21.15.83:0/3499879845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a5e00 con 0x7f07b80d2000 2024-05-17T22:02:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- 172.21.15.83:0/3499879845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80d2000 msgr2=0x7f07b80d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80d2000 0x7f07b80d2410 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f07b808e620 tx=0x7f07b001d450 comp rx=0 tx=0).stop 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- 172.21.15.83:0/3499879845 shutdown_connections 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80d2000 0x7f07b80d2410 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 --2- 172.21.15.83:0/3499879845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d0290 0x7f07b815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 --2- 172.21.15.83:0/3499879845 >> v2:172.21.15.155:3300/0 conn(0x7f07b80b4810 0x7f07b80d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- 172.21.15.83:0/3499879845 >> 172.21.15.83:0/3499879845 conn(0x7f07b806e010 msgr2=0x7f07b806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- 172.21.15.83:0/3499879845 shutdown_connections 2024-05-17T22:02:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- 172.21.15.83:0/3499879845 wait complete. 2024-05-17T22:02:49.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 Processor -- start 2024-05-17T22:02:49.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.514+0000 7f07bd489640 1 -- start start 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 0x7f07b8143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d2000 0x7f07b815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07b80b7aa0 con 0x7f07b80b4810 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8160050 con 0x7f07b80d0290 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f07b80b7c20 con 0x7f07b80d2000 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33854/0 (socket says 172.21.15.83:33854) 2024-05-17T22:02:49.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 -- 172.21.15.83:0/2598254434 learned_addr learned my addr 172.21.15.83:0/2598254434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07affff640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 0x7f07b8143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b77fe640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d2000 0x7f07b815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 -- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d2000 msgr2=0x7f07b815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d2000 0x7f07b815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 -- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 msgr2=0x7f07b8143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 0x7f07b8143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 -- 172.21.15.83:0/2598254434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b8143fa0 con 0x7f07b80b4810 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07affff640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 0x7f07b8143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07b6ffd640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c820 tx=0x7f07a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8015070 con 0x7f07b80b4810 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a8007a30 con 0x7f07b80b4810 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b8152660 con 0x7f07b80b4810 2024-05-17T22:02:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b8152c20 con 0x7f07b80b4810 2024-05-17T22:02:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8007e20 con 0x7f07b80b4810 2024-05-17T22:02:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b80b7fe0 con 0x7f07b80b4810 2024-05-17T22:02:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f07a8018070 con 0x7f07b80b4810 2024-05-17T22:02:49.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 0x7f0798071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7f07a808ef00 con 0x7f07b80b4810 2024-05-17T22:02:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.518+0000 7f07affff640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 0x7f0798071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:49.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.522+0000 7f07affff640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 0x7f0798071940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:49.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.522+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f07a805c070 con 0x7f07b80b4810 2024-05-17T22:02:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.674+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f07b806e5c0 con 0x7f07b80b4810 2024-05-17T22:02:49.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.682+0000 7f07af7fe640 1 -- 172.21.15.83:0/2598254434 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f07a805b970 con 0x7f07b80b4810 2024-05-17T22:02:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 msgr2=0x7f0798071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 0x7f0798071940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f07a0006fd0 tx=0x7f07a0008040 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 msgr2=0x7f07b8151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c820 tx=0x7f07a800cce0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 shutdown_connections 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b80d2000 0x7f07b815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f079806f430 0x7f0798071940 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b80d0290 0x7f07b8143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 --2- 172.21.15.83:0/2598254434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b80b4810 0x7f07b8151c80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 >> 172.21.15.83:0/2598254434 conn(0x7f07b806e010 msgr2=0x7f07b80d6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 shutdown_connections 2024-05-17T22:02:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:49.686+0000 7f07bd489640 1 -- 172.21.15.83:0/2598254434 wait complete. 2024-05-17T22:02:49.696 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:02:49.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:02:49.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:02:49.789 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:02:49.789 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:02:49.789 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:49.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:02:49.795 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:02:49.795 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:49.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:02:49.836 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:02:49.836 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:49.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:02:49.842 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:02:49.842 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:02:49.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:02:49.884 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T22:02:49.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 Processor -- start 2024-05-17T22:02:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 -- start start 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6b600728a0 0x7f6b60072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b6015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b60161620 con 0x7f6b600b2610 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6b600736d0 con 0x7f6b600728a0 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b67ea2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b600731f0 con 0x7f6b600c9c70 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b6543d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6b600b2610 0x7f6b600b2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59592/0 (socket says 172.21.15.83:59592) 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b6543d640 1 -- 172.21.15.83:0/1621849119 learned_addr learned my addr 172.21.15.83:0/1621849119 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b6015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1621849119 >> v2:172.21.15.155:3300/0 conn(0x7f6b600728a0 0x7f6b60072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336957429 0 0) 0x7f6b60161620 con 0x7f6b600b2610 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b4c003530 con 0x7f6b600b2610 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1053025572 0 0) 0x7f6b4c003530 con 0x7f6b600b2610 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 msgr2=0x7f6b6015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.042+0000 7f6b64c3c640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b6015d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 >> v2:172.21.15.155:3300/0 conn(0x7f6b600728a0 msgr2=0x7f6b60072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 --2- 172.21.15.83:0/1621849119 >> v2:172.21.15.155:3300/0 conn(0x7f6b600728a0 0x7f6b60072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b60161890 con 0x7f6b600b2610 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f6b50001700 con 0x7f6b600b2610 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1621849119 >> v2:172.21.15.155:3300/0 conn(0x7f6b600728a0 0x7f6b60072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 >> v1:172.21.15.83:6789/0 conn(0x7f6b600b2610 legacy=0x7f6b600b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b4c003a60 0x7f6b4c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b4c004d50 0x7f6b4c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b600731f0 con 0x7f6b4c004d50 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c00bc20 con 0x7f6b4c003a60 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4c00be20 con 0x7f6b600c9c70 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b4c004d50 0x7f6b4c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b4c003a60 0x7f6b4c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6543d640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 -- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 msgr2=0x7f6b4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600c9c70 0x7f6b4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 -- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b4c003a60 msgr2=0x7f6b4c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b4c003a60 0x7f6b4c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 -- 172.21.15.83:0/1621849119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c003530 con 0x7f6b4c004d50 2024-05-17T22:02:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b4c004d50 0x7f6b4c005220 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ba00 tx=0x7f6b5801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9674cabf4184ffbb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800b600 con 0x7f6b4c004d50 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b580b1070 con 0x7f6b4c004d50 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b64c3c640 1 -- 172.21.15.83:0/1621849119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b580ac860 con 0x7f6b4c004d50 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1621849119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b4c004d50 msgr2=0x7f6b4c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1621849119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b4c004d50 0x7f6b4c005220 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ba00 tx=0x7f6b5801c680 comp rx=0 tx=0).stop 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b6643f640 1 -- 172.21.15.83:0/1621849119 reap_dead start 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1621849119 shutdown_connections 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1621849119 >> 172.21.15.83:0/1621849119 conn(0x7f6b600b5250 msgr2=0x7f6b600b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1621849119 shutdown_connections 2024-05-17T22:02:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.046+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1621849119 wait complete. 2024-05-17T22:02:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 Processor -- start 2024-05-17T22:02:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- start start 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 0x7f6b60153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600cf990 0x7f6b600cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60162130 con 0x7f6b600ca730 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60161e50 con 0x7f6b601f48b0 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6015d9b0 con 0x7f6b600cf990 2024-05-17T22:02:50.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33880/0 (socket says 172.21.15.83:33880) 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 -- 172.21.15.83:0/1293891418 learned_addr learned my addr 172.21.15.83:0/1293891418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 0x7f6b60153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 -- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600cf990 msgr2=0x7f6b600cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600cf990 0x7f6b600cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 -- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 msgr2=0x7f6b60153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 0x7f6b60153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 -- 172.21.15.83:0/1293891418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b600d02e0 con 0x7f6b600ca730 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 0x7f6b60153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b6643f640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ba00 tx=0x7f6b4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800b600 con 0x7f6b600ca730 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b58000e50 con 0x7f6b600ca730 2024-05-17T22:02:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601f3af0 con 0x7f6b600ca730 2024-05-17T22:02:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58004420 con 0x7f6b600ca730 2024-05-17T22:02:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.050+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601f40b0 con 0x7f6b600ca730 2024-05-17T22:02:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b60072cb0 con 0x7f6b600ca730 2024-05-17T22:02:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6b580be030 con 0x7f6b600ca730 2024-05-17T22:02:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b567fc640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 0x7f6b44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 0x7f6b44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7f6b581294c0 con 0x7f6b600ca730 2024-05-17T22:02:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.054+0000 7f6b65c3e640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 0x7f6b44071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6b6015cca0 tx=0x7f6b5c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.058+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6b600d02e0 con 0x7f6b600ca730 2024-05-17T22:02:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b600d3130 con 0x7f6b600ca730 2024-05-17T22:02:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b567fc640 1 -- 172.21.15.83:0/1293891418 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+3872 (secure 0 0 0) 0x7f6b580a7070 con 0x7f6b600ca730 2024-05-17T22:02:50.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:50.247 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/105138085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":105138085},{"type":"v1","addr":"172.21.15.155:6834","nonce":105138085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2926959040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2926959040},{"type":"v1","addr":"172.21.15.155:6835","nonce":2926959040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3483615874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3483615874},{"type":"v1","addr":"172.21.15.83:6835","nonce":3483615874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4195682811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":4195682811},{"type":"v1","addr":"172.21.15.83:6837","nonce":4195682811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53}],"filesystems":[]} 2024-05-17T22:02:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 msgr2=0x7f6b44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 0x7f6b44071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6b6015cca0 tx=0x7f6b5c00b040 comp rx=0 tx=0).stop 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 msgr2=0x7f6b600cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ba00 tx=0x7f6b4c005840 comp rx=0 tx=0).stop 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 shutdown_connections 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b600ca730 0x7f6b600cab40 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b600cf990 0x7f6b600cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6b4406f400 0x7f6b44071910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 --2- 172.21.15.83:0/1293891418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b601f48b0 0x7f6b60153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.254+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 >> 172.21.15.83:0/1293891418 conn(0x7f6b600b5250 msgr2=0x7f6b600b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.258+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 shutdown_connections 2024-05-17T22:02:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.258+0000 7f6b67ea2640 1 -- 172.21.15.83:0/1293891418 wait complete. 2024-05-17T22:02:50.269 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T22:02:50.269 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T22:02:50.269 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T22:02:50.269 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:02:50.270 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T22:02:50.270 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T22:02:50.270 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T22:02:50.270 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:02:50.270 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T22:02:50.270 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T22:02:50.270 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T22:02:50.270 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:02:50.271 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T22:02:50.271 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T22:02:50.271 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T22:02:50.271 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:02:50.287 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T22:02:50.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T22:02:50.289 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T22:02:50.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T22:02:50.295 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T22:02:50.295 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T22:02:50.297 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T22:02:50.297 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T22:02:50.334 INFO:tasks.ceph.mds.c:Started 2024-05-17T22:02:50.334 DEBUG:teuthology.parallel:result is None 2024-05-17T22:02:50.335 INFO:tasks.ceph.mds.a:Started 2024-05-17T22:02:50.335 DEBUG:teuthology.parallel:result is None 2024-05-17T22:02:50.340 INFO:tasks.ceph.mds.d:Started 2024-05-17T22:02:50.340 DEBUG:teuthology.parallel:result is None 2024-05-17T22:02:50.341 INFO:tasks.ceph.mds.b:Started 2024-05-17T22:02:50.341 DEBUG:teuthology.parallel:result is None 2024-05-17T22:02:50.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T22:02:50.427 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:02:50.428 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:02:50.441 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T22:02:50.443 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T22:02:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.486+0000 7efe2c5f7640 1 Processor -- start 2024-05-17T22:02:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2c5f7640 1 -- start start 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2c5f7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efe240b7d90 0x7efe2406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2c5f7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe240ce450 con 0x7efe240b4bb0 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2c5f7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe240ce730 con 0x7efe240b3be0 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2c5f7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efe240cea10 con 0x7efe240b7d90 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29b92640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efe240b4bb0 0x7efe240cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59594/0 (socket says 172.21.15.83:59594) 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29b92640 1 -- 172.21.15.83:0/1482033239 learned_addr learned my addr 172.21.15.83:0/1482033239 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2ab94640 1 --2- 172.21.15.83:0/1482033239 >> v2:172.21.15.155:3300/0 conn(0x7efe240b7d90 0x7efe2406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237235031 0 0) 0x7efe240ce450 con 0x7efe240b4bb0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe10003530 con 0x7efe240b4bb0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949436865 0 0) 0x7efe240ce730 con 0x7efe240b3be0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe240ce450 con 0x7efe240b3be0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2537412663 0 0) 0x7efe10003530 con 0x7efe240b4bb0 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2ab94640 1 -- 172.21.15.83:0/1482033239 >> v1:172.21.15.155:6789/0 conn(0x7efe240b3be0 legacy=0x7efe240af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2ab94640 1 -- 172.21.15.83:0/1482033239 >> v1:172.21.15.83:6789/0 conn(0x7efe240b4bb0 legacy=0x7efe240cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.490+0000 7efe2ab94640 1 -- 172.21.15.83:0/1482033239 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe240cecf0 con 0x7efe240b7d90 2024-05-17T22:02:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2ab94640 1 --2- 172.21.15.83:0/1482033239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b7d90 0x7efe2406c790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009d90 tx=0x7efe1c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74cbaafffb97435 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14001700 con 0x7efe240b7d90 2024-05-17T22:02:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c0a64e0 con 0x7efe240b7d90 2024-05-17T22:02:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe29391640 1 -- 172.21.15.83:0/1482033239 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0a68b0 con 0x7efe240b7d90 2024-05-17T22:02:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1482033239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b7d90 msgr2=0x7efe2406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1482033239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b7d90 0x7efe2406c790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009d90 tx=0x7efe1c01c890 comp rx=0 tx=0).stop 2024-05-17T22:02:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1482033239 shutdown_connections 2024-05-17T22:02:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1482033239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b7d90 0x7efe2406c790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1482033239 >> 172.21.15.83:0/1482033239 conn(0x7efe240b4090 msgr2=0x7efe240b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1482033239 shutdown_connections 2024-05-17T22:02:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1482033239 wait complete. 2024-05-17T22:02:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 Processor -- start 2024-05-17T22:02:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.494+0000 7efe2c5f7640 1 -- start start 2024-05-17T22:02:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 0x7efe2415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b4bb0 0x7efe24152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cfd80 con 0x7efe240b7d90 2024-05-17T22:02:50.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cfc00 con 0x7efe240b4bb0 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe2416bf40 con 0x7efe240b3be0 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33910/0 (socket says 172.21.15.83:33910) 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 -- 172.21.15.83:0/1555344777 learned_addr learned my addr 172.21.15.83:0/1555344777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2a393640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 0x7efe2415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe29b92640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b4bb0 0x7efe24152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 -- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 msgr2=0x7efe2415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 0x7efe2415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 -- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b4bb0 msgr2=0x7efe24152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b4bb0 0x7efe24152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 -- 172.21.15.83:0/1555344777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2415de60 con 0x7efe240b7d90 2024-05-17T22:02:50.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2a393640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 0x7efe2415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2ab94640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0b0040 tx=0x7efe1c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0c2070 con 0x7efe240b7d90 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c003ba0 con 0x7efe240b7d90 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe240c61d0 con 0x7efe240b7d90 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c009450 con 0x7efe240b7d90 2024-05-17T22:02:50.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe240c6790 con 0x7efe240b7d90 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2406c790 con 0x7efe240b7d90 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe1c004030 con 0x7efe240b7d90 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 0x7efe08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2a393640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 0x7efe08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5029+0+0 (secure 0 0 0) 0x7efe1c129400 con 0x7efe240b7d90 2024-05-17T22:02:50.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.498+0000 7efe2a393640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 0x7efe08071910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe20001130 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:50.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.502+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efe1c016480 con 0x7efe240b7d90 2024-05-17T22:02:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:50.666+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7efe240b4640 con 0x7efe240b7d90 2024-05-17T22:02:51.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.350+0000 7efe1affd640 1 -- 172.21.15.83:0/1555344777 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7efe1c0f6590 con 0x7efe240b7d90 2024-05-17T22:02:51.343 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 msgr2=0x7efe08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 0x7efe08071910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efe20001130 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 msgr2=0x7efe24070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0b0040 tx=0x7efe1c019630 comp rx=0 tx=0).stop 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 shutdown_connections 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe240b7d90 0x7efe24070030 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe0806f400 0x7efe08071910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe240b4bb0 0x7efe24152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 --2- 172.21.15.83:0/1555344777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe240b3be0 0x7efe2415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 >> 172.21.15.83:0/1555344777 conn(0x7efe240b4090 msgr2=0x7efe2415a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 shutdown_connections 2024-05-17T22:02:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.354+0000 7efe2c5f7640 1 -- 172.21.15.83:0/1555344777 wait complete. 2024-05-17T22:02:51.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T22:02:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 Processor -- start 2024-05-17T22:02:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 -- start start 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba400cf2e0 0x7fba4009b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba40072d70 0x7fba4009bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba400d1950 con 0x7fba4009c570 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fba400d1670 con 0x7fba400cf2e0 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba46bea640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fba40071280 con 0x7fba40072d70 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba400cf2e0 0x7fba4009b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba400cf2e0 0x7fba4009b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33926/0 (socket says 172.21.15.83:33926) 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 -- 172.21.15.83:0/2907391948 learned_addr learned my addr 172.21.15.83:0/2907391948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba45187640 1 --2- 172.21.15.83:0/2907391948 >> v2:172.21.15.155:3300/0 conn(0x7fba40072d70 0x7fba4009bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 -- 172.21.15.83:0/2907391948 >> v2:172.21.15.155:3300/0 conn(0x7fba40072d70 msgr2=0x7fba4009bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 --2- 172.21.15.83:0/2907391948 >> v2:172.21.15.155:3300/0 conn(0x7fba40072d70 0x7fba4009bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 -- 172.21.15.83:0/2907391948 >> v1:172.21.15.155:6789/0 conn(0x7fba4009c570 legacy=0x7fba400ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.526+0000 7fba44986640 1 -- 172.21.15.83:0/2907391948 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba40071560 con 0x7fba400cf2e0 2024-05-17T22:02:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba44986640 1 --2- 172.21.15.83:0/2907391948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba400cf2e0 0x7fba4009b990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fba400b6030 tx=0x7fba3401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e247e89e34d5d1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba3f7fe640 1 -- 172.21.15.83:0/2907391948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340b5070 con 0x7fba400cf2e0 2024-05-17T22:02:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba3f7fe640 1 -- 172.21.15.83:0/2907391948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba340a89d0 con 0x7fba400cf2e0 2024-05-17T22:02:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba3f7fe640 1 -- 172.21.15.83:0/2907391948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340a8da0 con 0x7fba400cf2e0 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 -- 172.21.15.83:0/2907391948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba400cf2e0 msgr2=0x7fba4009b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 --2- 172.21.15.83:0/2907391948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba400cf2e0 0x7fba4009b990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fba400b6030 tx=0x7fba3401ed10 comp rx=0 tx=0).stop 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 -- 172.21.15.83:0/2907391948 shutdown_connections 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 --2- 172.21.15.83:0/2907391948 >> v2:172.21.15.155:3300/0 conn(0x7fba40072d70 0x7fba4009bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 --2- 172.21.15.83:0/2907391948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba400cf2e0 0x7fba4009b990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 -- 172.21.15.83:0/2907391948 >> 172.21.15.83:0/2907391948 conn(0x7fba400b5120 msgr2=0x7fba400b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 -- 172.21.15.83:0/2907391948 shutdown_connections 2024-05-17T22:02:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 -- 172.21.15.83:0/2907391948 wait complete. 2024-05-17T22:02:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.530+0000 7fba46bea640 1 Processor -- start 2024-05-17T22:02:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- start start 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba4009c570 0x7fba400c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba400cf2e0 0x7fba400b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba400d1380 con 0x7fba40072d70 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fba400d1200 con 0x7fba4009c570 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fba400b8de0 con 0x7fba400cf2e0 2024-05-17T22:02:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33938/0 (socket says 172.21.15.83:33938) 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 -- 172.21.15.83:0/2896889570 learned_addr learned my addr 172.21.15.83:0/2896889570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba3ffff640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba4009c570 0x7fba400c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 -- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba400cf2e0 msgr2=0x7fba400b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba400cf2e0 0x7fba400b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 -- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba4009c570 msgr2=0x7fba400c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba4009c570 0x7fba400c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 -- 172.21.15.83:0/2896889570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba400c6cc0 con 0x7fba40072d70 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba44986640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fba340b3040 tx=0x7fba3401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340c0070 con 0x7fba40072d70 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba340b0820 con 0x7fba40072d70 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba400cd060 con 0x7fba40072d70 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340b5040 con 0x7fba40072d70 2024-05-17T22:02:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba400cd620 con 0x7fba40072d70 2024-05-17T22:02:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.534+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4009b990 con 0x7fba40072d70 2024-05-17T22:02:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.538+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fba340c3070 con 0x7fba40072d70 2024-05-17T22:02:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.538+0000 7fba3d7fa640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 0x7fba2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.538+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7fba3412bda0 con 0x7fba40072d70 2024-05-17T22:02:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.538+0000 7fba3ffff640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 0x7fba2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.538+0000 7fba3ffff640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 0x7fba2c071910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba400b89f0 tx=0x7fba30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.542+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fba34018870 con 0x7fba40072d70 2024-05-17T22:02:51.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.758+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fba400b67b0 con 0x7fba40072d70 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba3d7fa640 1 -- 172.21.15.83:0/2896889570 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fba340f8fe0 con 0x7fba40072d70 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T22:02:51.753 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:02:51.754 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.759 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T22:02:51.761 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:02:51.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T22:02:51.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:02:51.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T22:02:51.765 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T22:02:51.766 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 msgr2=0x7fba2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 0x7fba2c071910 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba400b89f0 tx=0x7fba30008040 comp rx=0 tx=0).stop 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 msgr2=0x7fba400cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fba340b3040 tx=0x7fba3401bab0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 shutdown_connections 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba400cf2e0 0x7fba400b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fba2c06f400 0x7fba2c071910 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba4009c570 0x7fba400c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 --2- 172.21.15.83:0/2896889570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba40072d70 0x7fba400cc6f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 >> 172.21.15.83:0/2896889570 conn(0x7fba400b5120 msgr2=0x7fba40160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 shutdown_connections 2024-05-17T22:02:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.762+0000 7fba46bea640 1 -- 172.21.15.83:0/2896889570 wait complete. 2024-05-17T22:02:51.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T22:02:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.930+0000 7f76e5e49640 1 Processor -- start 2024-05-17T22:02:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.930+0000 7f76e5e49640 1 -- start start 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76e5e49640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e015d2f0 0x7f76e0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76e5e49640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f76e0073540 0x7f76e0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76e5e49640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e015c830 con 0x7f76e015fe20 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76e5e49640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f76e01608a0 con 0x7f76e015d2f0 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76e5e49640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f76e0160b80 con 0x7f76e0073540 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e015d2f0 0x7f76e0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76e015d2f0 0x7f76e0159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33952/0 (socket says 172.21.15.83:33952) 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 -- 172.21.15.83:0/3360571112 learned_addr learned my addr 172.21.15.83:0/3360571112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76dffff640 1 --2- 172.21.15.83:0/3360571112 >> v2:172.21.15.155:3300/0 conn(0x7f76e0073540 0x7f76e0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 -- 172.21.15.83:0/3360571112 >> v2:172.21.15.155:3300/0 conn(0x7f76e0073540 msgr2=0x7f76e0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 --2- 172.21.15.83:0/3360571112 >> v2:172.21.15.155:3300/0 conn(0x7f76e0073540 0x7f76e0160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 -- 172.21.15.83:0/3360571112 >> v1:172.21.15.155:6789/0 conn(0x7f76e015fe20 legacy=0x7f76e0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:51.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.934+0000 7f76df7fe640 1 -- 172.21.15.83:0/3360571112 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e015caf0 con 0x7f76e015d2f0 2024-05-17T22:02:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76df7fe640 1 --2- 172.21.15.83:0/3360571112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015d2f0 0x7f76e0159c20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f76e006d630 tx=0x7f76d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7230d73400d1fc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76de7fc640 1 -- 172.21.15.83:0/3360571112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d80b0070 con 0x7f76e015d2f0 2024-05-17T22:02:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76de7fc640 1 -- 172.21.15.83:0/3360571112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d80ab600 con 0x7f76e015d2f0 2024-05-17T22:02:51.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76de7fc640 1 -- 172.21.15.83:0/3360571112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d80ab9d0 con 0x7f76e015d2f0 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- 172.21.15.83:0/3360571112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015d2f0 msgr2=0x7f76e0159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 --2- 172.21.15.83:0/3360571112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015d2f0 0x7f76e0159c20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f76e006d630 tx=0x7f76d801c490 comp rx=0 tx=0).stop 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- 172.21.15.83:0/3360571112 shutdown_connections 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 --2- 172.21.15.83:0/3360571112 >> v2:172.21.15.155:3300/0 conn(0x7f76e0073540 0x7f76e0160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 --2- 172.21.15.83:0/3360571112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015d2f0 0x7f76e0159c20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- 172.21.15.83:0/3360571112 >> 172.21.15.83:0/3360571112 conn(0x7f76e006cd90 msgr2=0x7f76e00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- 172.21.15.83:0/3360571112 shutdown_connections 2024-05-17T22:02:51.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- 172.21.15.83:0/3360571112 wait complete. 2024-05-17T22:02:51.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 Processor -- start 2024-05-17T22:02:51.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.938+0000 7f76e5e49640 1 -- start start 2024-05-17T22:02:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76e015d2f0 0x7f76e00b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76e015b1f0 0x7f76e014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76e00b04f0 con 0x7f76e015fe20 2024-05-17T22:02:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76e00b0370 con 0x7f76e015b1f0 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76e00b0670 con 0x7f76e015d2f0 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33962/0 (socket says 172.21.15.83:33962) 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 -- 172.21.15.83:0/2429748123 learned_addr learned my addr 172.21.15.83:0/2429748123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 -- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76e015d2f0 msgr2=0x7f76e00b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76e015d2f0 0x7f76e00b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76dffff640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76e015b1f0 0x7f76e014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 -- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76e015b1f0 msgr2=0x7f76e014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76e015b1f0 0x7f76e014de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 -- 172.21.15.83:0/2429748123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e0148540 con 0x7f76e015fe20 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76deffd640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b3f0 tx=0x7f76c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8015020 con 0x7f76e015fe20 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c8011830 con 0x7f76e015fe20 2024-05-17T22:02:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e0148820 con 0x7f76e015fe20 2024-05-17T22:02:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011c00 con 0x7f76e015fe20 2024-05-17T22:02:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e015f1a0 con 0x7f76e015fe20 2024-05-17T22:02:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e0160230 con 0x7f76e015fe20 2024-05-17T22:02:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76c8018020 con 0x7f76e015fe20 2024-05-17T22:02:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 0x7f76c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f76c808ebc0 con 0x7f76e015fe20 2024-05-17T22:02:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76df7fe640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 0x7f76c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.942+0000 7f76df7fe640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 0x7f76c0071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76d80ad000 tx=0x7f76d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:51.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:51.946+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f76c801f030 con 0x7f76e015fe20 2024-05-17T22:02:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f76e014fbc0 con 0x7f76e015fe20 2024-05-17T22:02:52.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76d7fff640 1 -- 172.21.15.83:0/2429748123 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f76c805be90 con 0x7f76e015fe20 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:02:52.091 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 msgr2=0x7f76c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 0x7f76c0071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f76d80ad000 tx=0x7f76d8008040 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 msgr2=0x7f76e015acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b3f0 tx=0x7f76c800b8b0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 shutdown_connections 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76c006f400 0x7f76c0071910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76e015b1f0 0x7f76e014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76e015fe20 0x7f76e015acb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 --2- 172.21.15.83:0/2429748123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76e015d2f0 0x7f76e00b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 >> 172.21.15.83:0/2429748123 conn(0x7f76e006cd90 msgr2=0x7f76e00d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 shutdown_connections 2024-05-17T22:02:52.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.098+0000 7f76e5e49640 1 -- 172.21.15.83:0/2429748123 wait complete. 2024-05-17T22:02:52.112 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T22:02:52.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T22:02:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.266+0000 7fa7895d6640 1 Processor -- start 2024-05-17T22:02:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.266+0000 7fa7895d6640 1 -- start start 2024-05-17T22:02:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7895d6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7841570e0 0x7fa7840b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7895d6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7840d0820 con 0x7fa7840b4f20 2024-05-17T22:02:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7895d6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7840d0b00 con 0x7fa784160420 2024-05-17T22:02:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7895d6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840b4a90 con 0x7fa7841570e0 2024-05-17T22:02:52.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7841570e0 0x7fa7840b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7837fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa7840b4f20 0x7fa7840b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59606/0 (socket says 172.21.15.83:59606) 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7841570e0 0x7fa7840b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33968/0 (socket says 172.21.15.83:33968) 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7837fe640 1 -- 172.21.15.83:0/1305850310 learned_addr learned my addr 172.21.15.83:0/1305850310 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 -- 172.21.15.83:0/1305850310 >> v1:172.21.15.155:6789/0 conn(0x7fa784160420 legacy=0x7fa784160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 -- 172.21.15.83:0/1305850310 >> v1:172.21.15.83:6789/0 conn(0x7fa7840b4f20 legacy=0x7fa7840b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 -- 172.21.15.83:0/1305850310 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7840d0de0 con 0x7fa7841570e0 2024-05-17T22:02:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7827fc640 1 -- 172.21.15.83:0/1305850310 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435829833 0 0) 0x7fa7840d0820 con 0x7fa7840b4f20 2024-05-17T22:02:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa782ffd640 1 --2- 172.21.15.83:0/1305850310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7841570e0 0x7fa7840b3a60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa77401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f43ce36bc5ce95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7827fc640 1 -- 172.21.15.83:0/1305850310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b2070 con 0x7fa7841570e0 2024-05-17T22:02:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7827fc640 1 -- 172.21.15.83:0/1305850310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7740ad640 con 0x7fa7841570e0 2024-05-17T22:02:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.270+0000 7fa7827fc640 1 -- 172.21.15.83:0/1305850310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7840b6250 con 0x7fa7841570e0 2024-05-17T22:02:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- 172.21.15.83:0/1305850310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7841570e0 msgr2=0x7fa7840b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1305850310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7841570e0 0x7fa7840b3a60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa77401c470 comp rx=0 tx=0).stop 2024-05-17T22:02:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- 172.21.15.83:0/1305850310 shutdown_connections 2024-05-17T22:02:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1305850310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7841570e0 0x7fa7840b3a60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- 172.21.15.83:0/1305850310 >> 172.21.15.83:0/1305850310 conn(0x7fa78406cd90 msgr2=0x7fa7840b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- 172.21.15.83:0/1305850310 shutdown_connections 2024-05-17T22:02:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- 172.21.15.83:0/1305850310 wait complete. 2024-05-17T22:02:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 Processor -- start 2024-05-17T22:02:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- start start 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 0x7fa784151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 0x7fa78415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa78416a5c0 con 0x7fa784160420 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7840d05c0 con 0x7fa7841570e0 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7895d6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa78416a740 con 0x7fa7840b4f20 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33980/0 (socket says 172.21.15.83:33980) 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.274+0000 7fa7837fe640 1 -- 172.21.15.83:0/1010237938 learned_addr learned my addr 172.21.15.83:0/1010237938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa782ffd640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 0x7fa784151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77bfff640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 0x7fa78415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 -- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 msgr2=0x7fa784151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 0x7fa784151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 -- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 msgr2=0x7fa78415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 0x7fa78415b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 -- 172.21.15.83:0/1010237938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784151fd0 con 0x7fa784160420 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa782ffd640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 0x7fa784151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77bfff640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 0x7fa78415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7837fe640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c680 tx=0x7fa77c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c018070 con 0x7fa784160420 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77c004d70 con 0x7fa784160420 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00a880 con 0x7fa784160420 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7841522b0 con 0x7fa784160420 2024-05-17T22:02:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa784148420 con 0x7fa784160420 2024-05-17T22:02:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7840b3a60 con 0x7fa784160420 2024-05-17T22:02:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa77c01b070 con 0x7fa784160420 2024-05-17T22:02:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa782ffd640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7fa77c08e360 con 0x7fa784160420 2024-05-17T22:02:52.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.278+0000 7fa782ffd640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa7740af000 tx=0x7fa774008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.282+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa77c05b6c0 con 0x7fa784160420 2024-05-17T22:02:52.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:52.446+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa78415c8c0 con 0x7fa784160420 2024-05-17T22:02:53.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.358+0000 7fa77b7fe640 1 -- 172.21.15.83:0/1010237938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fa77c05afc0 con 0x7fa784160420 2024-05-17T22:02:53.350 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T22:02:53.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 msgr2=0x7fa764071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa7740af000 tx=0x7fa774008040 comp rx=0 tx=0).stop 2024-05-17T22:02:53.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 msgr2=0x7fa78415be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00c680 tx=0x7fa77c00cb40 comp rx=0 tx=0).stop 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 shutdown_connections 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa784160420 0x7fa78415be40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7841570e0 0x7fa78415b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 --2- 172.21.15.83:0/1010237938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7840b4f20 0x7fa784151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 >> 172.21.15.83:0/1010237938 conn(0x7fa78406cd90 msgr2=0x7fa784072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 shutdown_connections 2024-05-17T22:02:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.362+0000 7fa7895d6640 1 -- 172.21.15.83:0/1010237938 wait complete. 2024-05-17T22:02:53.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T22:02:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 Processor -- start 2024-05-17T22:02:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 -- start start 2024-05-17T22:02:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd2c145fd0 0x7fdd2c06eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c0737d0 con 0x7fdd2c0b4610 2024-05-17T22:02:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c0b5130 con 0x7fdd2c145fd0 2024-05-17T22:02:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd31007640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c1463e0 con 0x7fdd2c072e80 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd2c145fd0 0x7fdd2c06eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdd2c145fd0 0x7fdd2c06eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:54110/0 (socket says 172.21.15.83:54110) 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 -- 172.21.15.83:0/89026369 learned_addr learned my addr 172.21.15.83:0/89026369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2b59e640 1 --1- 172.21.15.83:0/89026369 >> v1:172.21.15.155:6789/0 conn(0x7fdd2c0b4610 0x7fdd2c0b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:41346/0 (socket says 172.21.15.83:41346) 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2a59c640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 -- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 msgr2=0x7fdd2c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 -- 172.21.15.83:0/89026369 >> v1:172.21.15.155:6789/0 conn(0x7fdd2c0b4610 legacy=0x7fdd2c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.558+0000 7fdd2ad9d640 1 -- 172.21.15.83:0/89026369 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0b5360 con 0x7fdd2c145fd0 2024-05-17T22:02:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd2ad9d640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c06eec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009870 tx=0x7fdd1c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8eb7ce99500f694 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd29d9b640 1 -- 172.21.15.83:0/89026369 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0a9620 con 0x7fdd2c145fd0 2024-05-17T22:02:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd29d9b640 1 -- 172.21.15.83:0/89026369 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0a97c0 con 0x7fdd2c145fd0 2024-05-17T22:02:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd29d9b640 1 -- 172.21.15.83:0/89026369 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0a9b90 con 0x7fdd2c145fd0 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 -- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c145fd0 msgr2=0x7fdd2c06eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c06eec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009870 tx=0x7fdd1c01c560 comp rx=0 tx=0).stop 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 -- 172.21.15.83:0/89026369 shutdown_connections 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 --2- 172.21.15.83:0/89026369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c06eec0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 -- 172.21.15.83:0/89026369 >> 172.21.15.83:0/89026369 conn(0x7fdd2c06cd90 msgr2=0x7fdd2c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.562+0000 7fdd31007640 1 -- 172.21.15.83:0/89026369 shutdown_connections 2024-05-17T22:02:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 -- 172.21.15.83:0/89026369 wait complete. 2024-05-17T22:02:53.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 Processor -- start 2024-05-17T22:02:53.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 -- start start 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c0b4610 0x7fdd2c1fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0b0670 con 0x7fdd2c145fd0 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0af9c0 con 0x7fdd2c0b4610 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd31007640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c0b07f0 con 0x7fdd2c072e80 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33994/0 (socket says 172.21.15.83:33994) 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 -- 172.21.15.83:0/1023262215 learned_addr learned my addr 172.21.15.83:0/1023262215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2a59c640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c0b4610 0x7fdd2c1fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2ad9d640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 -- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 msgr2=0x7fdd2c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 -- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c0b4610 msgr2=0x7fdd2c1fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:53.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c0b4610 0x7fdd2c1fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:53.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.566+0000 7fdd2b59e640 1 -- 172.21.15.83:0/1023262215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0cea30 con 0x7fdd2c145fd0 2024-05-17T22:02:53.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd2b59e640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c730 tx=0x7fdd2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24017070 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2400ce40 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400a860 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c0ced10 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c0cf2d0 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c1fd0a0 con 0x7fdd2c145fd0 2024-05-17T22:02:53.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdd2401a070 con 0x7fdd2c145fd0 2024-05-17T22:02:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 0x7fdd18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd2ad9d640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 0x7fdd18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5227+0+0 (secure 0 0 0) 0x7fdd2408e650 con 0x7fdd2c145fd0 2024-05-17T22:02:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.570+0000 7fdd2ad9d640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 0x7fdd18071910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ae000 tx=0x7fdd1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:53.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.574+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdd2c0737d0 con 0x7fdd2c145fd0 2024-05-17T22:02:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:53.734+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdd2c0cf7f0 con 0x7fdd2c145fd0 2024-05-17T22:02:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.362+0000 7fdd137fe640 1 -- 172.21.15.83:0/1023262215 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7fdd2405b0d0 con 0x7fdd2c145fd0 2024-05-17T22:02:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T22:02:54.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 msgr2=0x7fdd18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 0x7fdd18071910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ae000 tx=0x7fdd1c009040 comp rx=0 tx=0).stop 2024-05-17T22:02:54.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 msgr2=0x7fdd2c1fc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400c730 tx=0x7fdd2400cbf0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 shutdown_connections 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd2c145fd0 0x7fdd2c1fc860 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdd1806f400 0x7fdd18071910 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd2c0b4610 0x7fdd2c1fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 --2- 172.21.15.83:0/1023262215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd2c072e80 0x7fdd2c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 >> 172.21.15.83:0/1023262215 conn(0x7fdd2c06cd90 msgr2=0x7fdd2c0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 shutdown_connections 2024-05-17T22:02:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.366+0000 7fdd31007640 1 -- 172.21.15.83:0/1023262215 wait complete. 2024-05-17T22:02:54.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T22:02:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.558+0000 7f7a94ac7640 1 Processor -- start 2024-05-17T22:02:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.558+0000 7f7a94ac7640 1 -- start start 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a94ac7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a900d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a94ac7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a90073590 0x7f7a901447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a94ac7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a900d10d0 con 0x7f7a90143f20 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a94ac7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7a900b4110 con 0x7f7a90073590 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a94ac7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a90070770 con 0x7f7a900b3d00 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a90073590 0x7f7a901447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a90143f20 0x7f7a90072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59614/0 (socket says 172.21.15.83:59614) 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a90073590 0x7f7a901447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34006/0 (socket says 172.21.15.83:34006) 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8dd9b640 1 -- 172.21.15.83:0/1916372267 learned_addr learned my addr 172.21.15.83:0/1916372267 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8e59c640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a900d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/1916372267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 msgr2=0x7f7a900d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a900d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/1916372267 >> v1:172.21.15.83:6789/0 conn(0x7f7a90143f20 legacy=0x7f7a90072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/1916372267 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a90070120 con 0x7f7a90073590 2024-05-17T22:02:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8d59a640 1 -- 172.21.15.83:0/1916372267 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612275217 0 0) 0x7f7a900d10d0 con 0x7f7a90143f20 2024-05-17T22:02:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.562+0000 7f7a8ed9d640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90073590 0x7f7a901447f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009d90 tx=0x7f7a8801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a77afb745bfffb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a8d59a640 1 -- 172.21.15.83:0/1916372267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880b2070 con 0x7f7a90073590 2024-05-17T22:02:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a8d59a640 1 -- 172.21.15.83:0/1916372267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a880a58c0 con 0x7f7a90073590 2024-05-17T22:02:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a8d59a640 1 -- 172.21.15.83:0/1916372267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880a5c90 con 0x7f7a90073590 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- 172.21.15.83:0/1916372267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90073590 msgr2=0x7f7a901447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90073590 0x7f7a901447f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009d90 tx=0x7f7a8801c970 comp rx=0 tx=0).stop 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- 172.21.15.83:0/1916372267 shutdown_connections 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90073590 0x7f7a901447f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/1916372267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a900d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- 172.21.15.83:0/1916372267 >> 172.21.15.83:0/1916372267 conn(0x7f7a9006cd90 msgr2=0x7f7a900b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- 172.21.15.83:0/1916372267 shutdown_connections 2024-05-17T22:02:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- 172.21.15.83:0/1916372267 wait complete. 2024-05-17T22:02:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 Processor -- start 2024-05-17T22:02:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.566+0000 7f7a94ac7640 1 -- start start 2024-05-17T22:02:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a90073590 0x7f7a9014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a9015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90070c00 con 0x7f7a90143f20 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90145540 con 0x7f7a90073590 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9015c540 con 0x7f7a900b3d00 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34016/0 (socket says 172.21.15.83:34016) 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/4167197825 learned_addr learned my addr 172.21.15.83:0/4167197825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 msgr2=0x7f7a9015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a9015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a90073590 msgr2=0x7f7a9014d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8e59c640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a90073590 0x7f7a9014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a90073590 0x7f7a9014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 -- 172.21.15.83:0/4167197825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a90142290 con 0x7f7a90143f20 2024-05-17T22:02:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8ed9d640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009d90 tx=0x7f7a88019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880b2050 con 0x7f7a90143f20 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a880b3040 con 0x7f7a90143f20 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a90142570 con 0x7f7a90143f20 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880ad850 con 0x7f7a90143f20 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a90143050 con 0x7f7a90143f20 2024-05-17T22:02:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a901447f0 con 0x7f7a90143f20 2024-05-17T22:02:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a880c0020 con 0x7f7a90143f20 2024-05-17T22:02:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 0x7f7a70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5609+0+0 (secure 0 0 0) 0x7f7a8812a6d0 con 0x7f7a90143f20 2024-05-17T22:02:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.570+0000 7f7a8e59c640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 0x7f7a70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.574+0000 7f7a8e59c640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 0x7f7a70071910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a800096f0 tx=0x7f7a80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.574+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7a88016560 con 0x7f7a90143f20 2024-05-17T22:02:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:54.774+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7a9015f8f0 con 0x7f7a90143f20 2024-05-17T22:02:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.442+0000 7f7a7effd640 1 -- 172.21.15.83:0/4167197825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v56) ==== 174+0+0 (secure 0 0 0) 0x7f7a880f7700 con 0x7f7a90143f20 2024-05-17T22:02:55.440 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 12 and data pool 13 2024-05-17T22:02:55.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.458+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 msgr2=0x7f7a70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.458+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 0x7f7a70071910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a800096f0 tx=0x7f7a80008040 comp rx=0 tx=0).stop 2024-05-17T22:02:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.458+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 msgr2=0x7f7a9015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.458+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009d90 tx=0x7f7a88019710 comp rx=0 tx=0).stop 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 shutdown_connections 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a90143f20 0x7f7a9015d3a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a900b3d00 0x7f7a9015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a7006f400 0x7f7a70071910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 --2- 172.21.15.83:0/4167197825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a90073590 0x7f7a9014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.462+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 >> 172.21.15.83:0/4167197825 conn(0x7f7a9006cd90 msgr2=0x7f7a900761e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:55.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.466+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 shutdown_connections 2024-05-17T22:02:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.466+0000 7f7a94ac7640 1 -- 172.21.15.83:0/4167197825 wait complete. 2024-05-17T22:02:55.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 Processor -- start 2024-05-17T22:02:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 -- start start 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 0x7f8b2c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b2c09c570 0x7f8b2c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0d1950 con 0x7f8b2c0cf2e0 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8b2c0d1670 con 0x7f8b2c09c570 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3449f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c071280 con 0x7f8b2c09c0a0 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b32a3c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b2c09c570 0x7f8b2c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3223b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8b2c0cf2e0 0x7f8b2c09b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59628/0 (socket says 172.21.15.83:59628) 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b3223b640 1 -- 172.21.15.83:0/2066337114 learned_addr learned my addr 172.21.15.83:0/2066337114 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b31a3a640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 0x7f8b2c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b32a3c640 1 -- 172.21.15.83:0/2066337114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 msgr2=0x7f8b2c071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b32a3c640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 0x7f8b2c071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b31239640 1 -- 172.21.15.83:0/2066337114 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270669348 0 0) 0x7f8b2c0d1950 con 0x7f8b2c0cf2e0 2024-05-17T22:02:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b32a3c640 1 -- 172.21.15.83:0/2066337114 >> v1:172.21.15.83:6789/0 conn(0x7f8b2c0cf2e0 legacy=0x7f8b2c09b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b32a3c640 1 -- 172.21.15.83:0/2066337114 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c071560 con 0x7f8b2c09c570 2024-05-17T22:02:55.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.634+0000 7f8b31a3a640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 0x7f8b2c071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:55.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b32a3c640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0722a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009d90 tx=0x7f8b2401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9207683c007d786 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b31239640 1 -- 172.21.15.83:0/2066337114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240b4070 con 0x7f8b2c09c570 2024-05-17T22:02:55.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b31239640 1 -- 172.21.15.83:0/2066337114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240af600 con 0x7f8b2c09c570 2024-05-17T22:02:55.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b31239640 1 -- 172.21.15.83:0/2066337114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240af9d0 con 0x7f8b2c09c570 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 -- 172.21.15.83:0/2066337114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 msgr2=0x7f8b2c0722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0722a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009d90 tx=0x7f8b2401f150 comp rx=0 tx=0).stop 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 -- 172.21.15.83:0/2066337114 shutdown_connections 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0722a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2066337114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c09c0a0 0x7f8b2c071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 -- 172.21.15.83:0/2066337114 >> 172.21.15.83:0/2066337114 conn(0x7f8b2c0b5120 msgr2=0x7f8b2c0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 -- 172.21.15.83:0/2066337114 shutdown_connections 2024-05-17T22:02:55.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.638+0000 7f8b3449f640 1 -- 172.21.15.83:0/2066337114 wait complete. 2024-05-17T22:02:55.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 Processor -- start 2024-05-17T22:02:55.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- start start 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c0cf2e0 0x7f8b2c1529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 0x7f8b2c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d1380 con 0x7f8b2c09c570 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0d1200 con 0x7f8b2c15ec20 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c0b8de0 con 0x7f8b2c0cf2e0 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34044/0 (socket says 172.21.15.83:34044) 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 -- 172.21.15.83:0/2854592574 learned_addr learned my addr 172.21.15.83:0/2854592574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b31a3a640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c0cf2e0 0x7f8b2c1529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b32a3c640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 0x7f8b2c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 -- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c0cf2e0 msgr2=0x7f8b2c1529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c0cf2e0 0x7f8b2c1529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 -- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 msgr2=0x7f8b2c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 0x7f8b2c15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 -- 172.21.15.83:0/2854592574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0cba60 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b32a3c640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 0x7f8b2c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3223b640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c460 tx=0x7f8b2800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28018070 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28007500 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c0cbd40 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004a30 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.642+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c15c590 con 0x7f8b2c09c570 2024-05-17T22:02:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c09b990 con 0x7f8b2c09c570 2024-05-17T22:02:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b2801b070 con 0x7f8b2c09c570 2024-05-17T22:02:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b22ffd640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 0x7f8b10071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b31a3a640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 0x7f8b10071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2808e490 con 0x7f8b2c09c570 2024-05-17T22:02:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.646+0000 7f8b31a3a640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 0x7f8b10071940 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ccd60 tx=0x7f8b1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.650+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8b2805b410 con 0x7f8b2c09c570 2024-05-17T22:02:55.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.814+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b2c15ca30 con 0x7f8b2c09c570 2024-05-17T22:02:55.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.814+0000 7f8b22ffd640 1 -- 172.21.15.83:0/2854592574 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16649 (secure 0 0 0) 0x7f8b2805ad10 con 0x7f8b2c09c570 2024-05-17T22:02:55.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:55.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:02:54.779759+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:02:52.464989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-17T22:02:53.753100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 msgr2=0x7f8b10071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 0x7f8b10071940 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ccd60 tx=0x7f8b1c008040 comp rx=0 tx=0).stop 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 msgr2=0x7f8b2c0cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c460 tx=0x7f8b2800c920 comp rx=0 tx=0).stop 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 shutdown_connections 2024-05-17T22:02:55.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8b1006f430 0x7f8b10071940 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b2c15ec20 0x7f8b2c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b2c0cf2e0 0x7f8b2c1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 --2- 172.21.15.83:0/2854592574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b2c09c570 0x7f8b2c0cb0c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 >> 172.21.15.83:0/2854592574 conn(0x7f8b2c0b5120 msgr2=0x7f8b2c0ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 shutdown_connections 2024-05-17T22:02:55.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.818+0000 7f8b3449f640 1 -- 172.21.15.83:0/2854592574 wait complete. 2024-05-17T22:02:55.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 Processor -- start 2024-05-17T22:02:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 -- start start 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80ec15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f80ec0706e0 0x7f80ec1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec1465d0 con 0x7f80ec15e6b0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f80ec06ef30 con 0x7f80ec0706e0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80f1e4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec06f210 con 0x7f80ec1461c0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80eaffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f80ec15e6b0 0x7f80ec06ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59638/0 (socket says 172.21.15.83:59638) 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80eaffd640 1 -- 172.21.15.83:0/3872017010 learned_addr learned my addr 172.21.15.83:0/3872017010 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80ec15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2066504566 0 0) 0x7f80ec1465d0 con 0x7f80ec15e6b0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d4003530 con 0x7f80ec15e6b0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ebfff640 1 --2- 172.21.15.83:0/3872017010 >> v2:172.21.15.155:3300/0 conn(0x7f80ec0706e0 0x7f80ec1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3013001012 0 0) 0x7f80d4003530 con 0x7f80ec15e6b0 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 msgr2=0x7f80ec15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80ec15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 >> v2:172.21.15.155:3300/0 conn(0x7f80ec0706e0 msgr2=0x7f80ec1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 --2- 172.21.15.83:0/3872017010 >> v2:172.21.15.155:3300/0 conn(0x7f80ec0706e0 0x7f80ec1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec06f4f0 con 0x7f80ec15e6b0 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f80e0001700 con 0x7f80ec15e6b0 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 >> v1:172.21.15.83:6789/0 conn(0x7f80ec15e6b0 legacy=0x7f80ec06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80d4003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80d4003fe0 0x7f80d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80d4004d50 0x7f80d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec06ef30 con 0x7f80d4004d50 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec06f210 con 0x7f80d4003fe0 2024-05-17T22:02:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.990+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80d400bad0 con 0x7f80ec1461c0 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80d4004d50 0x7f80d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80eaffd640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80d4003fe0 0x7f80d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80d4003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 -- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 msgr2=0x7f80d4003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80d4003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 -- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80d4003fe0 msgr2=0x7f80d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80d4003fe0 0x7f80d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 -- 172.21.15.83:0/3872017010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d4003530 con 0x7f80d4004d50 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80eaffd640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80d4003fe0 0x7f80d4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:55.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec1461c0 0x7f80d4003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80d4004d50 0x7f80d4005160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009f90 tx=0x7f80e401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9a3d71c60a4c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b2070 con 0x7f80d4004d50 2024-05-17T22:02:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e40a5c50 con 0x7f80d4004d50 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ea7fc640 1 -- 172.21.15.83:0/3872017010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4009c10 con 0x7f80d4004d50 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 -- 172.21.15.83:0/3872017010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80d4004d50 msgr2=0x7f80d4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/3872017010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80d4004d50 0x7f80d4005160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009f90 tx=0x7f80e401cc40 comp rx=0 tx=0).stop 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80ebfff640 1 -- 172.21.15.83:0/3872017010 reap_dead start 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 -- 172.21.15.83:0/3872017010 shutdown_connections 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 -- 172.21.15.83:0/3872017010 >> 172.21.15.83:0/3872017010 conn(0x7f80ec06cd90 msgr2=0x7f80ec0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 -- 172.21.15.83:0/3872017010 shutdown_connections 2024-05-17T22:02:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.994+0000 7f80f1e4f640 1 -- 172.21.15.83:0/3872017010 wait complete. 2024-05-17T22:02:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 Processor -- start 2024-05-17T22:02:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 -- start start 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 0x7f80ec1e6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec15eac0 0x7f80ec15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec15e170 con 0x7f80ec0b2340 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec06fab0 con 0x7f80ec15a2e0 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80f1e4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec15e2f0 con 0x7f80ec15eac0 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34050/0 (socket says 172.21.15.83:34050) 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 -- 172.21.15.83:0/2831318307 learned_addr learned my addr 172.21.15.83:0/2831318307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80ebfff640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec15eac0 0x7f80ec15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 0x7f80ec1e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 -- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec15eac0 msgr2=0x7f80ec15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec15eac0 0x7f80ec15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 -- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 msgr2=0x7f80ec1e6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 0x7f80ec1e6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 -- 172.21.15.83:0/2831318307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec0b2c90 con 0x7f80ec0b2340 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 0x7f80ec1e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80eaffd640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f70 tx=0x7f80e000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:55.998+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004cc0 con 0x7f80ec0b2340 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e0007110 con 0x7f80ec0b2340 2024-05-17T22:02:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec1e6d50 con 0x7f80ec0b2340 2024-05-17T22:02:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004e60 con 0x7f80ec0b2340 2024-05-17T22:02:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec1e7310 con 0x7f80ec0b2340 2024-05-17T22:02:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec06ffd0 con 0x7f80ec0b2340 2024-05-17T22:02:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f80e001b020 con 0x7f80ec0b2340 2024-05-17T22:02:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80d3fff640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 0x7f80d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 0x7f80d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f80e008f1f0 con 0x7f80ec0b2340 2024-05-17T22:02:55.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.002+0000 7f80eb7fe640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 0x7f80d8071910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0b3990 tx=0x7f80dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:55.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.006+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f80e005c200 con 0x7f80ec0b2340 2024-05-17T22:02:56.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.170+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80ec0b41a0 con 0x7f80ec0b2340 2024-05-17T22:02:56.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.170+0000 7f80d3fff640 1 -- 172.21.15.83:0/2831318307 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16649 (secure 0 0 0) 0x7f80e005bb00 con 0x7f80ec0b2340 2024-05-17T22:02:56.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:56.165 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:02:54.779759+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:02:52.464989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-17T22:02:53.753100+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 msgr2=0x7f80d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 0x7f80d8071910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0b3990 tx=0x7f80dc008040 comp rx=0 tx=0).stop 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 msgr2=0x7f80ec0b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f70 tx=0x7f80e000df70 comp rx=0 tx=0).stop 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 shutdown_connections 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80ec15eac0 0x7f80ec15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80ec0b2340 0x7f80ec0b2750 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f80d806f400 0x7f80d8071910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 --2- 172.21.15.83:0/2831318307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80ec15a2e0 0x7f80ec1e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 >> 172.21.15.83:0/2831318307 conn(0x7f80ec06cd90 msgr2=0x7f80ec15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:56.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 shutdown_connections 2024-05-17T22:02:56.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.174+0000 7f80f1e4f640 1 -- 172.21.15.83:0/2831318307 wait complete. 2024-05-17T22:02:56.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T22:02:56.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 Processor -- start 2024-05-17T22:02:56.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 -- start start 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb77c09ba00 0x7fb77c09be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c09c350 con 0x7fb77c0b88b0 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c09c710 con 0x7fb77c071400 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb7838fb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c0d1a40 con 0x7fb77c09ba00 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb781697640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb77c0b88b0 0x7fb77c0d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59644/0 (socket says 172.21.15.83:59644) 2024-05-17T22:02:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb781697640 1 -- 172.21.15.83:0/2219637710 learned_addr learned my addr 172.21.15.83:0/2219637710 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:56.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb780e96640 1 --2- 172.21.15.83:0/2219637710 >> v2:172.21.15.155:3300/0 conn(0x7fb77c09ba00 0x7fb77c09be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319449108 0 0) 0x7fb7740014a0 con 0x7fb77c0b88b0 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb768003530 con 0x7fb77c0b88b0 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 363097971 0 0) 0x7fb768003530 con 0x7fb77c0b88b0 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.354+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 >> v2:172.21.15.155:3300/0 conn(0x7fb77c09ba00 msgr2=0x7fb77c09be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 --2- 172.21.15.83:0/2219637710 >> v2:172.21.15.155:3300/0 conn(0x7fb77c09ba00 0x7fb77c09be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 >> v1:172.21.15.155:6789/0 conn(0x7fb77c071400 legacy=0x7fb77c071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c09c9f0 con 0x7fb77c0b88b0 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fb774001c50 con 0x7fb77c0b88b0 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 >> v1:172.21.15.83:6789/0 conn(0x7fb77c0b88b0 legacy=0x7fb77c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 0x7fb768003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768004810 0x7fb768004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7680053b0 0x7fb76800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c0d1a40 con 0x7fb768003a60 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800c340 con 0x7fb768004810 2024-05-17T22:02:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb76800c540 con 0x7fb7680053b0 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 0x7fb768003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 -- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7680053b0 msgr2=0x7fb76800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7680053b0 0x7fb76800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 -- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768004810 msgr2=0x7fb768004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb780e96640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768004810 0x7fb768004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768004810 0x7fb768004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 -- 172.21.15.83:0/2219637710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768003530 con 0x7fb768003a60 2024-05-17T22:02:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781e98640 1 -- 172.21.15.83:0/2219637710 reap_dead start 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb781697640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 0x7fb768003ed0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb77400aa00 tx=0x7fb77401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4486afecbb282e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77400a620 con 0x7fb768003a60 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7740ad650 con 0x7fb768003a60 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb773fff640 1 -- 172.21.15.83:0/2219637710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740ada40 con 0x7fb768003a60 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 -- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 msgr2=0x7fb768003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 0x7fb768003ed0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb77400aa00 tx=0x7fb77401d2a0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 -- 172.21.15.83:0/2219637710 shutdown_connections 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 --2- 172.21.15.83:0/2219637710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb768003a60 0x7fb768003ed0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 -- 172.21.15.83:0/2219637710 >> 172.21.15.83:0/2219637710 conn(0x7fb77c0d3620 msgr2=0x7fb77c0d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 -- 172.21.15.83:0/2219637710 shutdown_connections 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.358+0000 7fb7838fb640 1 -- 172.21.15.83:0/2219637710 wait complete. 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 Processor -- start 2024-05-17T22:02:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 -- start start 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768003a60 0x7fb77c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 0x7fb77c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1614b0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c09d040 con 0x7fb768003a60 2024-05-17T22:02:56.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb7838fb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c0c8770 con 0x7fb77c0c7150 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34072/0 (socket says 172.21.15.83:34072) 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 -- 172.21.15.83:0/998026930 learned_addr learned my addr 172.21.15.83:0/998026930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb781697640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768003a60 0x7fb77c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb781e98640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 0x7fb77c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 -- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 msgr2=0x7fb77c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 0x7fb77c0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 -- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768003a60 msgr2=0x7fb77c15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768003a60 0x7fb77c15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 -- 172.21.15.83:0/998026930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c0ca2e0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb781e98640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 0x7fb77c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.362+0000 7fb780e96640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb76c010930 tx=0x7fb76c010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:56.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00f040 con 0x7fb77c06d8a0 2024-05-17T22:02:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c00ebc0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c0cbe30 con 0x7fb77c06d8a0 2024-05-17T22:02:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c01c400 con 0x7fb77c06d8a0 2024-05-17T22:02:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c0cc3f0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c0720b0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb76c0143f0 con 0x7fb77c06d8a0 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 0x7fb760071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7fb76c08f800 con 0x7fb77c06d8a0 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb781697640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 0x7fb760071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.366+0000 7fb781697640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 0x7fb760071940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb7740ad000 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:56.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.370+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb76c018030 con 0x7fb77c06d8a0 2024-05-17T22:02:56.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:56.574+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb77c0b3100 con 0x7fb77c06d8a0 2024-05-17T22:02:57.347 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:02:57.358+0000 7ffa3e3b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:02:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.418+0000 7fb771ffb640 1 -- 172.21.15.83:0/998026930 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) ==== 118+0+0 (secure 0 0 0) 0x7fb76c05c8d0 con 0x7fb77c06d8a0 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 msgr2=0x7fb760071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 0x7fb760071940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb7740ad000 tx=0x7fb774009040 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 msgr2=0x7fb77c0c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb76c010930 tx=0x7fb76c010df0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 shutdown_connections 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb77c0c7150 0x7fb77c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb77c06d8a0 0x7fb77c0c7cf0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb76006f430 0x7fb760071940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 --2- 172.21.15.83:0/998026930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb768003a60 0x7fb77c15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 >> 172.21.15.83:0/998026930 conn(0x7fb77c0d3620 msgr2=0x7fb77c0b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 shutdown_connections 2024-05-17T22:02:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.422+0000 7fb7838fb640 1 -- 172.21.15.83:0/998026930 wait complete. 2024-05-17T22:02:57.443 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T22:02:57.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T22:02:57.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 Processor -- start 2024-05-17T22:02:57.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 -- start start 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00b406eec0 0x7f00b40b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f00b40b44c0 0x7f00b40b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b406f2d0 con 0x7f00b415be70 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f00b406f5a0 con 0x7f00b40b44c0 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b94d1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f00b406f880 con 0x7f00b406eec0 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f00b40b44c0 0x7f00b40b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f00b40b44c0 0x7f00b40b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60516/0 (socket says 172.21.15.83:60516) 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.606+0000 7f00b37fe640 1 -- 172.21.15.83:0/4014062693 learned_addr learned my addr 172.21.15.83:0/4014062693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b2ffd640 1 --2- 172.21.15.83:0/4014062693 >> v2:172.21.15.155:3300/0 conn(0x7f00b406eec0 0x7f00b40b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b37fe640 1 -- 172.21.15.83:0/4014062693 >> v2:172.21.15.155:3300/0 conn(0x7f00b406eec0 msgr2=0x7f00b40b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b37fe640 1 --2- 172.21.15.83:0/4014062693 >> v2:172.21.15.155:3300/0 conn(0x7f00b406eec0 0x7f00b40b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b37fe640 1 -- 172.21.15.83:0/4014062693 >> v1:172.21.15.155:6789/0 conn(0x7f00b415be70 legacy=0x7f00b41601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:57.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b37fe640 1 -- 172.21.15.83:0/4014062693 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b406fb60 con 0x7f00b40b44c0 2024-05-17T22:02:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b37fe640 1 --2- 172.21.15.83:0/4014062693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b40b48d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f00b406d670 tx=0x7f00ac01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3827b0e68affd307 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b1ffb640 1 -- 172.21.15.83:0/4014062693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0a55e0 con 0x7f00b40b44c0 2024-05-17T22:02:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b1ffb640 1 -- 172.21.15.83:0/4014062693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac0a5780 con 0x7f00b40b44c0 2024-05-17T22:02:57.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b1ffb640 1 -- 172.21.15.83:0/4014062693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0a5b50 con 0x7f00b40b44c0 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 -- 172.21.15.83:0/4014062693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 msgr2=0x7f00b40b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 --2- 172.21.15.83:0/4014062693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b40b48d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f00b406d670 tx=0x7f00ac01c960 comp rx=0 tx=0).stop 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 -- 172.21.15.83:0/4014062693 shutdown_connections 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 --2- 172.21.15.83:0/4014062693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b40b48d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 --2- 172.21.15.83:0/4014062693 >> v2:172.21.15.155:3300/0 conn(0x7f00b406eec0 0x7f00b40b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.610+0000 7f00b94d1640 1 -- 172.21.15.83:0/4014062693 >> 172.21.15.83:0/4014062693 conn(0x7f00b406cd90 msgr2=0x7f00b40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- 172.21.15.83:0/4014062693 shutdown_connections 2024-05-17T22:02:57.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- 172.21.15.83:0/4014062693 wait complete. 2024-05-17T22:02:57.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 Processor -- start 2024-05-17T22:02:57.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- start start 2024-05-17T22:02:57.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00b406eec0 0x7f00b4153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b415be70 0x7f00b40b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00b40b3a50 con 0x7f00b40b44c0 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00b40b38d0 con 0x7f00b415be70 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b94d1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f00b40b3bd0 con 0x7f00b406eec0 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60530/0 (socket says 172.21.15.83:60530) 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 -- 172.21.15.83:0/2932695141 learned_addr learned my addr 172.21.15.83:0/2932695141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b2ffd640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00b406eec0 0x7f00b4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b37fe640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b415be70 0x7f00b40b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 -- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00b406eec0 msgr2=0x7f00b4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00b406eec0 0x7f00b4153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 -- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b415be70 msgr2=0x7f00b40b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b415be70 0x7f00b40b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.614+0000 7f00b27fc640 1 -- 172.21.15.83:0/2932695141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b415d9e0 con 0x7f00b40b44c0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b27fc640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f00a800b3f0 tx=0x7f00a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8015020 con 0x7f00b40b44c0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a8011830 con 0x7f00b40b44c0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b415dcc0 con 0x7f00b40b44c0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8011c00 con 0x7f00b40b44c0 2024-05-17T22:02:57.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b41f9b30 con 0x7f00b40b44c0 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b415ac30 con 0x7f00b40b44c0 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00a8018020 con 0x7f00b40b44c0 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 0x7f00a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f00a808f250 con 0x7f00b40b44c0 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b2ffd640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 0x7f00a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.618+0000 7f00b2ffd640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 0x7f00a0071940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f00a40096f0 tx=0x7f00a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:57.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.622+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f00a801f030 con 0x7f00b40b44c0 2024-05-17T22:02:57.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:57.818+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f00b40b08a0 con 0x7f00b40b44c0 2024-05-17T22:02:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.438+0000 7f009b7fe640 1 -- 172.21.15.83:0/2932695141 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f00a805c2b0 con 0x7f00b40b44c0 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 msgr2=0x7f00a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 0x7f00a0071940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f00a40096f0 tx=0x7f00a4008040 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 msgr2=0x7f00b415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f00a800b3f0 tx=0x7f00a800b8b0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 shutdown_connections 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f00a006f430 0x7f00a0071940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00b415be70 0x7f00b40b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b40b44c0 0x7f00b415a240 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 --2- 172.21.15.83:0/2932695141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00b406eec0 0x7f00b4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 >> 172.21.15.83:0/2932695141 conn(0x7f00b406cd90 msgr2=0x7f00b40cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 shutdown_connections 2024-05-17T22:02:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.442+0000 7f00b94d1640 1 -- 172.21.15.83:0/2932695141 wait complete. 2024-05-17T22:02:58.457 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T22:02:58.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:02:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 Processor -- start 2024-05-17T22:02:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 -- start start 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 0x7f6264144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f6264072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626415e490 0x7f6264073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f626415ea40 con 0x7f626415e490 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f626415ed20 con 0x7f62640d1040 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f626a587640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f626415f000 con 0x7f626415e080 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626415e490 0x7f6264073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f626415e490 0x7f6264073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60540/0 (socket says 172.21.15.83:60540) 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 -- 172.21.15.83:0/3256679993 learned_addr learned my addr 172.21.15.83:0/3256679993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f62637fe640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f6264072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6263fff640 1 --2- 172.21.15.83:0/3256679993 >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 0x7f6264144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 -- 172.21.15.83:0/3256679993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 msgr2=0x7f6264072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f6264072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 -- 172.21.15.83:0/3256679993 >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 msgr2=0x7f6264144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 --2- 172.21.15.83:0/3256679993 >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 0x7f6264144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6268b24640 1 -- 172.21.15.83:0/3256679993 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626406ef30 con 0x7f626415e490 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f62637fe640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f6264072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.618+0000 7f6263fff640 1 --2- 172.21.15.83:0/3256679993 >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 0x7f6264144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:58.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f6268b24640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f6264073370 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6264071d60 tx=0x7f625c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f7d6df409be920 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:58.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f6262ffd640 1 -- 172.21.15.83:0/3256679993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0a5780 con 0x7f626415e490 2024-05-17T22:02:58.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f6262ffd640 1 -- 172.21.15.83:0/3256679993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c0a5920 con 0x7f626415e490 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f6262ffd640 1 -- 172.21.15.83:0/3256679993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0a5cf0 con 0x7f626415e490 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 -- 172.21.15.83:0/3256679993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 msgr2=0x7f6264073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f6264073370 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6264071d60 tx=0x7f625c01c9f0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 -- 172.21.15.83:0/3256679993 shutdown_connections 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f6264073370 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 --2- 172.21.15.83:0/3256679993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f6264072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 --2- 172.21.15.83:0/3256679993 >> v2:172.21.15.155:3300/0 conn(0x7f62640d1040 0x7f6264144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 -- 172.21.15.83:0/3256679993 >> 172.21.15.83:0/3256679993 conn(0x7f626406cd90 msgr2=0x7f62640b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:58.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 -- 172.21.15.83:0/3256679993 shutdown_connections 2024-05-17T22:02:58.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.622+0000 7f626a587640 1 -- 172.21.15.83:0/3256679993 wait complete. 2024-05-17T22:02:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 Processor -- start 2024-05-17T22:02:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- start start 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62640d1040 0x7f626413aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f626413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62641622a0 con 0x7f626415e490 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6264162120 con 0x7f62640d1040 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6264162420 con 0x7f626415e080 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60554/0 (socket says 172.21.15.83:60554) 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 -- 172.21.15.83:0/3950392745 learned_addr learned my addr 172.21.15.83:0/3950392745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6263fff640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62640d1040 0x7f626413aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f62637fe640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f626413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 -- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 msgr2=0x7f626413cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f626413cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 -- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62640d1040 msgr2=0x7f626413aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62640d1040 0x7f626413aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 -- 172.21.15.83:0/3950392745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626413d760 con 0x7f626415e490 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f62637fe640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f626413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6268b24640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f625c0af040 tx=0x7f625c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0b8880 con 0x7f626415e490 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c0be070 con 0x7f626415e490 2024-05-17T22:02:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c0b8c80 con 0x7f626415e490 2024-05-17T22:02:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626413da40 con 0x7f626415e490 2024-05-17T22:02:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.626+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6264144e50 con 0x7f626415e490 2024-05-17T22:02:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6264144080 con 0x7f626415e490 2024-05-17T22:02:58.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f625c0c3460 con 0x7f626415e490 2024-05-17T22:02:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f6260ff9640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 0x7f6250071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f625c12a800 con 0x7f626415e490 2024-05-17T22:02:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f6263fff640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 0x7f6250071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.630+0000 7f6263fff640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 0x7f6250071940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f625400a990 tx=0x7f6254008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:58.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.634+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f625c0165e0 con 0x7f626415e490 2024-05-17T22:02:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.830+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6264140c60 con 0x7f626415e490 2024-05-17T22:02:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.830+0000 7f6260ff9640 1 -- 172.21.15.83:0/3950392745 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5189 (secure 0 0 0) 0x7f625c0f7790 con 0x7f626415e490 2024-05-17T22:02:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:58.824 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:02:57.824521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6618":{"gid":6618,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3213750564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3213750564},{"type":"v1","addr":"172.21.15.83:6836","nonce":3213750564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6618,"qdb_cluster":[6618]},"id":6}]} 2024-05-17T22:02:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 59 2024-05-17T22:02:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 msgr2=0x7f6250071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 0x7f6250071940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f625400a990 tx=0x7f6254008040 comp rx=0 tx=0).stop 2024-05-17T22:02:58.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 msgr2=0x7f626415c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f625c0af040 tx=0x7f625c019790 comp rx=0 tx=0).stop 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 shutdown_connections 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626415e490 0x7f626415c240 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f626415e080 0x7f626413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f625006f430 0x7f6250071940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 --2- 172.21.15.83:0/3950392745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62640d1040 0x7f626413aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 >> 172.21.15.83:0/3950392745 conn(0x7f626406cd90 msgr2=0x7f626406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 shutdown_connections 2024-05-17T22:02:58.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.834+0000 7f626a587640 1 -- 172.21.15.83:0/3950392745 wait complete. 2024-05-17T22:02:58.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:02:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.998+0000 7f1f6733e640 1 Processor -- start 2024-05-17T22:02:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:58.998+0000 7f1f6733e640 1 -- start start 2024-05-17T22:02:58.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f6733e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1f6015ffa0 0x7f1f600d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f6733e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f600d34e0 con 0x7f1f6006e3d0 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f6733e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f600d3780 con 0x7f1f600d1150 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f6733e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1f600d3a60 con 0x7f1f6015ffa0 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f658db640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f6006e3d0 0x7f1f600d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57088/0 (socket says 172.21.15.83:57088) 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f658db640 1 -- 172.21.15.83:0/1512794967 learned_addr learned my addr 172.21.15.83:0/1512794967 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f650da640 1 --2- 172.21.15.83:0/1512794967 >> v2:172.21.15.155:3300/0 conn(0x7f1f6015ffa0 0x7f1f600d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716781277 0 0) 0x7f1f600d34e0 con 0x7f1f6006e3d0 2024-05-17T22:02:58.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f4c003530 con 0x7f1f6006e3d0 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80375090 0 0) 0x7f1f4c003530 con 0x7f1f6006e3d0 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 >> v2:172.21.15.155:3300/0 conn(0x7f1f6015ffa0 msgr2=0x7f1f600d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 --2- 172.21.15.83:0/1512794967 >> v2:172.21.15.155:3300/0 conn(0x7f1f6015ffa0 0x7f1f600d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 >> v1:172.21.15.155:6789/0 conn(0x7f1f600d1150 legacy=0x7f1f6006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600d3d40 con 0x7f1f6006e3d0 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1f58001aa0 con 0x7f1f6006e3d0 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 >> v1:172.21.15.83:6789/0 conn(0x7f1f6006e3d0 legacy=0x7f1f600d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f4c003a60 0x7f1f4c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f4c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f4c005260 0x7f1f4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d3a60 con 0x7f1f4c004810 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c00c1f0 con 0x7f1f4c005260 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c00c3f0 con 0x7f1f4c003a60 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f4c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f658db640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f4c005260 0x7f1f4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f650da640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f4c003a60 0x7f1f4c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 -- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f4c003a60 msgr2=0x7f1f4c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f4c003a60 0x7f1f4c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 -- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f4c005260 msgr2=0x7f1f4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:58.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f4c005260 0x7f1f4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:58.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f658db640 1 -- 172.21.15.83:0/1512794967 reap_dead start 2024-05-17T22:02:58.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.002+0000 7f1f648d9640 1 -- 172.21.15.83:0/1512794967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c003530 con 0x7f1f4c004810 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f648d9640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f4c004c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1f50001790 tx=0x7f1f5001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafdbd94455e455d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500013e0 con 0x7f1f4c004810 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f500052d0 con 0x7f1f4c004810 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f57fff640 1 -- 172.21.15.83:0/1512794967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50003840 con 0x7f1f4c004810 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 msgr2=0x7f1f4c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f4c004c20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1f50001790 tx=0x7f1f5001c3d0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- 172.21.15.83:0/1512794967 shutdown_connections 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 --2- 172.21.15.83:0/1512794967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f4c004c20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- 172.21.15.83:0/1512794967 >> 172.21.15.83:0/1512794967 conn(0x7f1f600d28f0 msgr2=0x7f1f600d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- 172.21.15.83:0/1512794967 shutdown_connections 2024-05-17T22:02:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- 172.21.15.83:0/1512794967 wait complete. 2024-05-17T22:02:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 Processor -- start 2024-05-17T22:02:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.006+0000 7f1f6733e640 1 -- start start 2024-05-17T22:02:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f601542a0 0x7f1f6015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6015d090 0x7f1f60155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d17e0 con 0x7f1f4c004810 2024-05-17T22:02:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d1660 con 0x7f1f601542a0 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600d0c10 con 0x7f1f6015d090 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60568/0 (socket says 172.21.15.83:60568) 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 -- 172.21.15.83:0/2455435210 learned_addr learned my addr 172.21.15.83:0/2455435210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f648d9640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f601542a0 0x7f1f6015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f658db640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6015d090 0x7f1f60155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 -- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6015d090 msgr2=0x7f1f60155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6015d090 0x7f1f60155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 -- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f601542a0 msgr2=0x7f1f6015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f601542a0 0x7f1f6015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 -- 172.21.15.83:0/2455435210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f601611a0 con 0x7f1f4c004810 2024-05-17T22:02:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f650da640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c011500 tx=0x7f1f5c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c015870 con 0x7f1f4c004810 2024-05-17T22:02:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5c015a10 con 0x7f1f4c004810 2024-05-17T22:02:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f60161480 con 0x7f1f4c004810 2024-05-17T22:02:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c015dc0 con 0x7f1f4c004810 2024-05-17T22:02:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f60161a40 con 0x7f1f4c004810 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6006dcc0 con 0x7f1f4c004810 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.010+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f5c013400 con 0x7f1f4c004810 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.014+0000 7f1f55ffb640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 0x7f1f44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.014+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f1f5c08fba0 con 0x7f1f4c004810 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.014+0000 7f1f648d9640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 0x7f1f44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.014+0000 7f1f648d9640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 0x7f1f44071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ac000 tx=0x7f1f50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.018+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1f5c019030 con 0x7f1f4c004810 2024-05-17T22:02:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.182+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f60155880 con 0x7f1f4c004810 2024-05-17T22:02:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.182+0000 7f1f55ffb640 1 -- 172.21.15.83:0/2455435210 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16636 (secure 0 0 0) 0x7f1f5c05cb30 con 0x7f1f4c004810 2024-05-17T22:02:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:59.176 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:02:56.414313+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:02:52.464989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-17T22:02:53.753100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 msgr2=0x7f1f44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 0x7f1f44071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ac000 tx=0x7f1f50009040 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 msgr2=0x7f1f6015c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c011500 tx=0x7f1f5c0119c0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 shutdown_connections 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1f6015d090 0x7f1f60155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1f4406f430 0x7f1f44071940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f601542a0 0x7f1f6015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 --2- 172.21.15.83:0/2455435210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f4c004810 0x7f1f6015c610 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 >> 172.21.15.83:0/2455435210 conn(0x7f1f600d28f0 msgr2=0x7f1f600b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 shutdown_connections 2024-05-17T22:02:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.186+0000 7f1f6733e640 1 -- 172.21.15.83:0/2455435210 wait complete. 2024-05-17T22:02:59.200 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:02:59.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.354+0000 7fa8bc4af640 1 Processor -- start 2024-05-17T22:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 -- start start 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8b406e480 0x7fa8b415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40d44a0 0x7fa8b41698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b40d55e0 con 0x7fa8b40d0370 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b40b8d00 con 0x7fa8b406e480 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8bc4af640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b40d4a60 con 0x7fa8b40d44a0 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8b406e480 0x7fa8b415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8baa4c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40d44a0 0x7fa8b41698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8b406e480 0x7fa8b415fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:48416/0 (socket says 172.21.15.83:48416) 2024-05-17T22:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/1930695019 learned_addr learned my addr 172.21.15.83:0/1930695019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40d44a0 msgr2=0x7fa8b41698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 --2- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40d44a0 0x7fa8b41698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/1930695019 >> v1:172.21.15.155:6789/0 conn(0x7fa8b40d0370 legacy=0x7fa8b406dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.358+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/1930695019 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40d4d40 con 0x7fa8b406e480 2024-05-17T22:02:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8b9a4a640 1 --2- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b406e480 0x7fa8b415fd00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009870 tx=0x7fa8b001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898e3eb48d36d651 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8b9249640 1 -- 172.21.15.83:0/1930695019 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00b2070 con 0x7fa8b406e480 2024-05-17T22:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8b9249640 1 -- 172.21.15.83:0/1930695019 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b00a9070 con 0x7fa8b406e480 2024-05-17T22:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8b9249640 1 -- 172.21.15.83:0/1930695019 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00ad880 con 0x7fa8b406e480 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 -- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b406e480 msgr2=0x7fa8b415fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b406e480 0x7fa8b415fd00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009870 tx=0x7fa8b001c560 comp rx=0 tx=0).stop 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 -- 172.21.15.83:0/1930695019 shutdown_connections 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40d44a0 0x7fa8b41698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/1930695019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b406e480 0x7fa8b415fd00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 -- 172.21.15.83:0/1930695019 >> 172.21.15.83:0/1930695019 conn(0x7fa8b4074a40 msgr2=0x7fa8b40d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 -- 172.21.15.83:0/1930695019 shutdown_connections 2024-05-17T22:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.362+0000 7fa8bc4af640 1 -- 172.21.15.83:0/1930695019 wait complete. 2024-05-17T22:02:59.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 Processor -- start 2024-05-17T22:02:59.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 -- start start 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 0x7fa8b41fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 0x7fa8b40cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b40bb300 con 0x7fa8b40d44a0 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b40bb180 con 0x7fa8b40d0370 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8bc4af640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b416c650 con 0x7fa8b40b4e40 2024-05-17T22:02:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60578/0 (socket says 172.21.15.83:60578) 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/148115409 learned_addr learned my addr 172.21.15.83:0/148115409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8ba24b640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 0x7fa8b41fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8baa4c640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 0x7fa8b40cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 msgr2=0x7fa8b40cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 0x7fa8b40cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 msgr2=0x7fa8b41fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 0x7fa8b41fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 -- 172.21.15.83:0/148115409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b40b54c0 con 0x7fa8b40d44a0 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8ba24b640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 0x7fa8b41fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8baa4c640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 0x7fa8b40cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8b9a4a640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00b0040 tx=0x7fa8b0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.366+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00b2040 con 0x7fa8b40d44a0 2024-05-17T22:02:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b00bb070 con 0x7fa8b40d44a0 2024-05-17T22:02:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b40b57a0 con 0x7fa8b40d44a0 2024-05-17T22:02:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00b6440 con 0x7fa8b40d44a0 2024-05-17T22:02:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b40b5d60 con 0x7fa8b40d44a0 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b415fd00 con 0x7fa8b40d44a0 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa8b00ab070 con 0x7fa8b40d44a0 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8a6ffd640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 0x7fa898071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fa8b0128c30 con 0x7fa8b40d44a0 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8ba24b640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 0x7fa898071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.370+0000 7fa8ba24b640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 0x7fa898071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40d3a30 tx=0x7fa8a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.374+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa8b0016150 con 0x7fa8b40d44a0 2024-05-17T22:02:59.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.522+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8b40b5250 con 0x7fa89806f400 2024-05-17T22:02:59.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.534+0000 7fa8a6ffd640 1 -- 172.21.15.83:0/148115409 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252071 (secure 0 0 0) 0x7fa8b40b5250 con 0x7fa89806f400 2024-05-17T22:02:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:02:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 msgr2=0x7fa898071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 0x7fa898071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40d3a30 tx=0x7fa8a8009040 comp rx=0 tx=0).stop 2024-05-17T22:02:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 msgr2=0x7fa8b40b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00b0040 tx=0x7fa8b0019300 comp rx=0 tx=0).stop 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 shutdown_connections 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8b40b4e40 0x7fa8b40cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8b40d44a0 0x7fa8b40b4900 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa89806f400 0x7fa898071910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 --2- 172.21.15.83:0/148115409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8b40d0370 0x7fa8b41fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 >> 172.21.15.83:0/148115409 conn(0x7fa8b4074a40 msgr2=0x7fa8b4161fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 shutdown_connections 2024-05-17T22:02:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.538+0000 7fa8bc4af640 1 -- 172.21.15.83:0/148115409 wait complete. 2024-05-17T22:02:59.551 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-05-17T22:02:59.087499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258440,"kb_used_data":9000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716280,"statfs":{"total":773094113280,"available":772829470720,"internally_reserved":0,"allocated":9216000,"data_stored":4570308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.721210"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731529+0000","last_change":"2024-05-17T22:02:55.481664+0000","last_active":"2024-05-17T22:02:56.731529+0000","last_peered":"2024-05-17T22:02:56.731529+0000","last_clean":"2024-05-17T22:02:56.731529+0000","last_became_active":"2024-05-17T22:02:55.481035+0000","last_became_peered":"2024-05-17T22:02:55.481035+0000","last_unstale":"2024-05-17T22:02:56.731529+0000","last_undegraded":"2024-05-17T22:02:56.731529+0000","last_fullsized":"2024-05-17T22:02:56.731529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:25.275714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462026+0000","last_change":"2024-05-17T22:02:55.481814+0000","last_active":"2024-05-17T22:02:56.462026+0000","last_peered":"2024-05-17T22:02:56.462026+0000","last_clean":"2024-05-17T22:02:56.462026+0000","last_became_active":"2024-05-17T22:02:55.481436+0000","last_became_peered":"2024-05-17T22:02:55.481436+0000","last_unstale":"2024-05-17T22:02:56.462026+0000","last_undegraded":"2024-05-17T22:02:56.462026+0000","last_fullsized":"2024-05-17T22:02:56.462026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:27.148640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729816+0000","last_change":"2024-05-17T22:02:55.482132+0000","last_active":"2024-05-17T22:02:56.729816+0000","last_peered":"2024-05-17T22:02:56.729816+0000","last_clean":"2024-05-17T22:02:56.729816+0000","last_became_active":"2024-05-17T22:02:55.481564+0000","last_became_peered":"2024-05-17T22:02:55.481564+0000","last_unstale":"2024-05-17T22:02:56.729816+0000","last_undegraded":"2024-05-17T22:02:56.729816+0000","last_fullsized":"2024-05-17T22:02:56.729816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:40.911868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428743+0000","last_change":"2024-05-17T22:02:55.452224+0000","last_active":"2024-05-17T22:02:56.428743+0000","last_peered":"2024-05-17T22:02:56.428743+0000","last_clean":"2024-05-17T22:02:56.428743+0000","last_became_active":"2024-05-17T22:02:55.450992+0000","last_became_peered":"2024-05-17T22:02:55.450992+0000","last_unstale":"2024-05-17T22:02:56.428743+0000","last_undegraded":"2024-05-17T22:02:56.428743+0000","last_fullsized":"2024-05-17T22:02:56.428743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:02.356697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431170+0000","last_change":"2024-05-17T22:02:55.446573+0000","last_active":"2024-05-17T22:02:56.431170+0000","last_peered":"2024-05-17T22:02:56.431170+0000","last_clean":"2024-05-17T22:02:56.431170+0000","last_became_active":"2024-05-17T22:02:55.445716+0000","last_became_peered":"2024-05-17T22:02:55.445716+0000","last_unstale":"2024-05-17T22:02:56.431170+0000","last_undegraded":"2024-05-17T22:02:56.431170+0000","last_fullsized":"2024-05-17T22:02:56.431170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427758+0000","last_change":"2024-05-17T22:02:55.455553+0000","last_active":"2024-05-17T22:02:56.427758+0000","last_peered":"2024-05-17T22:02:56.427758+0000","last_clean":"2024-05-17T22:02:56.427758+0000","last_became_active":"2024-05-17T22:02:55.454741+0000","last_became_peered":"2024-05-17T22:02:55.454741+0000","last_unstale":"2024-05-17T22:02:56.427758+0000","last_undegraded":"2024-05-17T22:02:56.427758+0000","last_fullsized":"2024-05-17T22:02:56.427758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:03:24.023210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427509+0000","last_change":"2024-05-17T22:02:55.444843+0000","last_active":"2024-05-17T22:02:56.427509+0000","last_peered":"2024-05-17T22:02:56.427509+0000","last_clean":"2024-05-17T22:02:56.427509+0000","last_became_active":"2024-05-17T22:02:55.442085+0000","last_became_peered":"2024-05-17T22:02:55.442085+0000","last_unstale":"2024-05-17T22:02:56.427509+0000","last_undegraded":"2024-05-17T22:02:56.427509+0000","last_fullsized":"2024-05-17T22:02:56.427509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:24:56.114494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428084+0000","last_change":"2024-05-17T22:02:55.442971+0000","last_active":"2024-05-17T22:02:56.428084+0000","last_peered":"2024-05-17T22:02:56.428084+0000","last_clean":"2024-05-17T22:02:56.428084+0000","last_became_active":"2024-05-17T22:02:55.442107+0000","last_became_peered":"2024-05-17T22:02:55.442107+0000","last_unstale":"2024-05-17T22:02:56.428084+0000","last_undegraded":"2024-05-17T22:02:56.428084+0000","last_fullsized":"2024-05-17T22:02:56.428084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:53:00.432103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428777+0000","last_change":"2024-05-17T22:02:55.442865+0000","last_active":"2024-05-17T22:02:56.428777+0000","last_peered":"2024-05-17T22:02:56.428777+0000","last_clean":"2024-05-17T22:02:56.428777+0000","last_became_active":"2024-05-17T22:02:55.441970+0000","last_became_peered":"2024-05-17T22:02:55.441970+0000","last_unstale":"2024-05-17T22:02:56.428777+0000","last_undegraded":"2024-05-17T22:02:56.428777+0000","last_fullsized":"2024-05-17T22:02:56.428777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730887+0000","last_change":"2024-05-17T22:02:55.466764+0000","last_active":"2024-05-17T22:02:56.730887+0000","last_peered":"2024-05-17T22:02:56.730887+0000","last_clean":"2024-05-17T22:02:56.730887+0000","last_became_active":"2024-05-17T22:02:55.465851+0000","last_became_peered":"2024-05-17T22:02:55.465851+0000","last_unstale":"2024-05-17T22:02:56.730887+0000","last_undegraded":"2024-05-17T22:02:56.730887+0000","last_fullsized":"2024-05-17T22:02:56.730887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461648+0000","last_change":"2024-05-17T22:02:55.450794+0000","last_active":"2024-05-17T22:02:56.461648+0000","last_peered":"2024-05-17T22:02:56.461648+0000","last_clean":"2024-05-17T22:02:56.461648+0000","last_became_active":"2024-05-17T22:02:55.449661+0000","last_became_peered":"2024-05-17T22:02:55.449661+0000","last_unstale":"2024-05-17T22:02:56.461648+0000","last_undegraded":"2024-05-17T22:02:56.461648+0000","last_fullsized":"2024-05-17T22:02:56.461648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:24:07.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730839+0000","last_change":"2024-05-17T22:02:55.465513+0000","last_active":"2024-05-17T22:02:56.730839+0000","last_peered":"2024-05-17T22:02:56.730839+0000","last_clean":"2024-05-17T22:02:56.730839+0000","last_became_active":"2024-05-17T22:02:55.464930+0000","last_became_peered":"2024-05-17T22:02:55.464930+0000","last_unstale":"2024-05-17T22:02:56.730839+0000","last_undegraded":"2024-05-17T22:02:56.730839+0000","last_fullsized":"2024-05-17T22:02:56.730839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729445+0000","last_change":"2024-05-17T22:02:55.442051+0000","last_active":"2024-05-17T22:02:56.729445+0000","last_peered":"2024-05-17T22:02:56.729445+0000","last_clean":"2024-05-17T22:02:56.729445+0000","last_became_active":"2024-05-17T22:02:55.440836+0000","last_became_peered":"2024-05-17T22:02:55.440836+0000","last_unstale":"2024-05-17T22:02:56.729445+0000","last_undegraded":"2024-05-17T22:02:56.729445+0000","last_fullsized":"2024-05-17T22:02:56.729445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427649+0000","last_change":"2024-05-17T22:02:55.451950+0000","last_active":"2024-05-17T22:02:56.427649+0000","last_peered":"2024-05-17T22:02:56.427649+0000","last_clean":"2024-05-17T22:02:56.427649+0000","last_became_active":"2024-05-17T22:02:55.451126+0000","last_became_peered":"2024-05-17T22:02:55.451126+0000","last_unstale":"2024-05-17T22:02:56.427649+0000","last_undegraded":"2024-05-17T22:02:56.427649+0000","last_fullsized":"2024-05-17T22:02:56.427649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:27:57.929783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728805+0000","last_change":"2024-05-17T22:02:55.460969+0000","last_active":"2024-05-17T22:02:56.728805+0000","last_peered":"2024-05-17T22:02:56.728805+0000","last_clean":"2024-05-17T22:02:56.728805+0000","last_became_active":"2024-05-17T22:02:55.460374+0000","last_became_peered":"2024-05-17T22:02:55.460374+0000","last_unstale":"2024-05-17T22:02:56.728805+0000","last_undegraded":"2024-05-17T22:02:56.728805+0000","last_fullsized":"2024-05-17T22:02:56.728805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:57.177274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429940+0000","last_change":"2024-05-17T22:02:55.474041+0000","last_active":"2024-05-17T22:02:56.429940+0000","last_peered":"2024-05-17T22:02:56.429940+0000","last_clean":"2024-05-17T22:02:56.429940+0000","last_became_active":"2024-05-17T22:02:55.473270+0000","last_became_peered":"2024-05-17T22:02:55.473270+0000","last_unstale":"2024-05-17T22:02:56.429940+0000","last_undegraded":"2024-05-17T22:02:56.429940+0000","last_fullsized":"2024-05-17T22:02:56.429940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:31.755542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431765+0000","last_change":"2024-05-17T22:02:55.453993+0000","last_active":"2024-05-17T22:02:56.431765+0000","last_peered":"2024-05-17T22:02:56.431765+0000","last_clean":"2024-05-17T22:02:56.431765+0000","last_became_active":"2024-05-17T22:02:55.453177+0000","last_became_peered":"2024-05-17T22:02:55.453177+0000","last_unstale":"2024-05-17T22:02:56.431765+0000","last_undegraded":"2024-05-17T22:02:56.431765+0000","last_fullsized":"2024-05-17T22:02:56.431765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427649+0000","last_change":"2024-05-17T22:02:55.468588+0000","last_active":"2024-05-17T22:02:56.427649+0000","last_peered":"2024-05-17T22:02:56.427649+0000","last_clean":"2024-05-17T22:02:56.427649+0000","last_became_active":"2024-05-17T22:02:55.468282+0000","last_became_peered":"2024-05-17T22:02:55.468282+0000","last_unstale":"2024-05-17T22:02:56.427649+0000","last_undegraded":"2024-05-17T22:02:56.427649+0000","last_fullsized":"2024-05-17T22:02:56.427649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:03:52.766961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465601+0000","last_change":"2024-05-17T22:02:55.470291+0000","last_active":"2024-05-17T22:02:56.465601+0000","last_peered":"2024-05-17T22:02:56.465601+0000","last_clean":"2024-05-17T22:02:56.465601+0000","last_became_active":"2024-05-17T22:02:55.469913+0000","last_became_peered":"2024-05-17T22:02:55.469913+0000","last_unstale":"2024-05-17T22:02:56.465601+0000","last_undegraded":"2024-05-17T22:02:56.465601+0000","last_fullsized":"2024-05-17T22:02:56.465601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429112+0000","last_change":"2024-05-17T22:02:55.442185+0000","last_active":"2024-05-17T22:02:56.429112+0000","last_peered":"2024-05-17T22:02:56.429112+0000","last_clean":"2024-05-17T22:02:56.429112+0000","last_became_active":"2024-05-17T22:02:55.441517+0000","last_became_peered":"2024-05-17T22:02:55.441517+0000","last_unstale":"2024-05-17T22:02:56.429112+0000","last_undegraded":"2024-05-17T22:02:56.429112+0000","last_fullsized":"2024-05-17T22:02:56.429112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:23.407351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428306+0000","last_change":"2024-05-17T22:02:55.457330+0000","last_active":"2024-05-17T22:02:56.428306+0000","last_peered":"2024-05-17T22:02:56.428306+0000","last_clean":"2024-05-17T22:02:56.428306+0000","last_became_active":"2024-05-17T22:02:55.456770+0000","last_became_peered":"2024-05-17T22:02:55.456770+0000","last_unstale":"2024-05-17T22:02:56.428306+0000","last_undegraded":"2024-05-17T22:02:56.428306+0000","last_fullsized":"2024-05-17T22:02:56.428306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:11.630824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461326+0000","last_change":"2024-05-17T22:02:55.456909+0000","last_active":"2024-05-17T22:02:56.461326+0000","last_peered":"2024-05-17T22:02:56.461326+0000","last_clean":"2024-05-17T22:02:56.461326+0000","last_became_active":"2024-05-17T22:02:55.456507+0000","last_became_peered":"2024-05-17T22:02:55.456507+0000","last_unstale":"2024-05-17T22:02:56.461326+0000","last_undegraded":"2024-05-17T22:02:56.461326+0000","last_fullsized":"2024-05-17T22:02:56.461326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:02.154981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731536+0000","last_change":"2024-05-17T22:02:55.463482+0000","last_active":"2024-05-17T22:02:56.731536+0000","last_peered":"2024-05-17T22:02:56.731536+0000","last_clean":"2024-05-17T22:02:56.731536+0000","last_became_active":"2024-05-17T22:02:55.460891+0000","last_became_peered":"2024-05-17T22:02:55.460891+0000","last_unstale":"2024-05-17T22:02:56.731536+0000","last_undegraded":"2024-05-17T22:02:56.731536+0000","last_fullsized":"2024-05-17T22:02:56.731536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427412+0000","last_change":"2024-05-17T22:02:55.457666+0000","last_active":"2024-05-17T22:02:56.427412+0000","last_peered":"2024-05-17T22:02:56.427412+0000","last_clean":"2024-05-17T22:02:56.427412+0000","last_became_active":"2024-05-17T22:02:55.457268+0000","last_became_peered":"2024-05-17T22:02:55.457268+0000","last_unstale":"2024-05-17T22:02:56.427412+0000","last_undegraded":"2024-05-17T22:02:56.427412+0000","last_fullsized":"2024-05-17T22:02:56.427412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:48:11.061535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461886+0000","last_change":"2024-05-17T22:02:55.463006+0000","last_active":"2024-05-17T22:02:56.461886+0000","last_peered":"2024-05-17T22:02:56.461886+0000","last_clean":"2024-05-17T22:02:56.461886+0000","last_became_active":"2024-05-17T22:02:55.462634+0000","last_became_peered":"2024-05-17T22:02:55.462634+0000","last_unstale":"2024-05-17T22:02:56.461886+0000","last_undegraded":"2024-05-17T22:02:56.461886+0000","last_fullsized":"2024-05-17T22:02:56.461886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:31.719755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729411+0000","last_change":"2024-05-17T22:02:55.449570+0000","last_active":"2024-05-17T22:02:56.729411+0000","last_peered":"2024-05-17T22:02:56.729411+0000","last_clean":"2024-05-17T22:02:56.729411+0000","last_became_active":"2024-05-17T22:02:55.448799+0000","last_became_peered":"2024-05-17T22:02:55.448799+0000","last_unstale":"2024-05-17T22:02:56.729411+0000","last_undegraded":"2024-05-17T22:02:56.729411+0000","last_fullsized":"2024-05-17T22:02:56.729411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431391+0000","last_change":"2024-05-17T22:02:55.464728+0000","last_active":"2024-05-17T22:02:56.431391+0000","last_peered":"2024-05-17T22:02:56.431391+0000","last_clean":"2024-05-17T22:02:56.431391+0000","last_became_active":"2024-05-17T22:02:55.464036+0000","last_became_peered":"2024-05-17T22:02:55.464036+0000","last_unstale":"2024-05-17T22:02:56.431391+0000","last_undegraded":"2024-05-17T22:02:56.431391+0000","last_fullsized":"2024-05-17T22:02:56.431391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730559+0000","last_change":"2024-05-17T22:02:55.466563+0000","last_active":"2024-05-17T22:02:56.730559+0000","last_peered":"2024-05-17T22:02:56.730559+0000","last_clean":"2024-05-17T22:02:56.730559+0000","last_became_active":"2024-05-17T22:02:55.466028+0000","last_became_peered":"2024-05-17T22:02:55.466028+0000","last_unstale":"2024-05-17T22:02:56.730559+0000","last_undegraded":"2024-05-17T22:02:56.730559+0000","last_fullsized":"2024-05-17T22:02:56.730559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462093+0000","last_change":"2024-05-17T22:02:55.463246+0000","last_active":"2024-05-17T22:02:56.462093+0000","last_peered":"2024-05-17T22:02:56.462093+0000","last_clean":"2024-05-17T22:02:56.462093+0000","last_became_active":"2024-05-17T22:02:55.462919+0000","last_became_peered":"2024-05-17T22:02:55.462919+0000","last_unstale":"2024-05-17T22:02:56.462093+0000","last_undegraded":"2024-05-17T22:02:56.462093+0000","last_fullsized":"2024-05-17T22:02:56.462093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:54.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461970+0000","last_change":"2024-05-17T22:02:55.455660+0000","last_active":"2024-05-17T22:02:56.461970+0000","last_peered":"2024-05-17T22:02:56.461970+0000","last_clean":"2024-05-17T22:02:56.461970+0000","last_became_active":"2024-05-17T22:02:55.455230+0000","last_became_peered":"2024-05-17T22:02:55.455230+0000","last_unstale":"2024-05-17T22:02:56.461970+0000","last_undegraded":"2024-05-17T22:02:56.461970+0000","last_fullsized":"2024-05-17T22:02:56.461970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:27:34.886684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430161+0000","last_change":"2024-05-17T22:02:55.456874+0000","last_active":"2024-05-17T22:02:56.430161+0000","last_peered":"2024-05-17T22:02:56.430161+0000","last_clean":"2024-05-17T22:02:56.430161+0000","last_became_active":"2024-05-17T22:02:55.456052+0000","last_became_peered":"2024-05-17T22:02:55.456052+0000","last_unstale":"2024-05-17T22:02:56.430161+0000","last_undegraded":"2024-05-17T22:02:56.430161+0000","last_fullsized":"2024-05-17T22:02:56.430161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:17.940143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729160+0000","last_change":"2024-05-17T22:02:55.470082+0000","last_active":"2024-05-17T22:02:56.729160+0000","last_peered":"2024-05-17T22:02:56.729160+0000","last_clean":"2024-05-17T22:02:56.729160+0000","last_became_active":"2024-05-17T22:02:55.461057+0000","last_became_peered":"2024-05-17T22:02:55.461057+0000","last_unstale":"2024-05-17T22:02:56.729160+0000","last_undegraded":"2024-05-17T22:02:56.729160+0000","last_fullsized":"2024-05-17T22:02:56.729160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431532+0000","last_change":"2024-05-17T22:02:55.481982+0000","last_active":"2024-05-17T22:02:56.431532+0000","last_peered":"2024-05-17T22:02:56.431532+0000","last_clean":"2024-05-17T22:02:56.431532+0000","last_became_active":"2024-05-17T22:02:55.481442+0000","last_became_peered":"2024-05-17T22:02:55.481442+0000","last_unstale":"2024-05-17T22:02:56.431532+0000","last_undegraded":"2024-05-17T22:02:56.431532+0000","last_fullsized":"2024-05-17T22:02:56.431532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731685+0000","last_change":"2024-05-17T22:02:54.415327+0000","last_active":"2024-05-17T22:02:56.731685+0000","last_peered":"2024-05-17T22:02:56.731685+0000","last_clean":"2024-05-17T22:02:56.731685+0000","last_became_active":"2024-05-17T22:02:54.414446+0000","last_became_peered":"2024-05-17T22:02:54.414446+0000","last_unstale":"2024-05-17T22:02:56.731685+0000","last_undegraded":"2024-05-17T22:02:56.731685+0000","last_fullsized":"2024-05-17T22:02:56.731685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:54.142353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731592+0000","last_change":"2024-05-17T22:02:55.472359+0000","last_active":"2024-05-17T22:02:56.731592+0000","last_peered":"2024-05-17T22:02:56.731592+0000","last_clean":"2024-05-17T22:02:56.731592+0000","last_became_active":"2024-05-17T22:02:55.471369+0000","last_became_peered":"2024-05-17T22:02:55.471369+0000","last_unstale":"2024-05-17T22:02:56.731592+0000","last_undegraded":"2024-05-17T22:02:56.731592+0000","last_fullsized":"2024-05-17T22:02:56.731592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:52:50.952620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431913+0000","last_change":"2024-05-17T22:02:54.406636+0000","last_active":"2024-05-17T22:02:56.431913+0000","last_peered":"2024-05-17T22:02:56.431913+0000","last_clean":"2024-05-17T22:02:56.431913+0000","last_became_active":"2024-05-17T22:02:54.396309+0000","last_became_peered":"2024-05-17T22:02:54.396309+0000","last_unstale":"2024-05-17T22:02:56.431913+0000","last_undegraded":"2024-05-17T22:02:56.431913+0000","last_fullsized":"2024-05-17T22:02:56.431913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:54.142353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730404+0000","last_change":"2024-05-17T22:02:55.464575+0000","last_active":"2024-05-17T22:02:56.730404+0000","last_peered":"2024-05-17T22:02:56.730404+0000","last_clean":"2024-05-17T22:02:56.730404+0000","last_became_active":"2024-05-17T22:02:55.463588+0000","last_became_peered":"2024-05-17T22:02:55.463588+0000","last_unstale":"2024-05-17T22:02:56.730404+0000","last_undegraded":"2024-05-17T22:02:56.730404+0000","last_fullsized":"2024-05-17T22:02:56.730404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465661+0000","last_change":"2024-05-17T22:02:54.400480+0000","last_active":"2024-05-17T22:02:56.465661+0000","last_peered":"2024-05-17T22:02:56.465661+0000","last_clean":"2024-05-17T22:02:56.465661+0000","last_became_active":"2024-05-17T22:02:54.400075+0000","last_became_peered":"2024-05-17T22:02:54.400075+0000","last_unstale":"2024-05-17T22:02:56.465661+0000","last_undegraded":"2024-05-17T22:02:56.465661+0000","last_fullsized":"2024-05-17T22:02:56.465661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:19:35.392245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729789+0000","last_change":"2024-05-17T22:02:55.453211+0000","last_active":"2024-05-17T22:02:56.729789+0000","last_peered":"2024-05-17T22:02:56.729789+0000","last_clean":"2024-05-17T22:02:56.729789+0000","last_became_active":"2024-05-17T22:02:55.452273+0000","last_became_peered":"2024-05-17T22:02:55.452273+0000","last_unstale":"2024-05-17T22:02:56.729789+0000","last_undegraded":"2024-05-17T22:02:56.729789+0000","last_fullsized":"2024-05-17T22:02:56.729789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:52:50.952620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465555+0000","last_change":"2024-05-17T22:02:54.397665+0000","last_active":"2024-05-17T22:02:56.465555+0000","last_peered":"2024-05-17T22:02:56.465555+0000","last_clean":"2024-05-17T22:02:56.465555+0000","last_became_active":"2024-05-17T22:02:54.397243+0000","last_became_peered":"2024-05-17T22:02:54.397243+0000","last_unstale":"2024-05-17T22:02:56.465555+0000","last_undegraded":"2024-05-17T22:02:56.465555+0000","last_fullsized":"2024-05-17T22:02:56.465555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:22:08.923158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461606+0000","last_change":"2024-05-17T22:02:55.455212+0000","last_active":"2024-05-17T22:02:56.461606+0000","last_peered":"2024-05-17T22:02:56.461606+0000","last_clean":"2024-05-17T22:02:56.461606+0000","last_became_active":"2024-05-17T22:02:55.454724+0000","last_became_peered":"2024-05-17T22:02:55.454724+0000","last_unstale":"2024-05-17T22:02:56.461606+0000","last_undegraded":"2024-05-17T22:02:56.461606+0000","last_fullsized":"2024-05-17T22:02:56.461606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:01:18.432736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431399+0000","last_change":"2024-05-17T22:02:54.389397+0000","last_active":"2024-05-17T22:02:56.431399+0000","last_peered":"2024-05-17T22:02:56.431399+0000","last_clean":"2024-05-17T22:02:56.431399+0000","last_became_active":"2024-05-17T22:02:54.388478+0000","last_became_peered":"2024-05-17T22:02:54.388478+0000","last_unstale":"2024-05-17T22:02:56.431399+0000","last_undegraded":"2024-05-17T22:02:56.431399+0000","last_fullsized":"2024-05-17T22:02:56.431399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:44:55.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431166+0000","last_change":"2024-05-17T22:02:55.455488+0000","last_active":"2024-05-17T22:02:56.431166+0000","last_peered":"2024-05-17T22:02:56.431166+0000","last_clean":"2024-05-17T22:02:56.431166+0000","last_became_active":"2024-05-17T22:02:55.454734+0000","last_became_peered":"2024-05-17T22:02:55.454734+0000","last_unstale":"2024-05-17T22:02:56.431166+0000","last_undegraded":"2024-05-17T22:02:56.431166+0000","last_fullsized":"2024-05-17T22:02:56.431166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:53:00.432103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430110+0000","last_change":"2024-05-17T22:02:54.406571+0000","last_active":"2024-05-17T22:02:56.430110+0000","last_peered":"2024-05-17T22:02:56.430110+0000","last_clean":"2024-05-17T22:02:56.430110+0000","last_became_active":"2024-05-17T22:02:54.402851+0000","last_became_peered":"2024-05-17T22:02:54.402851+0000","last_unstale":"2024-05-17T22:02:56.430110+0000","last_undegraded":"2024-05-17T22:02:56.430110+0000","last_fullsized":"2024-05-17T22:02:56.430110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:53.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729060+0000","last_change":"2024-05-17T22:02:55.459888+0000","last_active":"2024-05-17T22:02:56.729060+0000","last_peered":"2024-05-17T22:02:56.729060+0000","last_clean":"2024-05-17T22:02:56.729060+0000","last_became_active":"2024-05-17T22:02:55.458995+0000","last_became_peered":"2024-05-17T22:02:55.458995+0000","last_unstale":"2024-05-17T22:02:56.729060+0000","last_undegraded":"2024-05-17T22:02:56.729060+0000","last_fullsized":"2024-05-17T22:02:56.729060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.464972+0000","last_change":"2024-05-17T22:02:54.396552+0000","last_active":"2024-05-17T22:02:56.464972+0000","last_peered":"2024-05-17T22:02:56.464972+0000","last_clean":"2024-05-17T22:02:56.464972+0000","last_became_active":"2024-05-17T22:02:54.396215+0000","last_became_peered":"2024-05-17T22:02:54.396215+0000","last_unstale":"2024-05-17T22:02:56.464972+0000","last_undegraded":"2024-05-17T22:02:56.464972+0000","last_fullsized":"2024-05-17T22:02:56.464972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:30.744450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729007+0000","last_change":"2024-05-17T22:02:55.460668+0000","last_active":"2024-05-17T22:02:56.729007+0000","last_peered":"2024-05-17T22:02:56.729007+0000","last_clean":"2024-05-17T22:02:56.729007+0000","last_became_active":"2024-05-17T22:02:55.459765+0000","last_became_peered":"2024-05-17T22:02:55.459765+0000","last_unstale":"2024-05-17T22:02:56.729007+0000","last_undegraded":"2024-05-17T22:02:56.729007+0000","last_fullsized":"2024-05-17T22:02:56.729007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:54:52.905846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461333+0000","last_change":"2024-05-17T22:02:54.390692+0000","last_active":"2024-05-17T22:02:56.461333+0000","last_peered":"2024-05-17T22:02:56.461333+0000","last_clean":"2024-05-17T22:02:56.461333+0000","last_became_active":"2024-05-17T22:02:54.390299+0000","last_became_peered":"2024-05-17T22:02:54.390299+0000","last_unstale":"2024-05-17T22:02:56.461333+0000","last_undegraded":"2024-05-17T22:02:56.461333+0000","last_fullsized":"2024-05-17T22:02:56.461333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:22.396260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729378+0000","last_change":"2024-05-17T22:02:55.461488+0000","last_active":"2024-05-17T22:02:56.729378+0000","last_peered":"2024-05-17T22:02:56.729378+0000","last_clean":"2024-05-17T22:02:56.729378+0000","last_became_active":"2024-05-17T22:02:55.461154+0000","last_became_peered":"2024-05-17T22:02:55.461154+0000","last_unstale":"2024-05-17T22:02:56.729378+0000","last_undegraded":"2024-05-17T22:02:56.729378+0000","last_fullsized":"2024-05-17T22:02:56.729378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:27:38.926685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731849+0000","last_change":"2024-05-17T22:02:54.422715+0000","last_active":"2024-05-17T22:02:56.731849+0000","last_peered":"2024-05-17T22:02:56.731849+0000","last_clean":"2024-05-17T22:02:56.731849+0000","last_became_active":"2024-05-17T22:02:54.421613+0000","last_became_peered":"2024-05-17T22:02:54.421613+0000","last_unstale":"2024-05-17T22:02:56.731849+0000","last_undegraded":"2024-05-17T22:02:56.731849+0000","last_fullsized":"2024-05-17T22:02:56.731849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:35:50.871077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428878+0000","last_change":"2024-05-17T22:02:55.439866+0000","last_active":"2024-05-17T22:02:56.428878+0000","last_peered":"2024-05-17T22:02:56.428878+0000","last_clean":"2024-05-17T22:02:56.428878+0000","last_became_active":"2024-05-17T22:02:55.439021+0000","last_became_peered":"2024-05-17T22:02:55.439021+0000","last_unstale":"2024-05-17T22:02:56.428878+0000","last_undegraded":"2024-05-17T22:02:56.428878+0000","last_fullsized":"2024-05-17T22:02:56.428878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730980+0000","last_change":"2024-05-17T22:02:54.416676+0000","last_active":"2024-05-17T22:02:56.730980+0000","last_peered":"2024-05-17T22:02:56.730980+0000","last_clean":"2024-05-17T22:02:56.730980+0000","last_became_active":"2024-05-17T22:02:54.416029+0000","last_became_peered":"2024-05-17T22:02:54.416029+0000","last_unstale":"2024-05-17T22:02:56.730980+0000","last_undegraded":"2024-05-17T22:02:56.730980+0000","last_fullsized":"2024-05-17T22:02:56.730980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:11.410888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465391+0000","last_change":"2024-05-17T22:02:55.481747+0000","last_active":"2024-05-17T22:02:56.465391+0000","last_peered":"2024-05-17T22:02:56.465391+0000","last_clean":"2024-05-17T22:02:56.465391+0000","last_became_active":"2024-05-17T22:02:55.481388+0000","last_became_peered":"2024-05-17T22:02:55.481388+0000","last_unstale":"2024-05-17T22:02:56.465391+0000","last_undegraded":"2024-05-17T22:02:56.465391+0000","last_fullsized":"2024-05-17T22:02:56.465391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729720+0000","last_change":"2024-05-17T22:02:54.416134+0000","last_active":"2024-05-17T22:02:56.729720+0000","last_peered":"2024-05-17T22:02:56.729720+0000","last_clean":"2024-05-17T22:02:56.729720+0000","last_became_active":"2024-05-17T22:02:54.415341+0000","last_became_peered":"2024-05-17T22:02:54.415341+0000","last_unstale":"2024-05-17T22:02:56.729720+0000","last_undegraded":"2024-05-17T22:02:56.729720+0000","last_fullsized":"2024-05-17T22:02:56.729720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:53.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431406+0000","last_change":"2024-05-17T22:02:55.457738+0000","last_active":"2024-05-17T22:02:56.431406+0000","last_peered":"2024-05-17T22:02:56.431406+0000","last_clean":"2024-05-17T22:02:56.431406+0000","last_became_active":"2024-05-17T22:02:55.456892+0000","last_became_peered":"2024-05-17T22:02:55.456892+0000","last_unstale":"2024-05-17T22:02:56.431406+0000","last_undegraded":"2024-05-17T22:02:56.431406+0000","last_fullsized":"2024-05-17T22:02:56.431406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428130+0000","last_change":"2024-05-17T22:02:54.393801+0000","last_active":"2024-05-17T22:02:56.428130+0000","last_peered":"2024-05-17T22:02:56.428130+0000","last_clean":"2024-05-17T22:02:56.428130+0000","last_became_active":"2024-05-17T22:02:54.392702+0000","last_became_peered":"2024-05-17T22:02:54.392702+0000","last_unstale":"2024-05-17T22:02:56.428130+0000","last_undegraded":"2024-05-17T22:02:56.428130+0000","last_fullsized":"2024-05-17T22:02:56.428130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:49.441722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465217+0000","last_change":"2024-05-17T22:02:55.447687+0000","last_active":"2024-05-17T22:02:56.465217+0000","last_peered":"2024-05-17T22:02:56.465217+0000","last_clean":"2024-05-17T22:02:56.465217+0000","last_became_active":"2024-05-17T22:02:55.446703+0000","last_became_peered":"2024-05-17T22:02:55.446703+0000","last_unstale":"2024-05-17T22:02:56.465217+0000","last_undegraded":"2024-05-17T22:02:56.465217+0000","last_fullsized":"2024-05-17T22:02:56.465217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461944+0000","last_change":"2024-05-17T22:02:54.406336+0000","last_active":"2024-05-17T22:02:56.461944+0000","last_peered":"2024-05-17T22:02:56.461944+0000","last_clean":"2024-05-17T22:02:56.461944+0000","last_became_active":"2024-05-17T22:02:54.405705+0000","last_became_peered":"2024-05-17T22:02:54.405705+0000","last_unstale":"2024-05-17T22:02:56.461944+0000","last_undegraded":"2024-05-17T22:02:56.461944+0000","last_fullsized":"2024-05-17T22:02:56.461944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:22:24.237054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729757+0000","last_change":"2024-05-17T22:02:55.460348+0000","last_active":"2024-05-17T22:02:56.729757+0000","last_peered":"2024-05-17T22:02:56.729757+0000","last_clean":"2024-05-17T22:02:56.729757+0000","last_became_active":"2024-05-17T22:02:55.459599+0000","last_became_peered":"2024-05-17T22:02:55.459599+0000","last_unstale":"2024-05-17T22:02:56.729757+0000","last_undegraded":"2024-05-17T22:02:56.729757+0000","last_fullsized":"2024-05-17T22:02:56.729757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:25.275714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728773+0000","last_change":"2024-05-17T22:02:54.393151+0000","last_active":"2024-05-17T22:02:56.728773+0000","last_peered":"2024-05-17T22:02:56.728773+0000","last_clean":"2024-05-17T22:02:56.728773+0000","last_became_active":"2024-05-17T22:02:54.392839+0000","last_became_peered":"2024-05-17T22:02:54.392839+0000","last_unstale":"2024-05-17T22:02:56.728773+0000","last_undegraded":"2024-05-17T22:02:56.728773+0000","last_fullsized":"2024-05-17T22:02:56.728773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:57.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461239+0000","last_change":"2024-05-17T22:02:55.452845+0000","last_active":"2024-05-17T22:02:56.461239+0000","last_peered":"2024-05-17T22:02:56.461239+0000","last_clean":"2024-05-17T22:02:56.461239+0000","last_became_active":"2024-05-17T22:02:55.452067+0000","last_became_peered":"2024-05-17T22:02:55.452067+0000","last_unstale":"2024-05-17T22:02:56.461239+0000","last_undegraded":"2024-05-17T22:02:56.461239+0000","last_fullsized":"2024-05-17T22:02:56.461239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:49:31.364847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427562+0000","last_change":"2024-05-17T22:02:54.386211+0000","last_active":"2024-05-17T22:02:56.427562+0000","last_peered":"2024-05-17T22:02:56.427562+0000","last_clean":"2024-05-17T22:02:56.427562+0000","last_became_active":"2024-05-17T22:02:54.385379+0000","last_became_peered":"2024-05-17T22:02:54.385379+0000","last_unstale":"2024-05-17T22:02:56.427562+0000","last_undegraded":"2024-05-17T22:02:56.427562+0000","last_fullsized":"2024-05-17T22:02:56.427562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:11.410888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462092+0000","last_change":"2024-05-17T22:02:55.465981+0000","last_active":"2024-05-17T22:02:56.462092+0000","last_peered":"2024-05-17T22:02:56.462092+0000","last_clean":"2024-05-17T22:02:56.462092+0000","last_became_active":"2024-05-17T22:02:55.465641+0000","last_became_peered":"2024-05-17T22:02:55.465641+0000","last_unstale":"2024-05-17T22:02:56.462092+0000","last_undegraded":"2024-05-17T22:02:56.462092+0000","last_fullsized":"2024-05-17T22:02:56.462092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:51.585951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.464818+0000","last_change":"2024-05-17T22:02:54.398314+0000","last_active":"2024-05-17T22:02:56.464818+0000","last_peered":"2024-05-17T22:02:56.464818+0000","last_clean":"2024-05-17T22:02:56.464818+0000","last_became_active":"2024-05-17T22:02:54.397616+0000","last_became_peered":"2024-05-17T22:02:54.397616+0000","last_unstale":"2024-05-17T22:02:56.464818+0000","last_undegraded":"2024-05-17T22:02:56.464818+0000","last_fullsized":"2024-05-17T22:02:56.464818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:16.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730286+0000","last_change":"2024-05-17T22:02:55.444995+0000","last_active":"2024-05-17T22:02:56.730286+0000","last_peered":"2024-05-17T22:02:56.730286+0000","last_clean":"2024-05-17T22:02:56.730286+0000","last_became_active":"2024-05-17T22:02:55.444212+0000","last_became_peered":"2024-05-17T22:02:55.444212+0000","last_unstale":"2024-05-17T22:02:56.730286+0000","last_undegraded":"2024-05-17T22:02:56.730286+0000","last_fullsized":"2024-05-17T22:02:56.730286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:58.062085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.732004+0000","last_change":"2024-05-17T22:02:54.412199+0000","last_active":"2024-05-17T22:02:56.732004+0000","last_peered":"2024-05-17T22:02:56.732004+0000","last_clean":"2024-05-17T22:02:56.732004+0000","last_became_active":"2024-05-17T22:02:54.409862+0000","last_became_peered":"2024-05-17T22:02:54.409862+0000","last_unstale":"2024-05-17T22:02:56.732004+0000","last_undegraded":"2024-05-17T22:02:56.732004+0000","last_fullsized":"2024-05-17T22:02:56.732004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:49.441722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731796+0000","last_change":"2024-05-17T22:02:55.463993+0000","last_active":"2024-05-17T22:02:56.731796+0000","last_peered":"2024-05-17T22:02:56.731796+0000","last_clean":"2024-05-17T22:02:56.731796+0000","last_became_active":"2024-05-17T22:02:55.461164+0000","last_became_peered":"2024-05-17T22:02:55.461164+0000","last_unstale":"2024-05-17T22:02:56.731796+0000","last_undegraded":"2024-05-17T22:02:56.731796+0000","last_fullsized":"2024-05-17T22:02:56.731796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:40.911868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465138+0000","last_change":"2024-05-17T22:02:54.397851+0000","last_active":"2024-05-17T22:02:56.465138+0000","last_peered":"2024-05-17T22:02:56.465138+0000","last_clean":"2024-05-17T22:02:56.465138+0000","last_became_active":"2024-05-17T22:02:54.397362+0000","last_became_peered":"2024-05-17T22:02:54.397362+0000","last_unstale":"2024-05-17T22:02:56.465138+0000","last_undegraded":"2024-05-17T22:02:56.465138+0000","last_fullsized":"2024-05-17T22:02:56.465138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:25:55.756695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431932+0000","last_change":"2024-05-17T22:02:55.448618+0000","last_active":"2024-05-17T22:02:56.431932+0000","last_peered":"2024-05-17T22:02:56.431932+0000","last_clean":"2024-05-17T22:02:56.431932+0000","last_became_active":"2024-05-17T22:02:55.447783+0000","last_became_peered":"2024-05-17T22:02:55.447783+0000","last_unstale":"2024-05-17T22:02:56.431932+0000","last_undegraded":"2024-05-17T22:02:56.431932+0000","last_fullsized":"2024-05-17T22:02:56.431932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465667+0000","last_change":"2024-05-17T22:02:54.398472+0000","last_active":"2024-05-17T22:02:56.465667+0000","last_peered":"2024-05-17T22:02:56.465667+0000","last_clean":"2024-05-17T22:02:56.465667+0000","last_became_active":"2024-05-17T22:02:54.398056+0000","last_became_peered":"2024-05-17T22:02:54.398056+0000","last_unstale":"2024-05-17T22:02:56.465667+0000","last_undegraded":"2024-05-17T22:02:56.465667+0000","last_fullsized":"2024-05-17T22:02:56.465667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:38.439857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427824+0000","last_change":"2024-05-17T22:02:55.444260+0000","last_active":"2024-05-17T22:02:56.427824+0000","last_peered":"2024-05-17T22:02:56.427824+0000","last_clean":"2024-05-17T22:02:56.427824+0000","last_became_active":"2024-05-17T22:02:55.442971+0000","last_became_peered":"2024-05-17T22:02:55.442971+0000","last_unstale":"2024-05-17T22:02:56.427824+0000","last_undegraded":"2024-05-17T22:02:56.427824+0000","last_fullsized":"2024-05-17T22:02:56.427824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:48.600098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728916+0000","last_change":"2024-05-17T22:02:54.399695+0000","last_active":"2024-05-17T22:02:56.728916+0000","last_peered":"2024-05-17T22:02:56.728916+0000","last_clean":"2024-05-17T22:02:56.728916+0000","last_became_active":"2024-05-17T22:02:54.399171+0000","last_became_peered":"2024-05-17T22:02:54.399171+0000","last_unstale":"2024-05-17T22:02:56.728916+0000","last_undegraded":"2024-05-17T22:02:56.728916+0000","last_fullsized":"2024-05-17T22:02:56.728916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:39.900776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427868+0000","last_change":"2024-05-17T22:02:55.465414+0000","last_active":"2024-05-17T22:02:56.427868+0000","last_peered":"2024-05-17T22:02:56.427868+0000","last_clean":"2024-05-17T22:02:56.427868+0000","last_became_active":"2024-05-17T22:02:55.464269+0000","last_became_peered":"2024-05-17T22:02:55.464269+0000","last_unstale":"2024-05-17T22:02:56.427868+0000","last_undegraded":"2024-05-17T22:02:56.427868+0000","last_fullsized":"2024-05-17T22:02:56.427868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:17.940143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427888+0000","last_change":"2024-05-17T22:02:54.386690+0000","last_active":"2024-05-17T22:02:56.427888+0000","last_peered":"2024-05-17T22:02:56.427888+0000","last_clean":"2024-05-17T22:02:56.427888+0000","last_became_active":"2024-05-17T22:02:54.385798+0000","last_became_peered":"2024-05-17T22:02:54.385798+0000","last_unstale":"2024-05-17T22:02:56.427888+0000","last_undegraded":"2024-05-17T22:02:56.427888+0000","last_fullsized":"2024-05-17T22:02:56.427888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:48:00.466420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427496+0000","last_change":"2024-05-17T22:02:55.439967+0000","last_active":"2024-05-17T22:02:56.427496+0000","last_peered":"2024-05-17T22:02:56.427496+0000","last_clean":"2024-05-17T22:02:56.427496+0000","last_became_active":"2024-05-17T22:02:55.439064+0000","last_became_peered":"2024-05-17T22:02:55.439064+0000","last_unstale":"2024-05-17T22:02:56.427496+0000","last_undegraded":"2024-05-17T22:02:56.427496+0000","last_fullsized":"2024-05-17T22:02:56.427496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:45:20.691126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427272+0000","last_change":"2024-05-17T22:02:54.413236+0000","last_active":"2024-05-17T22:02:56.427272+0000","last_peered":"2024-05-17T22:02:56.427272+0000","last_clean":"2024-05-17T22:02:56.427272+0000","last_became_active":"2024-05-17T22:02:54.412819+0000","last_became_peered":"2024-05-17T22:02:54.412819+0000","last_unstale":"2024-05-17T22:02:56.427272+0000","last_undegraded":"2024-05-17T22:02:56.427272+0000","last_fullsized":"2024-05-17T22:02:56.427272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:54:53.697247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427762+0000","last_change":"2024-05-17T22:02:55.457764+0000","last_active":"2024-05-17T22:02:56.427762+0000","last_peered":"2024-05-17T22:02:56.427762+0000","last_clean":"2024-05-17T22:02:56.427762+0000","last_became_active":"2024-05-17T22:02:55.456878+0000","last_became_peered":"2024-05-17T22:02:55.456878+0000","last_unstale":"2024-05-17T22:02:56.427762+0000","last_undegraded":"2024-05-17T22:02:56.427762+0000","last_fullsized":"2024-05-17T22:02:56.427762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:31.755542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728391+0000","last_change":"2024-05-17T22:02:54.396900+0000","last_active":"2024-05-17T22:02:56.728391+0000","last_peered":"2024-05-17T22:02:56.728391+0000","last_clean":"2024-05-17T22:02:56.728391+0000","last_became_active":"2024-05-17T22:02:54.396097+0000","last_became_peered":"2024-05-17T22:02:54.396097+0000","last_unstale":"2024-05-17T22:02:56.728391+0000","last_undegraded":"2024-05-17T22:02:56.728391+0000","last_fullsized":"2024-05-17T22:02:56.728391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:24.264622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431909+0000","last_change":"2024-05-17T22:02:55.432380+0000","last_active":"2024-05-17T22:02:56.431909+0000","last_peered":"2024-05-17T22:02:56.431909+0000","last_clean":"2024-05-17T22:02:56.431909+0000","last_became_active":"2024-05-17T22:02:55.431487+0000","last_became_peered":"2024-05-17T22:02:55.431487+0000","last_unstale":"2024-05-17T22:02:56.431909+0000","last_undegraded":"2024-05-17T22:02:56.431909+0000","last_fullsized":"2024-05-17T22:02:56.431909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461568+0000","last_change":"2024-05-17T22:02:54.410940+0000","last_active":"2024-05-17T22:02:56.461568+0000","last_peered":"2024-05-17T22:02:56.461568+0000","last_clean":"2024-05-17T22:02:56.461568+0000","last_became_active":"2024-05-17T22:02:54.410550+0000","last_became_peered":"2024-05-17T22:02:54.410550+0000","last_unstale":"2024-05-17T22:02:56.461568+0000","last_undegraded":"2024-05-17T22:02:56.461568+0000","last_fullsized":"2024-05-17T22:02:56.461568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:18:03.408849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730836+0000","last_change":"2024-05-17T22:02:55.483463+0000","last_active":"2024-05-17T22:02:56.730836+0000","last_peered":"2024-05-17T22:02:56.730836+0000","last_clean":"2024-05-17T22:02:56.730836+0000","last_became_active":"2024-05-17T22:02:55.482986+0000","last_became_peered":"2024-05-17T22:02:55.482986+0000","last_unstale":"2024-05-17T22:02:56.730836+0000","last_undegraded":"2024-05-17T22:02:56.730836+0000","last_fullsized":"2024-05-17T22:02:56.730836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:51.053150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462115+0000","last_change":"2024-05-17T22:02:54.408812+0000","last_active":"2024-05-17T22:02:56.462115+0000","last_peered":"2024-05-17T22:02:56.462115+0000","last_clean":"2024-05-17T22:02:56.462115+0000","last_became_active":"2024-05-17T22:02:54.406328+0000","last_became_peered":"2024-05-17T22:02:54.406328+0000","last_unstale":"2024-05-17T22:02:56.462115+0000","last_undegraded":"2024-05-17T22:02:56.462115+0000","last_fullsized":"2024-05-17T22:02:56.462115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:54:13.304269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730907+0000","last_change":"2024-05-17T22:02:55.464133+0000","last_active":"2024-05-17T22:02:56.730907+0000","last_peered":"2024-05-17T22:02:56.730907+0000","last_clean":"2024-05-17T22:02:56.730907+0000","last_became_active":"2024-05-17T22:02:55.460987+0000","last_became_peered":"2024-05-17T22:02:55.460987+0000","last_unstale":"2024-05-17T22:02:56.730907+0000","last_undegraded":"2024-05-17T22:02:56.730907+0000","last_fullsized":"2024-05-17T22:02:56.730907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730295+0000","last_change":"2024-05-17T22:02:54.413441+0000","last_active":"2024-05-17T22:02:56.730295+0000","last_peered":"2024-05-17T22:02:56.730295+0000","last_clean":"2024-05-17T22:02:56.730295+0000","last_became_active":"2024-05-17T22:02:54.412648+0000","last_became_peered":"2024-05-17T22:02:54.412648+0000","last_unstale":"2024-05-17T22:02:56.730295+0000","last_undegraded":"2024-05-17T22:02:56.730295+0000","last_fullsized":"2024-05-17T22:02:56.730295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:50.574859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431944+0000","last_change":"2024-05-17T22:02:55.451943+0000","last_active":"2024-05-17T22:02:56.431944+0000","last_peered":"2024-05-17T22:02:56.431944+0000","last_clean":"2024-05-17T22:02:56.431944+0000","last_became_active":"2024-05-17T22:02:55.451102+0000","last_became_peered":"2024-05-17T22:02:55.451102+0000","last_unstale":"2024-05-17T22:02:56.431944+0000","last_undegraded":"2024-05-17T22:02:56.431944+0000","last_fullsized":"2024-05-17T22:02:56.431944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730942+0000","last_change":"2024-05-17T22:02:54.422699+0000","last_active":"2024-05-17T22:02:56.730942+0000","last_peered":"2024-05-17T22:02:56.730942+0000","last_clean":"2024-05-17T22:02:56.730942+0000","last_became_active":"2024-05-17T22:02:54.421691+0000","last_became_peered":"2024-05-17T22:02:54.421691+0000","last_unstale":"2024-05-17T22:02:56.730942+0000","last_undegraded":"2024-05-17T22:02:56.730942+0000","last_fullsized":"2024-05-17T22:02:56.730942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:34:52.466385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428269+0000","last_change":"2024-05-17T22:02:55.463101+0000","last_active":"2024-05-17T22:02:56.428269+0000","last_peered":"2024-05-17T22:02:56.428269+0000","last_clean":"2024-05-17T22:02:56.428269+0000","last_became_active":"2024-05-17T22:02:55.462125+0000","last_became_peered":"2024-05-17T22:02:55.462125+0000","last_unstale":"2024-05-17T22:02:56.428269+0000","last_undegraded":"2024-05-17T22:02:56.428269+0000","last_fullsized":"2024-05-17T22:02:56.428269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465590+0000","last_change":"2024-05-17T22:02:54.403677+0000","last_active":"2024-05-17T22:02:56.465590+0000","last_peered":"2024-05-17T22:02:56.465590+0000","last_clean":"2024-05-17T22:02:56.465590+0000","last_became_active":"2024-05-17T22:02:54.402873+0000","last_became_peered":"2024-05-17T22:02:54.402873+0000","last_unstale":"2024-05-17T22:02:56.465590+0000","last_undegraded":"2024-05-17T22:02:56.465590+0000","last_fullsized":"2024-05-17T22:02:56.465590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:03:23.012119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465384+0000","last_change":"2024-05-17T22:02:55.458818+0000","last_active":"2024-05-17T22:02:56.465384+0000","last_peered":"2024-05-17T22:02:56.465384+0000","last_clean":"2024-05-17T22:02:56.465384+0000","last_became_active":"2024-05-17T22:02:55.458476+0000","last_became_peered":"2024-05-17T22:02:55.458476+0000","last_unstale":"2024-05-17T22:02:56.465384+0000","last_undegraded":"2024-05-17T22:02:56.465384+0000","last_fullsized":"2024-05-17T22:02:56.465384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430525+0000","last_change":"2024-05-17T22:02:54.403726+0000","last_active":"2024-05-17T22:02:56.430525+0000","last_peered":"2024-05-17T22:02:56.430525+0000","last_clean":"2024-05-17T22:02:56.430525+0000","last_became_active":"2024-05-17T22:02:54.401563+0000","last_became_peered":"2024-05-17T22:02:54.401563+0000","last_unstale":"2024-05-17T22:02:56.430525+0000","last_undegraded":"2024-05-17T22:02:56.430525+0000","last_fullsized":"2024-05-17T22:02:56.430525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:34:52.466385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729099+0000","last_change":"2024-05-17T22:02:55.449730+0000","last_active":"2024-05-17T22:02:56.729099+0000","last_peered":"2024-05-17T22:02:56.729099+0000","last_clean":"2024-05-17T22:02:56.729099+0000","last_became_active":"2024-05-17T22:02:55.448832+0000","last_became_peered":"2024-05-17T22:02:55.448832+0000","last_unstale":"2024-05-17T22:02:56.729099+0000","last_undegraded":"2024-05-17T22:02:56.729099+0000","last_fullsized":"2024-05-17T22:02:56.729099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731574+0000","last_change":"2024-05-17T22:02:54.422584+0000","last_active":"2024-05-17T22:02:56.731574+0000","last_peered":"2024-05-17T22:02:56.731574+0000","last_clean":"2024-05-17T22:02:56.731574+0000","last_became_active":"2024-05-17T22:02:54.421511+0000","last_became_peered":"2024-05-17T22:02:54.421511+0000","last_unstale":"2024-05-17T22:02:56.731574+0000","last_undegraded":"2024-05-17T22:02:56.731574+0000","last_fullsized":"2024-05-17T22:02:56.731574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:44:55.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731137+0000","last_change":"2024-05-17T22:02:55.474539+0000","last_active":"2024-05-17T22:02:56.731137+0000","last_peered":"2024-05-17T22:02:56.731137+0000","last_clean":"2024-05-17T22:02:56.731137+0000","last_became_active":"2024-05-17T22:02:55.472592+0000","last_became_peered":"2024-05-17T22:02:55.472592+0000","last_unstale":"2024-05-17T22:02:56.731137+0000","last_undegraded":"2024-05-17T22:02:56.731137+0000","last_fullsized":"2024-05-17T22:02:56.731137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429888+0000","last_change":"2024-05-17T22:02:54.414153+0000","last_active":"2024-05-17T22:02:56.429888+0000","last_peered":"2024-05-17T22:02:56.429888+0000","last_clean":"2024-05-17T22:02:56.429888+0000","last_became_active":"2024-05-17T22:02:54.413341+0000","last_became_peered":"2024-05-17T22:02:54.413341+0000","last_unstale":"2024-05-17T22:02:56.429888+0000","last_undegraded":"2024-05-17T22:02:56.429888+0000","last_fullsized":"2024-05-17T22:02:56.429888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:50.574859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461775+0000","last_change":"2024-05-17T22:02:55.442696+0000","last_active":"2024-05-17T22:02:56.461775+0000","last_peered":"2024-05-17T22:02:56.461775+0000","last_clean":"2024-05-17T22:02:56.461775+0000","last_became_active":"2024-05-17T22:02:55.441734+0000","last_became_peered":"2024-05-17T22:02:55.441734+0000","last_unstale":"2024-05-17T22:02:56.461775+0000","last_undegraded":"2024-05-17T22:02:56.461775+0000","last_fullsized":"2024-05-17T22:02:56.461775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:34.658305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729916+0000","last_change":"2024-05-17T22:02:54.411980+0000","last_active":"2024-05-17T22:02:56.729916+0000","last_peered":"2024-05-17T22:02:56.729916+0000","last_clean":"2024-05-17T22:02:56.729916+0000","last_became_active":"2024-05-17T22:02:54.411048+0000","last_became_peered":"2024-05-17T22:02:54.411048+0000","last_unstale":"2024-05-17T22:02:56.729916+0000","last_undegraded":"2024-05-17T22:02:56.729916+0000","last_fullsized":"2024-05-17T22:02:56.729916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:25:55.756695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":221,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729470+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:02:56.729470+0000","last_peered":"2024-05-17T22:02:56.729470+0000","last_clean":"2024-05-17T22:02:56.729470+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:02:56.729470+0000","last_undegraded":"2024-05-17T22:02:56.729470+0000","last_fullsized":"2024-05-17T22:02:56.729470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1013532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":423788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":424728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":423094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":424706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":423676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":423288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1013496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:02:59.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:02:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 Processor -- start 2024-05-17T22:02:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 -- start start 2024-05-17T22:02:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8000d0150 0x7fa80006c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa80006ccd0 0x7fa800163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80006f7e0 con 0x7fa800164190 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa80006d0e0 con 0x7fa80006ccd0 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa8074a2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa8000cfcb0 con 0x7fa8000d0150 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa80006ccd0 0x7fa800163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa80006ccd0 0x7fa800163c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60580/0 (socket says 172.21.15.83:60580) 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 -- 172.21.15.83:0/1859416725 learned_addr learned my addr 172.21.15.83:0/1859416725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 -- 172.21.15.83:0/1859416725 >> v2:172.21.15.155:3300/0 conn(0x7fa8000d0150 msgr2=0x7fa80006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 --2- 172.21.15.83:0/1859416725 >> v2:172.21.15.155:3300/0 conn(0x7fa8000d0150 0x7fa80006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 -- 172.21.15.83:0/1859416725 >> v1:172.21.15.155:6789/0 conn(0x7fa800164190 legacy=0x7fa8000ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:02:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.742+0000 7fa804a3d640 1 -- 172.21.15.83:0/1859416725 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8000d1e50 con 0x7fa80006ccd0 2024-05-17T22:02:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa804a3d640 1 --2- 172.21.15.83:0/1859416725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80006ccd0 0x7fa800163c50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa800074000 tx=0x7fa7f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f475e7bb36422 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa7f7fff640 1 -- 172.21.15.83:0/1859416725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00aa430 con 0x7fa80006ccd0 2024-05-17T22:02:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa7f7fff640 1 -- 172.21.15.83:0/1859416725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f00af070 con 0x7fa80006ccd0 2024-05-17T22:02:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa7f7fff640 1 -- 172.21.15.83:0/1859416725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00aa7f0 con 0x7fa80006ccd0 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 -- 172.21.15.83:0/1859416725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80006ccd0 msgr2=0x7fa800163c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 --2- 172.21.15.83:0/1859416725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80006ccd0 0x7fa800163c50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa800074000 tx=0x7fa7f001c4f0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 -- 172.21.15.83:0/1859416725 shutdown_connections 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 --2- 172.21.15.83:0/1859416725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa80006ccd0 0x7fa800163c50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 --2- 172.21.15.83:0/1859416725 >> v2:172.21.15.155:3300/0 conn(0x7fa8000d0150 0x7fa80006c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 -- 172.21.15.83:0/1859416725 >> 172.21.15.83:0/1859416725 conn(0x7fa8000b4170 msgr2=0x7fa8000b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 -- 172.21.15.83:0/1859416725 shutdown_connections 2024-05-17T22:02:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.746+0000 7fa8074a2640 1 -- 172.21.15.83:0/1859416725 wait complete. 2024-05-17T22:02:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 Processor -- start 2024-05-17T22:02:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- start start 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 0x7fa8000c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8000d0150 0x7fa800152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000cfa50 con 0x7fa800164190 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000cf8d0 con 0x7fa80006ccd0 2024-05-17T22:02:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8001603e0 con 0x7fa8000d0150 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60584/0 (socket says 172.21.15.83:60584) 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 -- 172.21.15.83:0/117739157 learned_addr learned my addr 172.21.15.83:0/117739157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa80523e640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 0x7fa8000c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 -- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8000d0150 msgr2=0x7fa800152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa804a3d640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8000d0150 0x7fa800152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8000d0150 0x7fa800152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 -- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 msgr2=0x7fa8000c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 0x7fa8000c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 -- 172.21.15.83:0/117739157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa800159cd0 con 0x7fa800164190 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa80523e640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 0x7fa8000c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa805a3f640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b960 tx=0x7fa7f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8019070 con 0x7fa800164190 2024-05-17T22:02:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f800cb20 con 0x7fa800164190 2024-05-17T22:02:59.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8014440 con 0x7fa800164190 2024-05-17T22:02:59.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80015a1d0 con 0x7fa800164190 2024-05-17T22:02:59.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80015a790 con 0x7fa800164190 2024-05-17T22:02:59.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.750+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa800159f40 con 0x7fa800164190 2024-05-17T22:02:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.754+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa7f801c070 con 0x7fa800164190 2024-05-17T22:02:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.754+0000 7fa7f5ffb640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 0x7fa7e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:02:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.754+0000 7fa80523e640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 0x7fa7e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:02:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.754+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fa7f808ef70 con 0x7fa800164190 2024-05-17T22:02:59.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.754+0000 7fa80523e640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 0x7fa7e4071910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa80006fee0 tx=0x7fa7fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:02:59.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.758+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa7f805be60 con 0x7fa800164190 2024-05-17T22:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.914+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8000b7f10 con 0x7fa7e406f400 2024-05-17T22:02:59.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.926+0000 7fa7f5ffb640 1 -- 172.21.15.83:0/117739157 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252071 (secure 0 0 0) 0x7fa8000b7f10 con 0x7fa7e406f400 2024-05-17T22:02:59.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:02:59.922 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 msgr2=0x7fa7e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 0x7fa7e4071910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa80006fee0 tx=0x7fa7fc008040 comp rx=0 tx=0).stop 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 msgr2=0x7fa80015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b960 tx=0x7fa7f800be20 comp rx=0 tx=0).stop 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 shutdown_connections 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa800164190 0x7fa80015b270 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8000d0150 0x7fa800152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7e406f400 0x7fa7e4071910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 --2- 172.21.15.83:0/117739157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa80006ccd0 0x7fa8000c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:02:59.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 >> 172.21.15.83:0/117739157 conn(0x7fa8000b4170 msgr2=0x7fa800158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:02:59.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 shutdown_connections 2024-05-17T22:02:59.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:02:59.930+0000 7fa8074a2640 1 -- 172.21.15.83:0/117739157 wait complete. 2024-05-17T22:02:59.942 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-05-17T22:02:59.087499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258440,"kb_used_data":9000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716280,"statfs":{"total":773094113280,"available":772829470720,"internally_reserved":0,"allocated":9216000,"data_stored":4570308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.721210"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731529+0000","last_change":"2024-05-17T22:02:55.481664+0000","last_active":"2024-05-17T22:02:56.731529+0000","last_peered":"2024-05-17T22:02:56.731529+0000","last_clean":"2024-05-17T22:02:56.731529+0000","last_became_active":"2024-05-17T22:02:55.481035+0000","last_became_peered":"2024-05-17T22:02:55.481035+0000","last_unstale":"2024-05-17T22:02:56.731529+0000","last_undegraded":"2024-05-17T22:02:56.731529+0000","last_fullsized":"2024-05-17T22:02:56.731529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:25.275714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462026+0000","last_change":"2024-05-17T22:02:55.481814+0000","last_active":"2024-05-17T22:02:56.462026+0000","last_peered":"2024-05-17T22:02:56.462026+0000","last_clean":"2024-05-17T22:02:56.462026+0000","last_became_active":"2024-05-17T22:02:55.481436+0000","last_became_peered":"2024-05-17T22:02:55.481436+0000","last_unstale":"2024-05-17T22:02:56.462026+0000","last_undegraded":"2024-05-17T22:02:56.462026+0000","last_fullsized":"2024-05-17T22:02:56.462026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:52:27.148640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729816+0000","last_change":"2024-05-17T22:02:55.482132+0000","last_active":"2024-05-17T22:02:56.729816+0000","last_peered":"2024-05-17T22:02:56.729816+0000","last_clean":"2024-05-17T22:02:56.729816+0000","last_became_active":"2024-05-17T22:02:55.481564+0000","last_became_peered":"2024-05-17T22:02:55.481564+0000","last_unstale":"2024-05-17T22:02:56.729816+0000","last_undegraded":"2024-05-17T22:02:56.729816+0000","last_fullsized":"2024-05-17T22:02:56.729816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:40.911868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428743+0000","last_change":"2024-05-17T22:02:55.452224+0000","last_active":"2024-05-17T22:02:56.428743+0000","last_peered":"2024-05-17T22:02:56.428743+0000","last_clean":"2024-05-17T22:02:56.428743+0000","last_became_active":"2024-05-17T22:02:55.450992+0000","last_became_peered":"2024-05-17T22:02:55.450992+0000","last_unstale":"2024-05-17T22:02:56.428743+0000","last_undegraded":"2024-05-17T22:02:56.428743+0000","last_fullsized":"2024-05-17T22:02:56.428743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:02.356697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431170+0000","last_change":"2024-05-17T22:02:55.446573+0000","last_active":"2024-05-17T22:02:56.431170+0000","last_peered":"2024-05-17T22:02:56.431170+0000","last_clean":"2024-05-17T22:02:56.431170+0000","last_became_active":"2024-05-17T22:02:55.445716+0000","last_became_peered":"2024-05-17T22:02:55.445716+0000","last_unstale":"2024-05-17T22:02:56.431170+0000","last_undegraded":"2024-05-17T22:02:56.431170+0000","last_fullsized":"2024-05-17T22:02:56.431170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427758+0000","last_change":"2024-05-17T22:02:55.455553+0000","last_active":"2024-05-17T22:02:56.427758+0000","last_peered":"2024-05-17T22:02:56.427758+0000","last_clean":"2024-05-17T22:02:56.427758+0000","last_became_active":"2024-05-17T22:02:55.454741+0000","last_became_peered":"2024-05-17T22:02:55.454741+0000","last_unstale":"2024-05-17T22:02:56.427758+0000","last_undegraded":"2024-05-17T22:02:56.427758+0000","last_fullsized":"2024-05-17T22:02:56.427758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:03:24.023210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427509+0000","last_change":"2024-05-17T22:02:55.444843+0000","last_active":"2024-05-17T22:02:56.427509+0000","last_peered":"2024-05-17T22:02:56.427509+0000","last_clean":"2024-05-17T22:02:56.427509+0000","last_became_active":"2024-05-17T22:02:55.442085+0000","last_became_peered":"2024-05-17T22:02:55.442085+0000","last_unstale":"2024-05-17T22:02:56.427509+0000","last_undegraded":"2024-05-17T22:02:56.427509+0000","last_fullsized":"2024-05-17T22:02:56.427509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:24:56.114494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428084+0000","last_change":"2024-05-17T22:02:55.442971+0000","last_active":"2024-05-17T22:02:56.428084+0000","last_peered":"2024-05-17T22:02:56.428084+0000","last_clean":"2024-05-17T22:02:56.428084+0000","last_became_active":"2024-05-17T22:02:55.442107+0000","last_became_peered":"2024-05-17T22:02:55.442107+0000","last_unstale":"2024-05-17T22:02:56.428084+0000","last_undegraded":"2024-05-17T22:02:56.428084+0000","last_fullsized":"2024-05-17T22:02:56.428084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:53:00.432103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428777+0000","last_change":"2024-05-17T22:02:55.442865+0000","last_active":"2024-05-17T22:02:56.428777+0000","last_peered":"2024-05-17T22:02:56.428777+0000","last_clean":"2024-05-17T22:02:56.428777+0000","last_became_active":"2024-05-17T22:02:55.441970+0000","last_became_peered":"2024-05-17T22:02:55.441970+0000","last_unstale":"2024-05-17T22:02:56.428777+0000","last_undegraded":"2024-05-17T22:02:56.428777+0000","last_fullsized":"2024-05-17T22:02:56.428777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730887+0000","last_change":"2024-05-17T22:02:55.466764+0000","last_active":"2024-05-17T22:02:56.730887+0000","last_peered":"2024-05-17T22:02:56.730887+0000","last_clean":"2024-05-17T22:02:56.730887+0000","last_became_active":"2024-05-17T22:02:55.465851+0000","last_became_peered":"2024-05-17T22:02:55.465851+0000","last_unstale":"2024-05-17T22:02:56.730887+0000","last_undegraded":"2024-05-17T22:02:56.730887+0000","last_fullsized":"2024-05-17T22:02:56.730887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461648+0000","last_change":"2024-05-17T22:02:55.450794+0000","last_active":"2024-05-17T22:02:56.461648+0000","last_peered":"2024-05-17T22:02:56.461648+0000","last_clean":"2024-05-17T22:02:56.461648+0000","last_became_active":"2024-05-17T22:02:55.449661+0000","last_became_peered":"2024-05-17T22:02:55.449661+0000","last_unstale":"2024-05-17T22:02:56.461648+0000","last_undegraded":"2024-05-17T22:02:56.461648+0000","last_fullsized":"2024-05-17T22:02:56.461648+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:24:07.738454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730839+0000","last_change":"2024-05-17T22:02:55.465513+0000","last_active":"2024-05-17T22:02:56.730839+0000","last_peered":"2024-05-17T22:02:56.730839+0000","last_clean":"2024-05-17T22:02:56.730839+0000","last_became_active":"2024-05-17T22:02:55.464930+0000","last_became_peered":"2024-05-17T22:02:55.464930+0000","last_unstale":"2024-05-17T22:02:56.730839+0000","last_undegraded":"2024-05-17T22:02:56.730839+0000","last_fullsized":"2024-05-17T22:02:56.730839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729445+0000","last_change":"2024-05-17T22:02:55.442051+0000","last_active":"2024-05-17T22:02:56.729445+0000","last_peered":"2024-05-17T22:02:56.729445+0000","last_clean":"2024-05-17T22:02:56.729445+0000","last_became_active":"2024-05-17T22:02:55.440836+0000","last_became_peered":"2024-05-17T22:02:55.440836+0000","last_unstale":"2024-05-17T22:02:56.729445+0000","last_undegraded":"2024-05-17T22:02:56.729445+0000","last_fullsized":"2024-05-17T22:02:56.729445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427649+0000","last_change":"2024-05-17T22:02:55.451950+0000","last_active":"2024-05-17T22:02:56.427649+0000","last_peered":"2024-05-17T22:02:56.427649+0000","last_clean":"2024-05-17T22:02:56.427649+0000","last_became_active":"2024-05-17T22:02:55.451126+0000","last_became_peered":"2024-05-17T22:02:55.451126+0000","last_unstale":"2024-05-17T22:02:56.427649+0000","last_undegraded":"2024-05-17T22:02:56.427649+0000","last_fullsized":"2024-05-17T22:02:56.427649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:27:57.929783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728805+0000","last_change":"2024-05-17T22:02:55.460969+0000","last_active":"2024-05-17T22:02:56.728805+0000","last_peered":"2024-05-17T22:02:56.728805+0000","last_clean":"2024-05-17T22:02:56.728805+0000","last_became_active":"2024-05-17T22:02:55.460374+0000","last_became_peered":"2024-05-17T22:02:55.460374+0000","last_unstale":"2024-05-17T22:02:56.728805+0000","last_undegraded":"2024-05-17T22:02:56.728805+0000","last_fullsized":"2024-05-17T22:02:56.728805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:57.177274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429940+0000","last_change":"2024-05-17T22:02:55.474041+0000","last_active":"2024-05-17T22:02:56.429940+0000","last_peered":"2024-05-17T22:02:56.429940+0000","last_clean":"2024-05-17T22:02:56.429940+0000","last_became_active":"2024-05-17T22:02:55.473270+0000","last_became_peered":"2024-05-17T22:02:55.473270+0000","last_unstale":"2024-05-17T22:02:56.429940+0000","last_undegraded":"2024-05-17T22:02:56.429940+0000","last_fullsized":"2024-05-17T22:02:56.429940+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:31.755542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431765+0000","last_change":"2024-05-17T22:02:55.453993+0000","last_active":"2024-05-17T22:02:56.431765+0000","last_peered":"2024-05-17T22:02:56.431765+0000","last_clean":"2024-05-17T22:02:56.431765+0000","last_became_active":"2024-05-17T22:02:55.453177+0000","last_became_peered":"2024-05-17T22:02:55.453177+0000","last_unstale":"2024-05-17T22:02:56.431765+0000","last_undegraded":"2024-05-17T22:02:56.431765+0000","last_fullsized":"2024-05-17T22:02:56.431765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427649+0000","last_change":"2024-05-17T22:02:55.468588+0000","last_active":"2024-05-17T22:02:56.427649+0000","last_peered":"2024-05-17T22:02:56.427649+0000","last_clean":"2024-05-17T22:02:56.427649+0000","last_became_active":"2024-05-17T22:02:55.468282+0000","last_became_peered":"2024-05-17T22:02:55.468282+0000","last_unstale":"2024-05-17T22:02:56.427649+0000","last_undegraded":"2024-05-17T22:02:56.427649+0000","last_fullsized":"2024-05-17T22:02:56.427649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:03:52.766961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465601+0000","last_change":"2024-05-17T22:02:55.470291+0000","last_active":"2024-05-17T22:02:56.465601+0000","last_peered":"2024-05-17T22:02:56.465601+0000","last_clean":"2024-05-17T22:02:56.465601+0000","last_became_active":"2024-05-17T22:02:55.469913+0000","last_became_peered":"2024-05-17T22:02:55.469913+0000","last_unstale":"2024-05-17T22:02:56.465601+0000","last_undegraded":"2024-05-17T22:02:56.465601+0000","last_fullsized":"2024-05-17T22:02:56.465601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429112+0000","last_change":"2024-05-17T22:02:55.442185+0000","last_active":"2024-05-17T22:02:56.429112+0000","last_peered":"2024-05-17T22:02:56.429112+0000","last_clean":"2024-05-17T22:02:56.429112+0000","last_became_active":"2024-05-17T22:02:55.441517+0000","last_became_peered":"2024-05-17T22:02:55.441517+0000","last_unstale":"2024-05-17T22:02:56.429112+0000","last_undegraded":"2024-05-17T22:02:56.429112+0000","last_fullsized":"2024-05-17T22:02:56.429112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:23.407351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428306+0000","last_change":"2024-05-17T22:02:55.457330+0000","last_active":"2024-05-17T22:02:56.428306+0000","last_peered":"2024-05-17T22:02:56.428306+0000","last_clean":"2024-05-17T22:02:56.428306+0000","last_became_active":"2024-05-17T22:02:55.456770+0000","last_became_peered":"2024-05-17T22:02:55.456770+0000","last_unstale":"2024-05-17T22:02:56.428306+0000","last_undegraded":"2024-05-17T22:02:56.428306+0000","last_fullsized":"2024-05-17T22:02:56.428306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:11.630824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461326+0000","last_change":"2024-05-17T22:02:55.456909+0000","last_active":"2024-05-17T22:02:56.461326+0000","last_peered":"2024-05-17T22:02:56.461326+0000","last_clean":"2024-05-17T22:02:56.461326+0000","last_became_active":"2024-05-17T22:02:55.456507+0000","last_became_peered":"2024-05-17T22:02:55.456507+0000","last_unstale":"2024-05-17T22:02:56.461326+0000","last_undegraded":"2024-05-17T22:02:56.461326+0000","last_fullsized":"2024-05-17T22:02:56.461326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:02:02.154981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731536+0000","last_change":"2024-05-17T22:02:55.463482+0000","last_active":"2024-05-17T22:02:56.731536+0000","last_peered":"2024-05-17T22:02:56.731536+0000","last_clean":"2024-05-17T22:02:56.731536+0000","last_became_active":"2024-05-17T22:02:55.460891+0000","last_became_peered":"2024-05-17T22:02:55.460891+0000","last_unstale":"2024-05-17T22:02:56.731536+0000","last_undegraded":"2024-05-17T22:02:56.731536+0000","last_fullsized":"2024-05-17T22:02:56.731536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427412+0000","last_change":"2024-05-17T22:02:55.457666+0000","last_active":"2024-05-17T22:02:56.427412+0000","last_peered":"2024-05-17T22:02:56.427412+0000","last_clean":"2024-05-17T22:02:56.427412+0000","last_became_active":"2024-05-17T22:02:55.457268+0000","last_became_peered":"2024-05-17T22:02:55.457268+0000","last_unstale":"2024-05-17T22:02:56.427412+0000","last_undegraded":"2024-05-17T22:02:56.427412+0000","last_fullsized":"2024-05-17T22:02:56.427412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:48:11.061535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461886+0000","last_change":"2024-05-17T22:02:55.463006+0000","last_active":"2024-05-17T22:02:56.461886+0000","last_peered":"2024-05-17T22:02:56.461886+0000","last_clean":"2024-05-17T22:02:56.461886+0000","last_became_active":"2024-05-17T22:02:55.462634+0000","last_became_peered":"2024-05-17T22:02:55.462634+0000","last_unstale":"2024-05-17T22:02:56.461886+0000","last_undegraded":"2024-05-17T22:02:56.461886+0000","last_fullsized":"2024-05-17T22:02:56.461886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:31.719755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729411+0000","last_change":"2024-05-17T22:02:55.449570+0000","last_active":"2024-05-17T22:02:56.729411+0000","last_peered":"2024-05-17T22:02:56.729411+0000","last_clean":"2024-05-17T22:02:56.729411+0000","last_became_active":"2024-05-17T22:02:55.448799+0000","last_became_peered":"2024-05-17T22:02:55.448799+0000","last_unstale":"2024-05-17T22:02:56.729411+0000","last_undegraded":"2024-05-17T22:02:56.729411+0000","last_fullsized":"2024-05-17T22:02:56.729411+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431391+0000","last_change":"2024-05-17T22:02:55.464728+0000","last_active":"2024-05-17T22:02:56.431391+0000","last_peered":"2024-05-17T22:02:56.431391+0000","last_clean":"2024-05-17T22:02:56.431391+0000","last_became_active":"2024-05-17T22:02:55.464036+0000","last_became_peered":"2024-05-17T22:02:55.464036+0000","last_unstale":"2024-05-17T22:02:56.431391+0000","last_undegraded":"2024-05-17T22:02:56.431391+0000","last_fullsized":"2024-05-17T22:02:56.431391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730559+0000","last_change":"2024-05-17T22:02:55.466563+0000","last_active":"2024-05-17T22:02:56.730559+0000","last_peered":"2024-05-17T22:02:56.730559+0000","last_clean":"2024-05-17T22:02:56.730559+0000","last_became_active":"2024-05-17T22:02:55.466028+0000","last_became_peered":"2024-05-17T22:02:55.466028+0000","last_unstale":"2024-05-17T22:02:56.730559+0000","last_undegraded":"2024-05-17T22:02:56.730559+0000","last_fullsized":"2024-05-17T22:02:56.730559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462093+0000","last_change":"2024-05-17T22:02:55.463246+0000","last_active":"2024-05-17T22:02:56.462093+0000","last_peered":"2024-05-17T22:02:56.462093+0000","last_clean":"2024-05-17T22:02:56.462093+0000","last_became_active":"2024-05-17T22:02:55.462919+0000","last_became_peered":"2024-05-17T22:02:55.462919+0000","last_unstale":"2024-05-17T22:02:56.462093+0000","last_undegraded":"2024-05-17T22:02:56.462093+0000","last_fullsized":"2024-05-17T22:02:56.462093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:54.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461970+0000","last_change":"2024-05-17T22:02:55.455660+0000","last_active":"2024-05-17T22:02:56.461970+0000","last_peered":"2024-05-17T22:02:56.461970+0000","last_clean":"2024-05-17T22:02:56.461970+0000","last_became_active":"2024-05-17T22:02:55.455230+0000","last_became_peered":"2024-05-17T22:02:55.455230+0000","last_unstale":"2024-05-17T22:02:56.461970+0000","last_undegraded":"2024-05-17T22:02:56.461970+0000","last_fullsized":"2024-05-17T22:02:56.461970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:27:34.886684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430161+0000","last_change":"2024-05-17T22:02:55.456874+0000","last_active":"2024-05-17T22:02:56.430161+0000","last_peered":"2024-05-17T22:02:56.430161+0000","last_clean":"2024-05-17T22:02:56.430161+0000","last_became_active":"2024-05-17T22:02:55.456052+0000","last_became_peered":"2024-05-17T22:02:55.456052+0000","last_unstale":"2024-05-17T22:02:56.430161+0000","last_undegraded":"2024-05-17T22:02:56.430161+0000","last_fullsized":"2024-05-17T22:02:56.430161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:17.940143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729160+0000","last_change":"2024-05-17T22:02:55.470082+0000","last_active":"2024-05-17T22:02:56.729160+0000","last_peered":"2024-05-17T22:02:56.729160+0000","last_clean":"2024-05-17T22:02:56.729160+0000","last_became_active":"2024-05-17T22:02:55.461057+0000","last_became_peered":"2024-05-17T22:02:55.461057+0000","last_unstale":"2024-05-17T22:02:56.729160+0000","last_undegraded":"2024-05-17T22:02:56.729160+0000","last_fullsized":"2024-05-17T22:02:56.729160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431532+0000","last_change":"2024-05-17T22:02:55.481982+0000","last_active":"2024-05-17T22:02:56.431532+0000","last_peered":"2024-05-17T22:02:56.431532+0000","last_clean":"2024-05-17T22:02:56.431532+0000","last_became_active":"2024-05-17T22:02:55.481442+0000","last_became_peered":"2024-05-17T22:02:55.481442+0000","last_unstale":"2024-05-17T22:02:56.431532+0000","last_undegraded":"2024-05-17T22:02:56.431532+0000","last_fullsized":"2024-05-17T22:02:56.431532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731685+0000","last_change":"2024-05-17T22:02:54.415327+0000","last_active":"2024-05-17T22:02:56.731685+0000","last_peered":"2024-05-17T22:02:56.731685+0000","last_clean":"2024-05-17T22:02:56.731685+0000","last_became_active":"2024-05-17T22:02:54.414446+0000","last_became_peered":"2024-05-17T22:02:54.414446+0000","last_unstale":"2024-05-17T22:02:56.731685+0000","last_undegraded":"2024-05-17T22:02:56.731685+0000","last_fullsized":"2024-05-17T22:02:56.731685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:54.142353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731592+0000","last_change":"2024-05-17T22:02:55.472359+0000","last_active":"2024-05-17T22:02:56.731592+0000","last_peered":"2024-05-17T22:02:56.731592+0000","last_clean":"2024-05-17T22:02:56.731592+0000","last_became_active":"2024-05-17T22:02:55.471369+0000","last_became_peered":"2024-05-17T22:02:55.471369+0000","last_unstale":"2024-05-17T22:02:56.731592+0000","last_undegraded":"2024-05-17T22:02:56.731592+0000","last_fullsized":"2024-05-17T22:02:56.731592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:52:50.952620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431913+0000","last_change":"2024-05-17T22:02:54.406636+0000","last_active":"2024-05-17T22:02:56.431913+0000","last_peered":"2024-05-17T22:02:56.431913+0000","last_clean":"2024-05-17T22:02:56.431913+0000","last_became_active":"2024-05-17T22:02:54.396309+0000","last_became_peered":"2024-05-17T22:02:54.396309+0000","last_unstale":"2024-05-17T22:02:56.431913+0000","last_undegraded":"2024-05-17T22:02:56.431913+0000","last_fullsized":"2024-05-17T22:02:56.431913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:37:54.142353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730404+0000","last_change":"2024-05-17T22:02:55.464575+0000","last_active":"2024-05-17T22:02:56.730404+0000","last_peered":"2024-05-17T22:02:56.730404+0000","last_clean":"2024-05-17T22:02:56.730404+0000","last_became_active":"2024-05-17T22:02:55.463588+0000","last_became_peered":"2024-05-17T22:02:55.463588+0000","last_unstale":"2024-05-17T22:02:56.730404+0000","last_undegraded":"2024-05-17T22:02:56.730404+0000","last_fullsized":"2024-05-17T22:02:56.730404+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:10:10.008749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465661+0000","last_change":"2024-05-17T22:02:54.400480+0000","last_active":"2024-05-17T22:02:56.465661+0000","last_peered":"2024-05-17T22:02:56.465661+0000","last_clean":"2024-05-17T22:02:56.465661+0000","last_became_active":"2024-05-17T22:02:54.400075+0000","last_became_peered":"2024-05-17T22:02:54.400075+0000","last_unstale":"2024-05-17T22:02:56.465661+0000","last_undegraded":"2024-05-17T22:02:56.465661+0000","last_fullsized":"2024-05-17T22:02:56.465661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:19:35.392245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729789+0000","last_change":"2024-05-17T22:02:55.453211+0000","last_active":"2024-05-17T22:02:56.729789+0000","last_peered":"2024-05-17T22:02:56.729789+0000","last_clean":"2024-05-17T22:02:56.729789+0000","last_became_active":"2024-05-17T22:02:55.452273+0000","last_became_peered":"2024-05-17T22:02:55.452273+0000","last_unstale":"2024-05-17T22:02:56.729789+0000","last_undegraded":"2024-05-17T22:02:56.729789+0000","last_fullsized":"2024-05-17T22:02:56.729789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:52:50.952620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465555+0000","last_change":"2024-05-17T22:02:54.397665+0000","last_active":"2024-05-17T22:02:56.465555+0000","last_peered":"2024-05-17T22:02:56.465555+0000","last_clean":"2024-05-17T22:02:56.465555+0000","last_became_active":"2024-05-17T22:02:54.397243+0000","last_became_peered":"2024-05-17T22:02:54.397243+0000","last_unstale":"2024-05-17T22:02:56.465555+0000","last_undegraded":"2024-05-17T22:02:56.465555+0000","last_fullsized":"2024-05-17T22:02:56.465555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:22:08.923158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461606+0000","last_change":"2024-05-17T22:02:55.455212+0000","last_active":"2024-05-17T22:02:56.461606+0000","last_peered":"2024-05-17T22:02:56.461606+0000","last_clean":"2024-05-17T22:02:56.461606+0000","last_became_active":"2024-05-17T22:02:55.454724+0000","last_became_peered":"2024-05-17T22:02:55.454724+0000","last_unstale":"2024-05-17T22:02:56.461606+0000","last_undegraded":"2024-05-17T22:02:56.461606+0000","last_fullsized":"2024-05-17T22:02:56.461606+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:01:18.432736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431399+0000","last_change":"2024-05-17T22:02:54.389397+0000","last_active":"2024-05-17T22:02:56.431399+0000","last_peered":"2024-05-17T22:02:56.431399+0000","last_clean":"2024-05-17T22:02:56.431399+0000","last_became_active":"2024-05-17T22:02:54.388478+0000","last_became_peered":"2024-05-17T22:02:54.388478+0000","last_unstale":"2024-05-17T22:02:56.431399+0000","last_undegraded":"2024-05-17T22:02:56.431399+0000","last_fullsized":"2024-05-17T22:02:56.431399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:44:55.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431166+0000","last_change":"2024-05-17T22:02:55.455488+0000","last_active":"2024-05-17T22:02:56.431166+0000","last_peered":"2024-05-17T22:02:56.431166+0000","last_clean":"2024-05-17T22:02:56.431166+0000","last_became_active":"2024-05-17T22:02:55.454734+0000","last_became_peered":"2024-05-17T22:02:55.454734+0000","last_unstale":"2024-05-17T22:02:56.431166+0000","last_undegraded":"2024-05-17T22:02:56.431166+0000","last_fullsized":"2024-05-17T22:02:56.431166+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:53:00.432103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430110+0000","last_change":"2024-05-17T22:02:54.406571+0000","last_active":"2024-05-17T22:02:56.430110+0000","last_peered":"2024-05-17T22:02:56.430110+0000","last_clean":"2024-05-17T22:02:56.430110+0000","last_became_active":"2024-05-17T22:02:54.402851+0000","last_became_peered":"2024-05-17T22:02:54.402851+0000","last_unstale":"2024-05-17T22:02:56.430110+0000","last_undegraded":"2024-05-17T22:02:56.430110+0000","last_fullsized":"2024-05-17T22:02:56.430110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:53.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729060+0000","last_change":"2024-05-17T22:02:55.459888+0000","last_active":"2024-05-17T22:02:56.729060+0000","last_peered":"2024-05-17T22:02:56.729060+0000","last_clean":"2024-05-17T22:02:56.729060+0000","last_became_active":"2024-05-17T22:02:55.458995+0000","last_became_peered":"2024-05-17T22:02:55.458995+0000","last_unstale":"2024-05-17T22:02:56.729060+0000","last_undegraded":"2024-05-17T22:02:56.729060+0000","last_fullsized":"2024-05-17T22:02:56.729060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.464972+0000","last_change":"2024-05-17T22:02:54.396552+0000","last_active":"2024-05-17T22:02:56.464972+0000","last_peered":"2024-05-17T22:02:56.464972+0000","last_clean":"2024-05-17T22:02:56.464972+0000","last_became_active":"2024-05-17T22:02:54.396215+0000","last_became_peered":"2024-05-17T22:02:54.396215+0000","last_unstale":"2024-05-17T22:02:56.464972+0000","last_undegraded":"2024-05-17T22:02:56.464972+0000","last_fullsized":"2024-05-17T22:02:56.464972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:30.744450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729007+0000","last_change":"2024-05-17T22:02:55.460668+0000","last_active":"2024-05-17T22:02:56.729007+0000","last_peered":"2024-05-17T22:02:56.729007+0000","last_clean":"2024-05-17T22:02:56.729007+0000","last_became_active":"2024-05-17T22:02:55.459765+0000","last_became_peered":"2024-05-17T22:02:55.459765+0000","last_unstale":"2024-05-17T22:02:56.729007+0000","last_undegraded":"2024-05-17T22:02:56.729007+0000","last_fullsized":"2024-05-17T22:02:56.729007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:54:52.905846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461333+0000","last_change":"2024-05-17T22:02:54.390692+0000","last_active":"2024-05-17T22:02:56.461333+0000","last_peered":"2024-05-17T22:02:56.461333+0000","last_clean":"2024-05-17T22:02:56.461333+0000","last_became_active":"2024-05-17T22:02:54.390299+0000","last_became_peered":"2024-05-17T22:02:54.390299+0000","last_unstale":"2024-05-17T22:02:56.461333+0000","last_undegraded":"2024-05-17T22:02:56.461333+0000","last_fullsized":"2024-05-17T22:02:56.461333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:13:22.396260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729378+0000","last_change":"2024-05-17T22:02:55.461488+0000","last_active":"2024-05-17T22:02:56.729378+0000","last_peered":"2024-05-17T22:02:56.729378+0000","last_clean":"2024-05-17T22:02:56.729378+0000","last_became_active":"2024-05-17T22:02:55.461154+0000","last_became_peered":"2024-05-17T22:02:55.461154+0000","last_unstale":"2024-05-17T22:02:56.729378+0000","last_undegraded":"2024-05-17T22:02:56.729378+0000","last_fullsized":"2024-05-17T22:02:56.729378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:27:38.926685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731849+0000","last_change":"2024-05-17T22:02:54.422715+0000","last_active":"2024-05-17T22:02:56.731849+0000","last_peered":"2024-05-17T22:02:56.731849+0000","last_clean":"2024-05-17T22:02:56.731849+0000","last_became_active":"2024-05-17T22:02:54.421613+0000","last_became_peered":"2024-05-17T22:02:54.421613+0000","last_unstale":"2024-05-17T22:02:56.731849+0000","last_undegraded":"2024-05-17T22:02:56.731849+0000","last_fullsized":"2024-05-17T22:02:56.731849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:35:50.871077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428878+0000","last_change":"2024-05-17T22:02:55.439866+0000","last_active":"2024-05-17T22:02:56.428878+0000","last_peered":"2024-05-17T22:02:56.428878+0000","last_clean":"2024-05-17T22:02:56.428878+0000","last_became_active":"2024-05-17T22:02:55.439021+0000","last_became_peered":"2024-05-17T22:02:55.439021+0000","last_unstale":"2024-05-17T22:02:56.428878+0000","last_undegraded":"2024-05-17T22:02:56.428878+0000","last_fullsized":"2024-05-17T22:02:56.428878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730980+0000","last_change":"2024-05-17T22:02:54.416676+0000","last_active":"2024-05-17T22:02:56.730980+0000","last_peered":"2024-05-17T22:02:56.730980+0000","last_clean":"2024-05-17T22:02:56.730980+0000","last_became_active":"2024-05-17T22:02:54.416029+0000","last_became_peered":"2024-05-17T22:02:54.416029+0000","last_unstale":"2024-05-17T22:02:56.730980+0000","last_undegraded":"2024-05-17T22:02:56.730980+0000","last_fullsized":"2024-05-17T22:02:56.730980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:11.410888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465391+0000","last_change":"2024-05-17T22:02:55.481747+0000","last_active":"2024-05-17T22:02:56.465391+0000","last_peered":"2024-05-17T22:02:56.465391+0000","last_clean":"2024-05-17T22:02:56.465391+0000","last_became_active":"2024-05-17T22:02:55.481388+0000","last_became_peered":"2024-05-17T22:02:55.481388+0000","last_unstale":"2024-05-17T22:02:56.465391+0000","last_undegraded":"2024-05-17T22:02:56.465391+0000","last_fullsized":"2024-05-17T22:02:56.465391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729720+0000","last_change":"2024-05-17T22:02:54.416134+0000","last_active":"2024-05-17T22:02:56.729720+0000","last_peered":"2024-05-17T22:02:56.729720+0000","last_clean":"2024-05-17T22:02:56.729720+0000","last_became_active":"2024-05-17T22:02:54.415341+0000","last_became_peered":"2024-05-17T22:02:54.415341+0000","last_unstale":"2024-05-17T22:02:56.729720+0000","last_undegraded":"2024-05-17T22:02:56.729720+0000","last_fullsized":"2024-05-17T22:02:56.729720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:53.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431406+0000","last_change":"2024-05-17T22:02:55.457738+0000","last_active":"2024-05-17T22:02:56.431406+0000","last_peered":"2024-05-17T22:02:56.431406+0000","last_clean":"2024-05-17T22:02:56.431406+0000","last_became_active":"2024-05-17T22:02:55.456892+0000","last_became_peered":"2024-05-17T22:02:55.456892+0000","last_unstale":"2024-05-17T22:02:56.431406+0000","last_undegraded":"2024-05-17T22:02:56.431406+0000","last_fullsized":"2024-05-17T22:02:56.431406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:34:37.187025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428130+0000","last_change":"2024-05-17T22:02:54.393801+0000","last_active":"2024-05-17T22:02:56.428130+0000","last_peered":"2024-05-17T22:02:56.428130+0000","last_clean":"2024-05-17T22:02:56.428130+0000","last_became_active":"2024-05-17T22:02:54.392702+0000","last_became_peered":"2024-05-17T22:02:54.392702+0000","last_unstale":"2024-05-17T22:02:56.428130+0000","last_undegraded":"2024-05-17T22:02:56.428130+0000","last_fullsized":"2024-05-17T22:02:56.428130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:49.441722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465217+0000","last_change":"2024-05-17T22:02:55.447687+0000","last_active":"2024-05-17T22:02:56.465217+0000","last_peered":"2024-05-17T22:02:56.465217+0000","last_clean":"2024-05-17T22:02:56.465217+0000","last_became_active":"2024-05-17T22:02:55.446703+0000","last_became_peered":"2024-05-17T22:02:55.446703+0000","last_unstale":"2024-05-17T22:02:56.465217+0000","last_undegraded":"2024-05-17T22:02:56.465217+0000","last_fullsized":"2024-05-17T22:02:56.465217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:49.524725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461944+0000","last_change":"2024-05-17T22:02:54.406336+0000","last_active":"2024-05-17T22:02:56.461944+0000","last_peered":"2024-05-17T22:02:56.461944+0000","last_clean":"2024-05-17T22:02:56.461944+0000","last_became_active":"2024-05-17T22:02:54.405705+0000","last_became_peered":"2024-05-17T22:02:54.405705+0000","last_unstale":"2024-05-17T22:02:56.461944+0000","last_undegraded":"2024-05-17T22:02:56.461944+0000","last_fullsized":"2024-05-17T22:02:56.461944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:22:24.237054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729757+0000","last_change":"2024-05-17T22:02:55.460348+0000","last_active":"2024-05-17T22:02:56.729757+0000","last_peered":"2024-05-17T22:02:56.729757+0000","last_clean":"2024-05-17T22:02:56.729757+0000","last_became_active":"2024-05-17T22:02:55.459599+0000","last_became_peered":"2024-05-17T22:02:55.459599+0000","last_unstale":"2024-05-17T22:02:56.729757+0000","last_undegraded":"2024-05-17T22:02:56.729757+0000","last_fullsized":"2024-05-17T22:02:56.729757+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:25.275714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728773+0000","last_change":"2024-05-17T22:02:54.393151+0000","last_active":"2024-05-17T22:02:56.728773+0000","last_peered":"2024-05-17T22:02:56.728773+0000","last_clean":"2024-05-17T22:02:56.728773+0000","last_became_active":"2024-05-17T22:02:54.392839+0000","last_became_peered":"2024-05-17T22:02:54.392839+0000","last_unstale":"2024-05-17T22:02:56.728773+0000","last_undegraded":"2024-05-17T22:02:56.728773+0000","last_fullsized":"2024-05-17T22:02:56.728773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:57.050994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461239+0000","last_change":"2024-05-17T22:02:55.452845+0000","last_active":"2024-05-17T22:02:56.461239+0000","last_peered":"2024-05-17T22:02:56.461239+0000","last_clean":"2024-05-17T22:02:56.461239+0000","last_became_active":"2024-05-17T22:02:55.452067+0000","last_became_peered":"2024-05-17T22:02:55.452067+0000","last_unstale":"2024-05-17T22:02:56.461239+0000","last_undegraded":"2024-05-17T22:02:56.461239+0000","last_fullsized":"2024-05-17T22:02:56.461239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:49:31.364847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427562+0000","last_change":"2024-05-17T22:02:54.386211+0000","last_active":"2024-05-17T22:02:56.427562+0000","last_peered":"2024-05-17T22:02:56.427562+0000","last_clean":"2024-05-17T22:02:56.427562+0000","last_became_active":"2024-05-17T22:02:54.385379+0000","last_became_peered":"2024-05-17T22:02:54.385379+0000","last_unstale":"2024-05-17T22:02:56.427562+0000","last_undegraded":"2024-05-17T22:02:56.427562+0000","last_fullsized":"2024-05-17T22:02:56.427562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:00:11.410888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462092+0000","last_change":"2024-05-17T22:02:55.465981+0000","last_active":"2024-05-17T22:02:56.462092+0000","last_peered":"2024-05-17T22:02:56.462092+0000","last_clean":"2024-05-17T22:02:56.462092+0000","last_became_active":"2024-05-17T22:02:55.465641+0000","last_became_peered":"2024-05-17T22:02:55.465641+0000","last_unstale":"2024-05-17T22:02:56.462092+0000","last_undegraded":"2024-05-17T22:02:56.462092+0000","last_fullsized":"2024-05-17T22:02:56.462092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:51.585951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.464818+0000","last_change":"2024-05-17T22:02:54.398314+0000","last_active":"2024-05-17T22:02:56.464818+0000","last_peered":"2024-05-17T22:02:56.464818+0000","last_clean":"2024-05-17T22:02:56.464818+0000","last_became_active":"2024-05-17T22:02:54.397616+0000","last_became_peered":"2024-05-17T22:02:54.397616+0000","last_unstale":"2024-05-17T22:02:56.464818+0000","last_undegraded":"2024-05-17T22:02:56.464818+0000","last_fullsized":"2024-05-17T22:02:56.464818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:16.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730286+0000","last_change":"2024-05-17T22:02:55.444995+0000","last_active":"2024-05-17T22:02:56.730286+0000","last_peered":"2024-05-17T22:02:56.730286+0000","last_clean":"2024-05-17T22:02:56.730286+0000","last_became_active":"2024-05-17T22:02:55.444212+0000","last_became_peered":"2024-05-17T22:02:55.444212+0000","last_unstale":"2024-05-17T22:02:56.730286+0000","last_undegraded":"2024-05-17T22:02:56.730286+0000","last_fullsized":"2024-05-17T22:02:56.730286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:04:58.062085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.732004+0000","last_change":"2024-05-17T22:02:54.412199+0000","last_active":"2024-05-17T22:02:56.732004+0000","last_peered":"2024-05-17T22:02:56.732004+0000","last_clean":"2024-05-17T22:02:56.732004+0000","last_became_active":"2024-05-17T22:02:54.409862+0000","last_became_peered":"2024-05-17T22:02:54.409862+0000","last_unstale":"2024-05-17T22:02:56.732004+0000","last_undegraded":"2024-05-17T22:02:56.732004+0000","last_fullsized":"2024-05-17T22:02:56.732004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:49.441722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731796+0000","last_change":"2024-05-17T22:02:55.463993+0000","last_active":"2024-05-17T22:02:56.731796+0000","last_peered":"2024-05-17T22:02:56.731796+0000","last_clean":"2024-05-17T22:02:56.731796+0000","last_became_active":"2024-05-17T22:02:55.461164+0000","last_became_peered":"2024-05-17T22:02:55.461164+0000","last_unstale":"2024-05-17T22:02:56.731796+0000","last_undegraded":"2024-05-17T22:02:56.731796+0000","last_fullsized":"2024-05-17T22:02:56.731796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:40.911868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465138+0000","last_change":"2024-05-17T22:02:54.397851+0000","last_active":"2024-05-17T22:02:56.465138+0000","last_peered":"2024-05-17T22:02:56.465138+0000","last_clean":"2024-05-17T22:02:56.465138+0000","last_became_active":"2024-05-17T22:02:54.397362+0000","last_became_peered":"2024-05-17T22:02:54.397362+0000","last_unstale":"2024-05-17T22:02:56.465138+0000","last_undegraded":"2024-05-17T22:02:56.465138+0000","last_fullsized":"2024-05-17T22:02:56.465138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:25:55.756695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431932+0000","last_change":"2024-05-17T22:02:55.448618+0000","last_active":"2024-05-17T22:02:56.431932+0000","last_peered":"2024-05-17T22:02:56.431932+0000","last_clean":"2024-05-17T22:02:56.431932+0000","last_became_active":"2024-05-17T22:02:55.447783+0000","last_became_peered":"2024-05-17T22:02:55.447783+0000","last_unstale":"2024-05-17T22:02:56.431932+0000","last_undegraded":"2024-05-17T22:02:56.431932+0000","last_fullsized":"2024-05-17T22:02:56.431932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465667+0000","last_change":"2024-05-17T22:02:54.398472+0000","last_active":"2024-05-17T22:02:56.465667+0000","last_peered":"2024-05-17T22:02:56.465667+0000","last_clean":"2024-05-17T22:02:56.465667+0000","last_became_active":"2024-05-17T22:02:54.398056+0000","last_became_peered":"2024-05-17T22:02:54.398056+0000","last_unstale":"2024-05-17T22:02:56.465667+0000","last_undegraded":"2024-05-17T22:02:56.465667+0000","last_fullsized":"2024-05-17T22:02:56.465667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:26:38.439857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427824+0000","last_change":"2024-05-17T22:02:55.444260+0000","last_active":"2024-05-17T22:02:56.427824+0000","last_peered":"2024-05-17T22:02:56.427824+0000","last_clean":"2024-05-17T22:02:56.427824+0000","last_became_active":"2024-05-17T22:02:55.442971+0000","last_became_peered":"2024-05-17T22:02:55.442971+0000","last_unstale":"2024-05-17T22:02:56.427824+0000","last_undegraded":"2024-05-17T22:02:56.427824+0000","last_fullsized":"2024-05-17T22:02:56.427824+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:48.600098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728916+0000","last_change":"2024-05-17T22:02:54.399695+0000","last_active":"2024-05-17T22:02:56.728916+0000","last_peered":"2024-05-17T22:02:56.728916+0000","last_clean":"2024-05-17T22:02:56.728916+0000","last_became_active":"2024-05-17T22:02:54.399171+0000","last_became_peered":"2024-05-17T22:02:54.399171+0000","last_unstale":"2024-05-17T22:02:56.728916+0000","last_undegraded":"2024-05-17T22:02:56.728916+0000","last_fullsized":"2024-05-17T22:02:56.728916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:42:39.900776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427868+0000","last_change":"2024-05-17T22:02:55.465414+0000","last_active":"2024-05-17T22:02:56.427868+0000","last_peered":"2024-05-17T22:02:56.427868+0000","last_clean":"2024-05-17T22:02:56.427868+0000","last_became_active":"2024-05-17T22:02:55.464269+0000","last_became_peered":"2024-05-17T22:02:55.464269+0000","last_unstale":"2024-05-17T22:02:56.427868+0000","last_undegraded":"2024-05-17T22:02:56.427868+0000","last_fullsized":"2024-05-17T22:02:56.427868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:17.940143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427888+0000","last_change":"2024-05-17T22:02:54.386690+0000","last_active":"2024-05-17T22:02:56.427888+0000","last_peered":"2024-05-17T22:02:56.427888+0000","last_clean":"2024-05-17T22:02:56.427888+0000","last_became_active":"2024-05-17T22:02:54.385798+0000","last_became_peered":"2024-05-17T22:02:54.385798+0000","last_unstale":"2024-05-17T22:02:56.427888+0000","last_undegraded":"2024-05-17T22:02:56.427888+0000","last_fullsized":"2024-05-17T22:02:56.427888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:48:00.466420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427496+0000","last_change":"2024-05-17T22:02:55.439967+0000","last_active":"2024-05-17T22:02:56.427496+0000","last_peered":"2024-05-17T22:02:56.427496+0000","last_clean":"2024-05-17T22:02:56.427496+0000","last_became_active":"2024-05-17T22:02:55.439064+0000","last_became_peered":"2024-05-17T22:02:55.439064+0000","last_unstale":"2024-05-17T22:02:56.427496+0000","last_undegraded":"2024-05-17T22:02:56.427496+0000","last_fullsized":"2024-05-17T22:02:56.427496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:45:20.691126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427272+0000","last_change":"2024-05-17T22:02:54.413236+0000","last_active":"2024-05-17T22:02:56.427272+0000","last_peered":"2024-05-17T22:02:56.427272+0000","last_clean":"2024-05-17T22:02:56.427272+0000","last_became_active":"2024-05-17T22:02:54.412819+0000","last_became_peered":"2024-05-17T22:02:54.412819+0000","last_unstale":"2024-05-17T22:02:56.427272+0000","last_undegraded":"2024-05-17T22:02:56.427272+0000","last_fullsized":"2024-05-17T22:02:56.427272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:54:53.697247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.427762+0000","last_change":"2024-05-17T22:02:55.457764+0000","last_active":"2024-05-17T22:02:56.427762+0000","last_peered":"2024-05-17T22:02:56.427762+0000","last_clean":"2024-05-17T22:02:56.427762+0000","last_became_active":"2024-05-17T22:02:55.456878+0000","last_became_peered":"2024-05-17T22:02:55.456878+0000","last_unstale":"2024-05-17T22:02:56.427762+0000","last_undegraded":"2024-05-17T22:02:56.427762+0000","last_fullsized":"2024-05-17T22:02:56.427762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:58:31.755542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.728391+0000","last_change":"2024-05-17T22:02:54.396900+0000","last_active":"2024-05-17T22:02:56.728391+0000","last_peered":"2024-05-17T22:02:56.728391+0000","last_clean":"2024-05-17T22:02:56.728391+0000","last_became_active":"2024-05-17T22:02:54.396097+0000","last_became_peered":"2024-05-17T22:02:54.396097+0000","last_unstale":"2024-05-17T22:02:56.728391+0000","last_undegraded":"2024-05-17T22:02:56.728391+0000","last_fullsized":"2024-05-17T22:02:56.728391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:15:24.264622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431909+0000","last_change":"2024-05-17T22:02:55.432380+0000","last_active":"2024-05-17T22:02:56.431909+0000","last_peered":"2024-05-17T22:02:56.431909+0000","last_clean":"2024-05-17T22:02:56.431909+0000","last_became_active":"2024-05-17T22:02:55.431487+0000","last_became_peered":"2024-05-17T22:02:55.431487+0000","last_unstale":"2024-05-17T22:02:56.431909+0000","last_undegraded":"2024-05-17T22:02:56.431909+0000","last_fullsized":"2024-05-17T22:02:56.431909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461568+0000","last_change":"2024-05-17T22:02:54.410940+0000","last_active":"2024-05-17T22:02:56.461568+0000","last_peered":"2024-05-17T22:02:56.461568+0000","last_clean":"2024-05-17T22:02:56.461568+0000","last_became_active":"2024-05-17T22:02:54.410550+0000","last_became_peered":"2024-05-17T22:02:54.410550+0000","last_unstale":"2024-05-17T22:02:56.461568+0000","last_undegraded":"2024-05-17T22:02:56.461568+0000","last_fullsized":"2024-05-17T22:02:56.461568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:18:03.408849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730836+0000","last_change":"2024-05-17T22:02:55.483463+0000","last_active":"2024-05-17T22:02:56.730836+0000","last_peered":"2024-05-17T22:02:56.730836+0000","last_clean":"2024-05-17T22:02:56.730836+0000","last_became_active":"2024-05-17T22:02:55.482986+0000","last_became_peered":"2024-05-17T22:02:55.482986+0000","last_unstale":"2024-05-17T22:02:56.730836+0000","last_undegraded":"2024-05-17T22:02:56.730836+0000","last_fullsized":"2024-05-17T22:02:56.730836+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:51.053150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.462115+0000","last_change":"2024-05-17T22:02:54.408812+0000","last_active":"2024-05-17T22:02:56.462115+0000","last_peered":"2024-05-17T22:02:56.462115+0000","last_clean":"2024-05-17T22:02:56.462115+0000","last_became_active":"2024-05-17T22:02:54.406328+0000","last_became_peered":"2024-05-17T22:02:54.406328+0000","last_unstale":"2024-05-17T22:02:56.462115+0000","last_undegraded":"2024-05-17T22:02:56.462115+0000","last_fullsized":"2024-05-17T22:02:56.462115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:54:13.304269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730907+0000","last_change":"2024-05-17T22:02:55.464133+0000","last_active":"2024-05-17T22:02:56.730907+0000","last_peered":"2024-05-17T22:02:56.730907+0000","last_clean":"2024-05-17T22:02:56.730907+0000","last_became_active":"2024-05-17T22:02:55.460987+0000","last_became_peered":"2024-05-17T22:02:55.460987+0000","last_unstale":"2024-05-17T22:02:56.730907+0000","last_undegraded":"2024-05-17T22:02:56.730907+0000","last_fullsized":"2024-05-17T22:02:56.730907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730295+0000","last_change":"2024-05-17T22:02:54.413441+0000","last_active":"2024-05-17T22:02:56.730295+0000","last_peered":"2024-05-17T22:02:56.730295+0000","last_clean":"2024-05-17T22:02:56.730295+0000","last_became_active":"2024-05-17T22:02:54.412648+0000","last_became_peered":"2024-05-17T22:02:54.412648+0000","last_unstale":"2024-05-17T22:02:56.730295+0000","last_undegraded":"2024-05-17T22:02:56.730295+0000","last_fullsized":"2024-05-17T22:02:56.730295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:50.574859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.431944+0000","last_change":"2024-05-17T22:02:55.451943+0000","last_active":"2024-05-17T22:02:56.431944+0000","last_peered":"2024-05-17T22:02:56.431944+0000","last_clean":"2024-05-17T22:02:56.431944+0000","last_became_active":"2024-05-17T22:02:55.451102+0000","last_became_peered":"2024-05-17T22:02:55.451102+0000","last_unstale":"2024-05-17T22:02:56.431944+0000","last_undegraded":"2024-05-17T22:02:56.431944+0000","last_fullsized":"2024-05-17T22:02:56.431944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:18:00.387493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.730942+0000","last_change":"2024-05-17T22:02:54.422699+0000","last_active":"2024-05-17T22:02:56.730942+0000","last_peered":"2024-05-17T22:02:56.730942+0000","last_clean":"2024-05-17T22:02:56.730942+0000","last_became_active":"2024-05-17T22:02:54.421691+0000","last_became_peered":"2024-05-17T22:02:54.421691+0000","last_unstale":"2024-05-17T22:02:56.730942+0000","last_undegraded":"2024-05-17T22:02:56.730942+0000","last_fullsized":"2024-05-17T22:02:56.730942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:34:52.466385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.428269+0000","last_change":"2024-05-17T22:02:55.463101+0000","last_active":"2024-05-17T22:02:56.428269+0000","last_peered":"2024-05-17T22:02:56.428269+0000","last_clean":"2024-05-17T22:02:56.428269+0000","last_became_active":"2024-05-17T22:02:55.462125+0000","last_became_peered":"2024-05-17T22:02:55.462125+0000","last_unstale":"2024-05-17T22:02:56.428269+0000","last_undegraded":"2024-05-17T22:02:56.428269+0000","last_fullsized":"2024-05-17T22:02:56.428269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:37.082532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465590+0000","last_change":"2024-05-17T22:02:54.403677+0000","last_active":"2024-05-17T22:02:56.465590+0000","last_peered":"2024-05-17T22:02:56.465590+0000","last_clean":"2024-05-17T22:02:56.465590+0000","last_became_active":"2024-05-17T22:02:54.402873+0000","last_became_peered":"2024-05-17T22:02:54.402873+0000","last_unstale":"2024-05-17T22:02:56.465590+0000","last_undegraded":"2024-05-17T22:02:56.465590+0000","last_fullsized":"2024-05-17T22:02:56.465590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:03:23.012119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.465384+0000","last_change":"2024-05-17T22:02:55.458818+0000","last_active":"2024-05-17T22:02:56.465384+0000","last_peered":"2024-05-17T22:02:56.465384+0000","last_clean":"2024-05-17T22:02:56.465384+0000","last_became_active":"2024-05-17T22:02:55.458476+0000","last_became_peered":"2024-05-17T22:02:55.458476+0000","last_unstale":"2024-05-17T22:02:56.465384+0000","last_undegraded":"2024-05-17T22:02:56.465384+0000","last_fullsized":"2024-05-17T22:02:56.465384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:56:47.011307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.430525+0000","last_change":"2024-05-17T22:02:54.403726+0000","last_active":"2024-05-17T22:02:56.430525+0000","last_peered":"2024-05-17T22:02:56.430525+0000","last_clean":"2024-05-17T22:02:56.430525+0000","last_became_active":"2024-05-17T22:02:54.401563+0000","last_became_peered":"2024-05-17T22:02:54.401563+0000","last_unstale":"2024-05-17T22:02:56.430525+0000","last_undegraded":"2024-05-17T22:02:56.430525+0000","last_fullsized":"2024-05-17T22:02:56.430525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:34:52.466385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729099+0000","last_change":"2024-05-17T22:02:55.449730+0000","last_active":"2024-05-17T22:02:56.729099+0000","last_peered":"2024-05-17T22:02:56.729099+0000","last_clean":"2024-05-17T22:02:56.729099+0000","last_became_active":"2024-05-17T22:02:55.448832+0000","last_became_peered":"2024-05-17T22:02:55.448832+0000","last_unstale":"2024-05-17T22:02:56.729099+0000","last_undegraded":"2024-05-17T22:02:56.729099+0000","last_fullsized":"2024-05-17T22:02:56.729099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:33:44.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731574+0000","last_change":"2024-05-17T22:02:54.422584+0000","last_active":"2024-05-17T22:02:56.731574+0000","last_peered":"2024-05-17T22:02:56.731574+0000","last_clean":"2024-05-17T22:02:56.731574+0000","last_became_active":"2024-05-17T22:02:54.421511+0000","last_became_peered":"2024-05-17T22:02:54.421511+0000","last_unstale":"2024-05-17T22:02:56.731574+0000","last_undegraded":"2024-05-17T22:02:56.731574+0000","last_fullsized":"2024-05-17T22:02:56.731574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:44:55.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.731137+0000","last_change":"2024-05-17T22:02:55.474539+0000","last_active":"2024-05-17T22:02:56.731137+0000","last_peered":"2024-05-17T22:02:56.731137+0000","last_clean":"2024-05-17T22:02:56.731137+0000","last_became_active":"2024-05-17T22:02:55.472592+0000","last_became_peered":"2024-05-17T22:02:55.472592+0000","last_unstale":"2024-05-17T22:02:56.731137+0000","last_undegraded":"2024-05-17T22:02:56.731137+0000","last_fullsized":"2024-05-17T22:02:56.731137+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:13.595173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.429888+0000","last_change":"2024-05-17T22:02:54.414153+0000","last_active":"2024-05-17T22:02:56.429888+0000","last_peered":"2024-05-17T22:02:56.429888+0000","last_clean":"2024-05-17T22:02:56.429888+0000","last_became_active":"2024-05-17T22:02:54.413341+0000","last_became_peered":"2024-05-17T22:02:54.413341+0000","last_unstale":"2024-05-17T22:02:56.429888+0000","last_undegraded":"2024-05-17T22:02:56.429888+0000","last_fullsized":"2024-05-17T22:02:56.429888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:12:50.574859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.461775+0000","last_change":"2024-05-17T22:02:55.442696+0000","last_active":"2024-05-17T22:02:56.461775+0000","last_peered":"2024-05-17T22:02:56.461775+0000","last_clean":"2024-05-17T22:02:56.461775+0000","last_became_active":"2024-05-17T22:02:55.441734+0000","last_became_peered":"2024-05-17T22:02:55.441734+0000","last_unstale":"2024-05-17T22:02:56.461775+0000","last_undegraded":"2024-05-17T22:02:56.461775+0000","last_fullsized":"2024-05-17T22:02:56.461775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:54.362289+0000","last_clean_scrub_stamp":"2024-05-17T22:02:54.362289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:34.658305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729916+0000","last_change":"2024-05-17T22:02:54.411980+0000","last_active":"2024-05-17T22:02:56.729916+0000","last_peered":"2024-05-17T22:02:56.729916+0000","last_clean":"2024-05-17T22:02:56.729916+0000","last_became_active":"2024-05-17T22:02:54.411048+0000","last_became_peered":"2024-05-17T22:02:54.411048+0000","last_unstale":"2024-05-17T22:02:56.729916+0000","last_undegraded":"2024-05-17T22:02:56.729916+0000","last_fullsized":"2024-05-17T22:02:56.729916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:02:53.351197+0000","last_clean_scrub_stamp":"2024-05-17T22:02:53.351197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:25:55.756695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":221,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-17T22:02:56.729470+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:02:56.729470+0000","last_peered":"2024-05-17T22:02:56.729470+0000","last_clean":"2024-05-17T22:02:56.729470+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:02:56.729470+0000","last_undegraded":"2024-05-17T22:02:56.729470+0000","last_fullsized":"2024-05-17T22:02:56.729470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1013532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":423788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":424728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":423094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":424706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":423676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":423288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1013496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:02:59.947 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:02:59.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T22:03:00.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.150+0000 7fefb32ba640 1 Processor -- start 2024-05-17T22:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.150+0000 7fefb32ba640 1 -- start start 2024-05-17T22:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb32ba640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fefac0b4810 0x7fefac0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb32ba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefac0d2000 0x7fefac0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb32ba640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefac0d4170 con 0x7fefac0d0290 2024-05-17T22:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb32ba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fefac0d4450 con 0x7fefac0d2000 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb32ba640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fefac0d4730 con 0x7fefac0b4810 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fefac0d2000 0x7fefac0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb0855640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fefac0d0290 0x7fefac15fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57100/0 (socket says 172.21.15.83:57100) 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb0855640 1 -- 172.21.15.83:0/1820743951 learned_addr learned my addr 172.21.15.83:0/1820743951 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1056640 1 --2- 172.21.15.83:0/1820743951 >> v2:172.21.15.155:3300/0 conn(0x7fefac0b4810 0x7fefac0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 -- 172.21.15.83:0/1820743951 >> v2:172.21.15.155:3300/0 conn(0x7fefac0b4810 msgr2=0x7fefac0d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 --2- 172.21.15.83:0/1820743951 >> v2:172.21.15.155:3300/0 conn(0x7fefac0b4810 0x7fefac0d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 -- 172.21.15.83:0/1820743951 >> v1:172.21.15.83:6789/0 conn(0x7fefac0d0290 legacy=0x7fefac15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 -- 172.21.15.83:0/1820743951 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac0d4a10 con 0x7fefac0d2000 2024-05-17T22:03:00.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fefb1857640 1 --2- 172.21.15.83:0/1820743951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d2000 0x7fefac0d2410 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fefac08e620 tx=0x7fefa801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc0ebb67e655408 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fef9ffff640 1 -- 172.21.15.83:0/1820743951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80aa600 con 0x7fefac0d2000 2024-05-17T22:03:00.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fef9ffff640 1 -- 172.21.15.83:0/1820743951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa80af070 con 0x7fefac0d2000 2024-05-17T22:03:00.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.154+0000 7fef9ffff640 1 -- 172.21.15.83:0/1820743951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa80aa9c0 con 0x7fefac0d2000 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- 172.21.15.83:0/1820743951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d2000 msgr2=0x7fefac0d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- 172.21.15.83:0/1820743951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d2000 0x7fefac0d2410 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fefac08e620 tx=0x7fefa801c480 comp rx=0 tx=0).stop 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- 172.21.15.83:0/1820743951 shutdown_connections 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- 172.21.15.83:0/1820743951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d2000 0x7fefac0d2410 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- 172.21.15.83:0/1820743951 >> v2:172.21.15.155:3300/0 conn(0x7fefac0b4810 0x7fefac0d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- 172.21.15.83:0/1820743951 >> 172.21.15.83:0/1820743951 conn(0x7fefac06e010 msgr2=0x7fefac06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- 172.21.15.83:0/1820743951 shutdown_connections 2024-05-17T22:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- 172.21.15.83:0/1820743951 wait complete. 2024-05-17T22:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 Processor -- start 2024-05-17T22:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- start start 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefac0b4810 0x7fefac14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefac0d2000 0x7fefac151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0b9670 con 0x7fefac0d0290 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0b94f0 con 0x7fefac0d2000 2024-05-17T22:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.158+0000 7fefb32ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fefac0b97f0 con 0x7fefac0b4810 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60596/0 (socket says 172.21.15.83:60596) 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 -- 172.21.15.83:0/421724686 learned_addr learned my addr 172.21.15.83:0/421724686 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb1857640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefac0d2000 0x7fefac151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb1056640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefac0b4810 0x7fefac14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 -- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefac0b4810 msgr2=0x7fefac14e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefac0b4810 0x7fefac14e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 -- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefac0d2000 msgr2=0x7fefac151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefac0d2000 0x7fefac151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 -- 172.21.15.83:0/421724686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac15dd30 con 0x7fefac0d0290 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb0855640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c430 tx=0x7fefa000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0017070 con 0x7fefac0d0290 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0005d10 con 0x7fefac0d0290 2024-05-17T22:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac14f930 con 0x7fefac0d0290 2024-05-17T22:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0007310 con 0x7fefac0d0290 2024-05-17T22:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac14fef0 con 0x7fefac0d0290 2024-05-17T22:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac0d5210 con 0x7fefac0d0290 2024-05-17T22:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fefa001a070 con 0x7fefac0d0290 2024-05-17T22:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 0x7fef90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb1056640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 0x7fef90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fefa008e680 con 0x7fefac0d0290 2024-05-17T22:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.162+0000 7fefb1056640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 0x7fef90071910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fefa4003fd0 tx=0x7fefa40039a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.166+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fefa005b570 con 0x7fefac0d0290 2024-05-17T22:03:00.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.374+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fefac1589b0 con 0x7fefac0d0290 2024-05-17T22:03:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.378+0000 7fef9dffb640 1 -- 172.21.15.83:0/421724686 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fefa005ae70 con 0x7fefac0d0290 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T22:03:00.372 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 msgr2=0x7fef90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 0x7fef90071910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fefa4003fd0 tx=0x7fefa40039a0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 msgr2=0x7fefac1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fefa000c430 tx=0x7fefa000c8f0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 shutdown_connections 2024-05-17T22:03:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9006f400 0x7fef90071910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefac0d2000 0x7fefac151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefac0d0290 0x7fefac1518d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 --2- 172.21.15.83:0/421724686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefac0b4810 0x7fefac14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 >> 172.21.15.83:0/421724686 conn(0x7fefac06e010 msgr2=0x7fefac154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 shutdown_connections 2024-05-17T22:03:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.382+0000 7fefb32ba640 1 -- 172.21.15.83:0/421724686 wait complete. 2024-05-17T22:03:00.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 Processor -- start 2024-05-17T22:03:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 -- start start 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3f48144080 0x7f3f48072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f48073370 0x7f3f4806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f48073780 con 0x7f3f480d1040 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4815e790 con 0x7f3f48073370 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f506fa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4815ea40 con 0x7f3f48144080 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f48073370 0x7f3f4806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f48073370 0x7f3f4806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60604/0 (socket says 172.21.15.83:60604) 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 -- 172.21.15.83:0/480738010 learned_addr learned my addr 172.21.15.83:0/480738010 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4dc95640 1 --2- 172.21.15.83:0/480738010 >> v2:172.21.15.155:3300/0 conn(0x7f3f48144080 0x7f3f48072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 -- 172.21.15.83:0/480738010 >> v2:172.21.15.155:3300/0 conn(0x7f3f48144080 msgr2=0x7f3f48072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 --2- 172.21.15.83:0/480738010 >> v2:172.21.15.155:3300/0 conn(0x7f3f48144080 0x7f3f48072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 -- 172.21.15.83:0/480738010 >> v1:172.21.15.83:6789/0 conn(0x7f3f480d1040 legacy=0x7f3f4815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.534+0000 7f3f4ec97640 1 -- 172.21.15.83:0/480738010 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4815ed20 con 0x7f3f48073370 2024-05-17T22:03:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f4ec97640 1 --2- 172.21.15.83:0/480738010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48073370 0x7f3f4806f730 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3f480b6b10 tx=0x7f3f4001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d15c9ee6ebe308 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f4d494640 1 -- 172.21.15.83:0/480738010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400a5880 con 0x7f3f48073370 2024-05-17T22:03:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f4d494640 1 -- 172.21.15.83:0/480738010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f400a5a20 con 0x7f3f48073370 2024-05-17T22:03:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f4d494640 1 -- 172.21.15.83:0/480738010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400a5df0 con 0x7f3f48073370 2024-05-17T22:03:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 -- 172.21.15.83:0/480738010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48073370 msgr2=0x7f3f4806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 --2- 172.21.15.83:0/480738010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48073370 0x7f3f4806f730 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3f480b6b10 tx=0x7f3f4001c9c0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 -- 172.21.15.83:0/480738010 shutdown_connections 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 --2- 172.21.15.83:0/480738010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48073370 0x7f3f4806f730 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 --2- 172.21.15.83:0/480738010 >> v2:172.21.15.155:3300/0 conn(0x7f3f48144080 0x7f3f48072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 -- 172.21.15.83:0/480738010 >> 172.21.15.83:0/480738010 conn(0x7f3f4806cd90 msgr2=0x7f3f480b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 -- 172.21.15.83:0/480738010 shutdown_connections 2024-05-17T22:03:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.538+0000 7f3f506fa640 1 -- 172.21.15.83:0/480738010 wait complete. 2024-05-17T22:03:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 Processor -- start 2024-05-17T22:03:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- start start 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f48073370 0x7f3f4815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f480d1040 0x7f3f480aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48070480 con 0x7f3f48144080 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3f48144490 con 0x7f3f480d1040 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3f48070600 con 0x7f3f48073370 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60612/0 (socket says 172.21.15.83:60612) 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 -- 172.21.15.83:0/819041430 learned_addr learned my addr 172.21.15.83:0/819041430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4e496640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f48073370 0x7f3f4815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4dc95640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f480d1040 0x7f3f480aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 -- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f48073370 msgr2=0x7f3f4815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f48073370 0x7f3f4815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 -- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f480d1040 msgr2=0x7f3f480aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f480d1040 0x7f3f480aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 -- 172.21.15.83:0/819041430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f481fd0c0 con 0x7f3f48144080 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f4ec97640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3f4001a5f0 tx=0x7f3f40019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400b8670 con 0x7f3f48144080 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f400be070 con 0x7f3f48144080 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400b8a90 con 0x7f3f48144080 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4815da00 con 0x7f3f48144080 2024-05-17T22:03:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f480af430 con 0x7f3f48144080 2024-05-17T22:03:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.542+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f481fce30 con 0x7f3f48144080 2024-05-17T22:03:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.546+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f400b8c30 con 0x7f3f48144080 2024-05-17T22:03:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.546+0000 7f3f3effd640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 0x7f3f2c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.546+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f3f4012b010 con 0x7f3f48144080 2024-05-17T22:03:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.546+0000 7f3f4e496640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 0x7f3f2c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.546+0000 7f3f4e496640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 0x7f3f2c071940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3f44001570 tx=0x7f3f44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:00.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.550+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3f400165e0 con 0x7f3f48144080 2024-05-17T22:03:00.724 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:03:00.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:03:00.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.738+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f48070990 con 0x7f3f48144080 2024-05-17T22:03:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.738+0000 7f3f3effd640 1 -- 172.21.15.83:0/819041430 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5189 (secure 0 0 0) 0x7f3f400f7fe0 con 0x7f3f48144080 2024-05-17T22:03:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:00.733 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:02:57.824521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6618":{"gid":6618,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3213750564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3213750564},{"type":"v1","addr":"172.21.15.83:6836","nonce":3213750564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6618,"qdb_cluster":[6618]},"id":6}]} 2024-05-17T22:03:00.733 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 59 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 msgr2=0x7f3f2c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 0x7f3f2c071940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3f44001570 tx=0x7f3f44009040 comp rx=0 tx=0).stop 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 msgr2=0x7f3f480aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3f4001a5f0 tx=0x7f3f40019760 comp rx=0 tx=0).stop 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 shutdown_connections 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f48144080 0x7f3f480aff10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3f2c06f430 0x7f3f2c071940 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f480d1040 0x7f3f480aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 --2- 172.21.15.83:0/819041430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3f48073370 0x7f3f4815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 >> 172.21.15.83:0/819041430 conn(0x7f3f4806cd90 msgr2=0x7f3f48146620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 shutdown_connections 2024-05-17T22:03:00.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:00.742+0000 7f3f506fa640 1 -- 172.21.15.83:0/819041430 wait complete. 2024-05-17T22:03:01.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:02:57.824521+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6618}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6618': {'gid': 6618, 'name': 'a', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/3213750564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3213750564}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 3213750564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6618, 'qdb_cluster': [6618]} 2024-05-17T22:03:01.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:03:01.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:03:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 Processor -- start 2024-05-17T22:03:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 -- start start 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f930c073590 0x7f930c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f930c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930c0d10d0 con 0x7f930c0b3d00 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f930c0b4110 con 0x7f930c073590 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.326+0000 7f93120a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f930c070770 con 0x7f930c143f20 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930b7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f930c0b3d00 0x7f930c072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57120/0 (socket says 172.21.15.83:57120) 2024-05-17T22:03:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930b7fe640 1 -- 172.21.15.83:0/4282688993 learned_addr learned my addr 172.21.15.83:0/4282688993 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f930c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692907774 0 0) 0x7f930c0d10d0 con 0x7f930c0b3d00 2024-05-17T22:03:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f9302ffd640 1 --2- 172.21.15.83:0/4282688993 >> v2:172.21.15.155:3300/0 conn(0x7f930c073590 0x7f930c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92f8003530 con 0x7f930c0b3d00 2024-05-17T22:03:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 120305303 0 0) 0x7f92f8003530 con 0x7f930c0b3d00 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 msgr2=0x7f930c1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f930c1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 >> v2:172.21.15.155:3300/0 conn(0x7f930c073590 msgr2=0x7f930c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 --2- 172.21.15.83:0/4282688993 >> v2:172.21.15.155:3300/0 conn(0x7f930c073590 0x7f930c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c070120 con 0x7f930c0b3d00 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f92fc001700 con 0x7f930c0b3d00 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 >> v1:172.21.15.83:6789/0 conn(0x7f930c0b3d00 legacy=0x7f930c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f8003a10 0x7f92f8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f92f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f8004d50 0x7f92f8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f930c0b4110 con 0x7f92f8004d50 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f930c070770 con 0x7f92f8003a10 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f92f800ba80 con 0x7f930c143f20 2024-05-17T22:03:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f8004d50 0x7f92f8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930b7fe640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f8003a10 0x7f92f8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f9302ffd640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f92f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 -- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 msgr2=0x7f92f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c143f20 0x7f92f8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 -- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f8003a10 msgr2=0x7f92f8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f8003a10 0x7f92f8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 -- 172.21.15.83:0/4282688993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8003530 con 0x7f92f8004d50 2024-05-17T22:03:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930b7fe640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92f8003a10 0x7f92f8003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930bfff640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f8004d50 0x7f92f8005160 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9304009f90 tx=0x7f930401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9877b6b755581aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040b2070 con 0x7f92f8004d50 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93040a5c70 con 0x7f92f8004d50 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.330+0000 7f930affd640 1 -- 172.21.15.83:0/4282688993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304009c10 con 0x7f92f8004d50 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- 172.21.15.83:0/4282688993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f8004d50 msgr2=0x7f92f8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 --2- 172.21.15.83:0/4282688993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f8004d50 0x7f92f8005160 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9304009f90 tx=0x7f930401cc60 comp rx=0 tx=0).stop 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 -- 172.21.15.83:0/4282688993 reap_dead start 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- 172.21.15.83:0/4282688993 shutdown_connections 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- 172.21.15.83:0/4282688993 >> 172.21.15.83:0/4282688993 conn(0x7f930c06cd90 msgr2=0x7f930c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- 172.21.15.83:0/4282688993 shutdown_connections 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- 172.21.15.83:0/4282688993 wait complete. 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 Processor -- start 2024-05-17T22:03:01.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- start start 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c142370 0x7f930c13bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ae4f0 0x7f930c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f930c147600 con 0x7f930c0aee40 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f930c06fed0 con 0x7f930c0ae4f0 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f93120a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f930c144330 con 0x7f930c142370 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60624/0 (socket says 172.21.15.83:60624) 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 -- 172.21.15.83:0/847109539 learned_addr learned my addr 172.21.15.83:0/847109539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f9302ffd640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ae4f0 0x7f930c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930b7fe640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c142370 0x7f930c13bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 -- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c142370 msgr2=0x7f930c13bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.334+0000 7f930bfff640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c142370 0x7f930c13bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930bfff640 1 -- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ae4f0 msgr2=0x7f930c0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930bfff640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ae4f0 0x7f930c0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930bfff640 1 -- 172.21.15.83:0/847109539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c0af680 con 0x7f930c0aee40 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930bfff640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f93040b0040 tx=0x7f9304009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 -- 172.21.15.83:0/847109539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040b2040 con 0x7f930c0aee40 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 -- 172.21.15.83:0/847109539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93040bc070 con 0x7f930c0aee40 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c0c6ae0 con 0x7f930c0aee40 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 -- 172.21.15.83:0/847109539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304009c10 con 0x7f930c0aee40 2024-05-17T22:03:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c0c6770 con 0x7f930c0aee40 2024-05-17T22:03:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 -- 172.21.15.83:0/847109539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93040aeb40 con 0x7f930c0aee40 2024-05-17T22:03:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f9308ff9640 1 -- 172.21.15.83:0/847109539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9304129ba0 con 0x7f930c0aee40 2024-05-17T22:03:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930b7fe640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f930b7fe640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 0x7f92ec071910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000f70 tx=0x7f92fc000f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f931931e000 1 Processor -- start 2024-05-17T22:03:01.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.338+0000 7f931931e000 1 -- start start 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0339c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d0339d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d0323aca0 con 0x561d0324c600 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561d031ad110 con 0x561d03235d30 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561d0319ccb0 con 0x561d03256fa0 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60640/0 (socket says 172.21.15.83:60640) 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 -- 172.21.15.83:0/3581508689 learned_addr learned my addr 172.21.15.83:0/3581508689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 -- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 msgr2=0x561d0339c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0339c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 -- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 msgr2=0x561d0339d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d0339d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 -- 172.21.15.83:0/3581508689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d03161370 con 0x561d0324c600 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009980 tx=0x7f92f4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ce82c662a02ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f93017fa640 1 -- 172.21.15.83:0/3581508689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4019070 con 0x561d0324c600 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f93017fa640 1 -- 172.21.15.83:0/3581508689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f4011ac0 con 0x561d0324c600 2024-05-17T22:03:01.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f93017fa640 1 -- 172.21.15.83:0/3581508689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4015460 con 0x561d0324c600 2024-05-17T22:03:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- 172.21.15.83:0/3581508689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 msgr2=0x561d0339cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009980 tx=0x7f92f4009190 comp rx=0 tx=0).stop 2024-05-17T22:03:01.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- 172.21.15.83:0/3581508689 shutdown_connections 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d0339d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d0339cdf0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 --2- 172.21.15.83:0/3581508689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0339c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- 172.21.15.83:0/3581508689 >> 172.21.15.83:0/3581508689 conn(0x561d02e5e460 msgr2=0x561d03351e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- 172.21.15.83:0/3581508689 shutdown_connections 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- 172.21.15.83:0/3581508689 wait complete. 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 -- start start 2024-05-17T22:03:01.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.342+0000 7f931931e000 1 Processor -- start 2024-05-17T22:03:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d033592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0335e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d03158d40 con 0x561d0324c600 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561d031a0cf0 con 0x561d03235d30 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561d03145e90 con 0x561d03256fa0 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 10 client.0 _ll_register_callbacks cb 0x561d0317afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60652/0 (socket says 172.21.15.83:60652) 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 -- 172.21.15.83:0/3879366730 learned_addr learned my addr 172.21.15.83:0/3879366730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d033592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 -- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 msgr2=0x561d0335e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f930bfff640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0335e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0335e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 -- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 msgr2=0x561d033592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d033592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 -- 172.21.15.83:0/3879366730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0316b6e0 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f92f4000c10 tx=0x7f92f40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e424bf0dc066de8e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f402d060 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f92f4003620 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f4011ac0 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f92f4033050 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 10 client.6750 fetch_fsmap learned FSMap version 59 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x561d0315b690 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f4011ac0 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f92f4036020 con 0x561d0324c600 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 10 client.6750 fetch_fsmap finished waiting for FSMap version 59 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 10 client.6750 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 20 client.6750 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e437d0 0x561d033a3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 4 client.6750 mds_command: new command op to 6618 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 --> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d03138a00 con 0x561d02e437d0 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e437d0 0x561d033a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 10 client.6750 ms_handle_connect on v2:172.21.15.83:6834/3213750564 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e437d0 0x561d033a3920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b31627961d636f server_cookie=7ae9644775c9597d in_seq=0 out_seq=0 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3879366730 <== mds.0 v2:172.21.15.83:6834/3213750564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d03138a00 con 0x561d02e437d0 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f92ea7fc640 10 client.6750 handle_command_reply: tid=0 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.346+0000 7f931931e000 1 client.6750 shutdown 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 msgr2=0x561d03359810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f92f4000c10 tx=0x7f92f40046d0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 shutdown_connections 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d03256fa0 0x561d0335e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d0324c600 0x561d03359810 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e437d0 0x561d033a3920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 --2- 172.21.15.83:0/3879366730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03235d30 0x561d033592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f92ea7fc640 0 client.6750 ms_handle_reset on v2:172.21.15.83:6834/3213750564 2024-05-17T22:03:01.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 >> 172.21.15.83:0/3879366730 conn(0x561d02e5e460 msgr2=0x561d03351e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 shutdown_connections 2024-05-17T22:03:01.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 1 -- 172.21.15.83:0/3879366730 wait complete. 2024-05-17T22:03:01.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.350+0000 7f931931e000 20 client.6750 trim_cache size 0 max 16384 2024-05-17T22:03:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.366+0000 7f931931e000 1 Processor -- start 2024-05-17T22:03:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.366+0000 7f931931e000 1 -- start start 2024-05-17T22:03:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d0340f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d0340f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1ffef0 con 0x561d03235d30 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1ff250 con 0x561d03256fa0 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f930c1ff8a0 con 0x561d0324c600 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60664/0 (socket says 172.21.15.83:60664) 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 -- 172.21.15.83:0/2700619011 learned_addr learned my addr 172.21.15.83:0/2700619011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 -- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 msgr2=0x561d0340f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d0340f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 -- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 msgr2=0x561d0340f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d0340f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 -- 172.21.15.83:0/2700619011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d03223cb0 con 0x561d03235d30 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f92fc005bb0 tx=0x7f92fc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d328295a16c86b19 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f9300ff9640 1 -- 172.21.15.83:0/2700619011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc028070 con 0x561d03235d30 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f9300ff9640 1 -- 172.21.15.83:0/2700619011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc007390 con 0x561d03235d30 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f9300ff9640 1 -- 172.21.15.83:0/2700619011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc007740 con 0x561d03235d30 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- 172.21.15.83:0/2700619011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 msgr2=0x561d0340ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f92fc005bb0 tx=0x7f92fc011000 comp rx=0 tx=0).stop 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- 172.21.15.83:0/2700619011 shutdown_connections 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d0340f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d0340f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- 172.21.15.83:0/2700619011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d0340ec30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- 172.21.15.83:0/2700619011 >> 172.21.15.83:0/2700619011 conn(0x561d02e5e460 msgr2=0x561d033b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- 172.21.15.83:0/2700619011 shutdown_connections 2024-05-17T22:03:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- 172.21.15.83:0/2700619011 wait complete. 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- start start 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 Processor -- start 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d034134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d034139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d031a0000 con 0x561d03235d30 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561d0335cc20 con 0x561d03256fa0 2024-05-17T22:03:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561d031abee0 con 0x561d0324c600 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f931931e000 10 client.0 _ll_register_callbacks cb 0x7f92f40097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.370+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60668/0 (socket says 172.21.15.83:60668) 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 -- 172.21.15.83:0/3529281426 learned_addr learned my addr 172.21.15.83:0/3529281426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d034134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930bfff640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d034139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 -- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 msgr2=0x561d034134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d034134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 -- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 msgr2=0x561d034139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d034139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 -- 172.21.15.83:0/3529281426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0315c4f0 con 0x561d03235d30 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f930b7fe640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f92fc016040 tx=0x7f92fc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3511500fb2fa0e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc028040 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f92fc013680 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc007390 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f92fc007750 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 10 client.6756 fetch_fsmap learned FSMap version 59 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x561d031664f0 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92fc007390 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f92fc013880 con 0x561d03235d30 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 10 client.6756 fetch_fsmap finished waiting for FSMap version 59 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 10 client.6756 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 20 client.6756 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e31830 0x561d033ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 4 client.6756 mds_command: new command op to 6618 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:03:01.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 --> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] -- command(tid 0: {"prefix": "status"}) -- 0x561d031af060 con 0x561d02e31830 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e31830 0x561d033ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 10 client.6756 ms_handle_connect on v2:172.21.15.83:6834/3213750564 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f9302ffd640 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e31830 0x561d033ced90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67eab0292227e9b server_cookie=8cfd25954b6a783b in_seq=0 out_seq=0 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 1 -- 172.21.15.83:0/3529281426 <== mds.0 v2:172.21.15.83:6834/3213750564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x561d031af060 con 0x561d02e31830 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 10 client.6756 handle_command_reply: tid=0 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 client.6756 shutdown 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 msgr2=0x561d03412f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f92fc016040 tx=0x7f92fc009040 comp rx=0 tx=0).stop 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 shutdown_connections 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:6834/3213750564,v1:172.21.15.83:6836/3213750564] conn(0x561d02e31830 0x561d033ced90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d03256fa0 0x561d034139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f92ea7fc640 0 client.6756 ms_handle_reset on v2:172.21.15.83:6834/3213750564 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561d0324c600 0x561d034134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 --2- 172.21.15.83:0/3529281426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d03235d30 0x561d03412f70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 >> 172.21.15.83:0/3529281426 conn(0x561d02e5e460 msgr2=0x561d0339c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 shutdown_connections 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 1 -- 172.21.15.83:0/3529281426 wait complete. 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f931931e000 20 client.6756 trim_cache size 0 max 16384 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6618, 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:03:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.926507494, 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 59, 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 81, 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 81, 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.937851487 2024-05-17T22:03:01.374 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 msgr2=0x7f92ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 0x7f92ec071910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92fc000f70 tx=0x7f92fc000f00 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 msgr2=0x7f930c1f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f93040b0040 tx=0x7f9304009750 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 shutdown_connections 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c0aee40 0x7f930c1f6d50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f930c0ae4f0 0x7f930c0ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 --2- 172.21.15.83:0/847109539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f930c142370 0x7f930c13bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 >> 172.21.15.83:0/847109539 conn(0x7f930c06cd90 msgr2=0x7f930c1428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 shutdown_connections 2024-05-17T22:03:01.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.374+0000 7f93120a4640 1 -- 172.21.15.83:0/847109539 wait complete. 2024-05-17T22:03:01.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:03:01.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 Processor -- start 2024-05-17T22:03:01.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 -- start start 2024-05-17T22:03:01.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 0x7f5d0c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0c0700e0 con 0x7f5d0c06d7a0 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5d0c0709f0 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d11ca3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c0aff10 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d0bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d0b7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 0x7f5d0c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d0affd640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5d0c06d7a0 0x7f5d0c06dbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:49072/0 (socket says 172.21.15.83:49072) 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d0bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:48554/0 (socket says 172.21.15.83:48554) 2024-05-17T22:03:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.526+0000 7f5d0affd640 1 -- 172.21.15.83:0/1434418222 learned_addr learned my addr 172.21.15.83:0/1434418222 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0bfff640 1 -- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 msgr2=0x7f5d0c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0bfff640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 0x7f5d0c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0bfff640 1 -- 172.21.15.83:0/1434418222 >> v1:172.21.15.155:6789/0 conn(0x7f5d0c06d7a0 legacy=0x7f5d0c06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0bfff640 1 -- 172.21.15.83:0/1434418222 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c0703a0 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0b7fe640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 0x7f5d0c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:03:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0bfff640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d040097c0 tx=0x7f5d0401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7882b9b8bad51e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0a7fc640 1 -- 172.21.15.83:0/1434418222 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040ab650 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0a7fc640 1 -- 172.21.15.83:0/1434418222 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d040b0070 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d0a7fc640 1 -- 172.21.15.83:0/1434418222 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040aba10 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 msgr2=0x7f5d0c15dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d040097c0 tx=0x7f5d0401c470 comp rx=0 tx=0).stop 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1434418222 shutdown_connections 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15dce0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1434418222 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c0af7c0 0x7f5d0c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1434418222 >> 172.21.15.83:0/1434418222 conn(0x7f5d0c073670 msgr2=0x7f5d0c073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1434418222 shutdown_connections 2024-05-17T22:03:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.530+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1434418222 wait complete. 2024-05-17T22:03:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 Processor -- start 2024-05-17T22:03:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 -- start start 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c06d7a0 0x7f5d0c1415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c071540 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c0713c0 con 0x7f5d0c15d8d0 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d11ca3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0c068b30 con 0x7f5d0c06d7a0 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d0affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d0affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60678/0 (socket says 172.21.15.83:60678) 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.534+0000 7f5d0affd640 1 -- 172.21.15.83:0/1089846758 learned_addr learned my addr 172.21.15.83:0/1089846758 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0b7fe640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c06d7a0 0x7f5d0c1415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0bfff640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 -- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c06d7a0 msgr2=0x7f5d0c1415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c06d7a0 0x7f5d0c1415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 -- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 msgr2=0x7f5d0c15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 -- 172.21.15.83:0/1089846758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c150d00 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0affd640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00c430 tx=0x7f5cfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc017070 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cfc005d10 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c141ae0 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc007310 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c151450 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c070e80 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5cfc01a070 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 0x7f5cec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0b7fe640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 0x7f5cec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f5cfc08e680 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.538+0000 7f5d0b7fe640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 0x7f5cec071910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007440 tx=0x7f5cf400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.542+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5cfc05b570 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.726+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d0c0b29b0 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.726+0000 7f5d03fff640 1 -- 172.21.15.83:0/1089846758 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7f5cfc05ae70 con 0x7f5d0c0af7c0 2024-05-17T22:03:01.720 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 msgr2=0x7f5cec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 0x7f5cec071910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4007440 tx=0x7f5cf400b040 comp rx=0 tx=0).stop 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 msgr2=0x7f5d0c1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00c430 tx=0x7f5cfc00c8f0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 shutdown_connections 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5cec06f400 0x7f5cec071910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d0c15d8d0 0x7f5d0c15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c0af7c0 0x7f5d0c1526d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 --2- 172.21.15.83:0/1089846758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d0c06d7a0 0x7f5d0c1415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 >> 172.21.15.83:0/1089846758 conn(0x7f5d0c073670 msgr2=0x7f5d0c09c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 shutdown_connections 2024-05-17T22:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01.730+0000 7f5d11ca3640 1 -- 172.21.15.83:0/1089846758 wait complete. 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:03:01.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:01.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:01.741 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.747 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:03:01.748 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:03:01.749 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:01.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:01.750 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:03:01.797 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:03:01.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:01.799 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:03:01.849 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:03:01.849 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:03:01.849 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:01.850 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:03:01.850 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:03:01.850 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:03:01.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:03:01.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:03:01.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:03:01.851 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:03:01.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:03:01.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:03:01.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:01.971 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:03:02.018 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:03:02.018 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:03:02.018 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:02.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:02.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:03:02.074 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:03:02.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:03:02.074 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:02.122 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:02.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:03:02.122 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:02.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:02.227 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:02.275 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:02.276 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:02.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:02.277 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:02.321 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:03:02.321 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:03:02.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:02.380 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:02.430 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:02.432 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:02.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:02.433 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:02.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:03:02.482+0000 7f267e791580 -1 init, newargv = 0x7f266c0054f0 newargc=15 2024-05-17T22:03:02.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43488]: starting ceph client 2024-05-17T22:03:02.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43488]: starting fuse 2024-05-17T22:03:03.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:03.595 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:03.642 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:03.643 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:03.644 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:03.645 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:03.646 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:03.646 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:03.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:03.649 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:03.694 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:03:03.694 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:03:03.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:03:03.694 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:03:03.694 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:03:03.694 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:03:03.695 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:03:03.696 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:03:03.819 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.43488.asok 2024-05-17T22:03:03.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:03:03.831 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43488.asok 2024-05-17T22:03:03.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:03.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43488.asok status 2024-05-17T22:03:04.013 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:03:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "43488", 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6768, 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:03:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6768 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1317063753 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1317063753 2024-05-17T22:03:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6768 192.168.0.1:0/1317063753", 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1317063753", 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 59, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 81, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:03:04.016 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:04.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.029 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:03:04.030 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:04.036 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:03:04.036 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:04.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:04.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.091 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-17T22:03:04.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.151 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-17T22:03:04.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.206 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-17T22:03:04.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:04.257 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-17T22:03:04.317 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:03:04.428 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:03:04.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:03:04.439 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:03:04.439 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:03:04.486 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:03:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 Processor -- start 2024-05-17T22:03:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 -- start start 2024-05-17T22:03:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efd780b3be0 0x7efd780af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd780b4bb0 0x7efd780cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780ce450 con 0x7efd780af8b0 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efd780ce730 con 0x7efd780b4bb0 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd7db31640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efd780cea10 con 0x7efd780b3be0 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd780b4bb0 0x7efd780cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd780b4bb0 0x7efd780cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60694/0 (socket says 172.21.15.83:60694) 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 -- 172.21.15.83:0/3117708101 learned_addr learned my addr 172.21.15.83:0/3117708101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd777fe640 1 --2- 172.21.15.83:0/3117708101 >> v2:172.21.15.155:3300/0 conn(0x7efd780b3be0 0x7efd780af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 -- 172.21.15.83:0/3117708101 >> v2:172.21.15.155:3300/0 conn(0x7efd780b3be0 msgr2=0x7efd780af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 --2- 172.21.15.83:0/3117708101 >> v2:172.21.15.155:3300/0 conn(0x7efd780b3be0 0x7efd780af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 -- 172.21.15.83:0/3117708101 >> v1:172.21.15.155:6789/0 conn(0x7efd780af8b0 legacy=0x7efd7806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.698+0000 7efd76ffd640 1 -- 172.21.15.83:0/3117708101 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780cecf0 con 0x7efd780b4bb0 2024-05-17T22:03:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd76ffd640 1 --2- 172.21.15.83:0/3117708101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b4bb0 0x7efd780cf4f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efd780d0040 tx=0x7efd6801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe5d56c74a6eb4b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd767fc640 1 -- 172.21.15.83:0/3117708101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680b0070 con 0x7efd780b4bb0 2024-05-17T22:03:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd767fc640 1 -- 172.21.15.83:0/3117708101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd680ab600 con 0x7efd780b4bb0 2024-05-17T22:03:04.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd767fc640 1 -- 172.21.15.83:0/3117708101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680ab9d0 con 0x7efd780b4bb0 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 -- 172.21.15.83:0/3117708101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b4bb0 msgr2=0x7efd780cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 --2- 172.21.15.83:0/3117708101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b4bb0 0x7efd780cf4f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efd780d0040 tx=0x7efd6801c430 comp rx=0 tx=0).stop 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 -- 172.21.15.83:0/3117708101 shutdown_connections 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 --2- 172.21.15.83:0/3117708101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b4bb0 0x7efd780cf4f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 --2- 172.21.15.83:0/3117708101 >> v2:172.21.15.155:3300/0 conn(0x7efd780b3be0 0x7efd780af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 -- 172.21.15.83:0/3117708101 >> 172.21.15.83:0/3117708101 conn(0x7efd780b4090 msgr2=0x7efd780b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 -- 172.21.15.83:0/3117708101 shutdown_connections 2024-05-17T22:03:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.702+0000 7efd7db31640 1 -- 172.21.15.83:0/3117708101 wait complete. 2024-05-17T22:03:04.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 Processor -- start 2024-05-17T22:03:04.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- start start 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 0x7efd781f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 0x7efd78162c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd780b7d90 con 0x7efd780b3be0 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd7806d0b0 con 0x7efd780b4bb0 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efd780b7f10 con 0x7efd780af8b0 2024-05-17T22:03:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60702/0 (socket says 172.21.15.83:60702) 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 -- 172.21.15.83:0/2408032723 learned_addr learned my addr 172.21.15.83:0/2408032723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd777fe640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 0x7efd781f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd77fff640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 0x7efd78162c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 -- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 msgr2=0x7efd781f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 0x7efd781f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 -- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 msgr2=0x7efd78162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 0x7efd78162c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 -- 172.21.15.83:0/2408032723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780d1200 con 0x7efd780b3be0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd77fff640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 0x7efd78162c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd777fe640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 0x7efd781f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd76ffd640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7efd6801ccd0 tx=0x7efd68019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680b0040 con 0x7efd780b3be0 2024-05-17T22:03:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd680b4640 con 0x7efd780b3be0 2024-05-17T22:03:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680b4a30 con 0x7efd780b3be0 2024-05-17T22:03:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780d14e0 con 0x7efd780b3be0 2024-05-17T22:03:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.706+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd780cb070 con 0x7efd780b3be0 2024-05-17T22:03:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd780d0f70 con 0x7efd780b3be0 2024-05-17T22:03:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efd680ab4c0 con 0x7efd780b3be0 2024-05-17T22:03:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd5ffff640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 0x7efd64071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7efd681299b0 con 0x7efd780b3be0 2024-05-17T22:03:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd777fe640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 0x7efd64071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.710+0000 7efd777fe640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 0x7efd64071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0097a0 tx=0x7efd6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:04.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.714+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efd68016110 con 0x7efd780b3be0 2024-05-17T22:03:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.886+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd780af370 con 0x7efd780b3be0 2024-05-17T22:03:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.886+0000 7efd5ffff640 1 -- 172.21.15.83:0/2408032723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7efd680f6910 con 0x7efd780b3be0 2024-05-17T22:03:04.881 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:03:04.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 msgr2=0x7efd64071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:04.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 0x7efd64071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0097a0 tx=0x7efd6c008040 comp rx=0 tx=0).stop 2024-05-17T22:03:04.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 msgr2=0x7efd781626f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:04.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7efd6801ccd0 tx=0x7efd68019230 comp rx=0 tx=0).stop 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 shutdown_connections 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efd6406f430 0x7efd64071940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd780b4bb0 0x7efd78162c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd780b3be0 0x7efd781626f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 --2- 172.21.15.83:0/2408032723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd780af8b0 0x7efd781f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 >> 172.21.15.83:0/2408032723 conn(0x7efd780b4090 msgr2=0x7efd780b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 shutdown_connections 2024-05-17T22:03:04.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:04.890+0000 7efd7db31640 1 -- 172.21.15.83:0/2408032723 wait complete. 2024-05-17T22:03:04.902 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:03:04.902 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:03:04.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:03:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 Processor -- start 2024-05-17T22:03:05.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 -- start start 2024-05-17T22:03:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f67800cdf40 0x7f678015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f678015d530 0x7f67801663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67800b5640 con 0x7f67800cf830 2024-05-17T22:03:05.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6780070870 con 0x7f67800cdf40 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6787302640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f67800b58e0 con 0x7f678015d530 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678509e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f67800cf830 0x7f67800b3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57132/0 (socket says 172.21.15.83:57132) 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678509e640 1 -- 172.21.15.83:0/1820991240 learned_addr learned my addr 172.21.15.83:0/1820991240 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678489d640 1 --2- 172.21.15.83:0/1820991240 >> v2:172.21.15.83:3300/0 conn(0x7f67800cdf40 0x7f678015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678589f640 1 --2- 172.21.15.83:0/1820991240 >> v2:172.21.15.155:3300/0 conn(0x7f678015d530 0x7f67801663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6777fff640 1 -- 172.21.15.83:0/1820991240 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643819104 0 0) 0x7f67800b5640 con 0x7f67800cf830 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f6777fff640 1 -- 172.21.15.83:0/1820991240 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676c003530 con 0x7f67800cf830 2024-05-17T22:03:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678489d640 1 -- 172.21.15.83:0/1820991240 >> v2:172.21.15.155:3300/0 conn(0x7f678015d530 msgr2=0x7f67801663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:05.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678489d640 1 --2- 172.21.15.83:0/1820991240 >> v2:172.21.15.155:3300/0 conn(0x7f678015d530 0x7f67801663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678489d640 1 -- 172.21.15.83:0/1820991240 >> v1:172.21.15.83:6789/0 conn(0x7f67800cf830 legacy=0x7f67800b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.110+0000 7f678489d640 1 -- 172.21.15.83:0/1820991240 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678006f060 con 0x7f67800cdf40 2024-05-17T22:03:05.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f678489d640 1 --2- 172.21.15.83:0/1820991240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f678015cff0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6780074020 tx=0x7f677001e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935b3ecd4985b299 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:05.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6777fff640 1 -- 172.21.15.83:0/1820991240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700ac610 con 0x7f67800cdf40 2024-05-17T22:03:05.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6777fff640 1 -- 172.21.15.83:0/1820991240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67700b1070 con 0x7f67800cdf40 2024-05-17T22:03:05.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6777fff640 1 -- 172.21.15.83:0/1820991240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700ac9d0 con 0x7f67800cdf40 2024-05-17T22:03:05.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 -- 172.21.15.83:0/1820991240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 msgr2=0x7f678015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 --2- 172.21.15.83:0/1820991240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f678015cff0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6780074020 tx=0x7f677001e690 comp rx=0 tx=0).stop 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 -- 172.21.15.83:0/1820991240 shutdown_connections 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 --2- 172.21.15.83:0/1820991240 >> v2:172.21.15.155:3300/0 conn(0x7f678015d530 0x7f67801663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 --2- 172.21.15.83:0/1820991240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f678015cff0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 -- 172.21.15.83:0/1820991240 >> 172.21.15.83:0/1820991240 conn(0x7f678006cd90 msgr2=0x7f67800b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 -- 172.21.15.83:0/1820991240 shutdown_connections 2024-05-17T22:03:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.114+0000 7f6787302640 1 -- 172.21.15.83:0/1820991240 wait complete. 2024-05-17T22:03:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 Processor -- start 2024-05-17T22:03:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- start start 2024-05-17T22:03:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67800cf830 0x7f678015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f678015d530 0x7f6780158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6780073a20 con 0x7f67800cdf40 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67800738a0 con 0x7f67800cf830 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6780073ba0 con 0x7f678015d530 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60724/0 (socket says 172.21.15.83:60724) 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 -- 172.21.15.83:0/1474094815 learned_addr learned my addr 172.21.15.83:0/1474094815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 -- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f678015d530 msgr2=0x7f6780158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f678015d530 0x7f6780158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 -- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67800cf830 msgr2=0x7f678015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67800cf830 0x7f678015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 -- 172.21.15.83:0/1474094815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678015c7e0 con 0x7f67800cdf40 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f678509e640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c450 tx=0x7f677c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c017070 con 0x7f67800cdf40 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c005cf0 con 0x7f67800cdf40 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0072d0 con 0x7f67800cdf40 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67800aa4f0 con 0x7f67800cdf40 2024-05-17T22:03:05.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67800aaab0 con 0x7f67800cdf40 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.118+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678015cff0 con 0x7f67800cdf40 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.122+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f677c01a070 con 0x7f67800cdf40 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.122+0000 7f6775ffb640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 0x7f6764071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.122+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f677c08e970 con 0x7f67800cdf40 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.122+0000 7f678489d640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 0x7f6764071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:05.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.122+0000 7f678489d640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 0x7f6764071940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67700b0000 tx=0x7f6770004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.126+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f677c05b2f0 con 0x7f67800cdf40 2024-05-17T22:03:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.302+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67800766e0 con 0x7f67800cdf40 2024-05-17T22:03:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.302+0000 7f6775ffb640 1 -- 172.21.15.83:0/1474094815 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) ==== 80+0+0 (secure 0 0 0) 0x7f677c05f5f0 con 0x7f67800cdf40 2024-05-17T22:03:05.296 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 msgr2=0x7f6764071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 0x7f6764071940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67700b0000 tx=0x7f6770004740 comp rx=0 tx=0).stop 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 msgr2=0x7f6780152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c450 tx=0x7f677c00c910 comp rx=0 tx=0).stop 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 shutdown_connections 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f678015d530 0x7f6780158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f676406f430 0x7f6764071940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67800cf830 0x7f678015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 --2- 172.21.15.83:0/1474094815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67800cdf40 0x7f6780152de0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:05.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 >> 172.21.15.83:0/1474094815 conn(0x7f678006cd90 msgr2=0x7f67801597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:05.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 shutdown_connections 2024-05-17T22:03:05.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:05.306+0000 7f6787302640 1 -- 172.21.15.83:0/1474094815 wait complete. 2024-05-17T22:03:05.317 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:03:05.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:05.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:05.327 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43488]: fuse finished with error 0 and tester_r 0 2024-05-17T22:03:05.329 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:03:11.431 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:03:11.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:11.431 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:03:11.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:11.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 Processor -- start 2024-05-17T22:03:11.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 -- start start 2024-05-17T22:03:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a4009c0a0 0x7f0a40071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 0x7f0a400722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a400d1950 con 0x7f0a400cf2e0 2024-05-17T22:03:11.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0a400d1670 con 0x7f0a4009c0a0 2024-05-17T22:03:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a46560640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a40071280 con 0x7f0a4009c570 2024-05-17T22:03:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a44afd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 0x7f0a400722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a4009c0a0 0x7f0a40071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0a4009c0a0 0x7f0a40071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:60778/0 (socket says 172.21.15.83:60778) 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 -- 172.21.15.83:0/2253706535 learned_addr learned my addr 172.21.15.83:0/2253706535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 -- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 msgr2=0x7f0a400722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 0x7f0a400722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3effd640 1 -- 172.21.15.83:0/2253706535 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3543334491 0 0) 0x7f0a400d1950 con 0x7f0a400cf2e0 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 -- 172.21.15.83:0/2253706535 >> v1:172.21.15.155:6789/0 conn(0x7f0a400cf2e0 legacy=0x7f0a4009b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a3f7fe640 1 -- 172.21.15.83:0/2253706535 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40071560 con 0x7f0a4009c0a0 2024-05-17T22:03:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.654+0000 7f0a44afd640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 0x7f0a400722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:03:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a3f7fe640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a4009c0a0 0x7f0a40071d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009870 tx=0x7f0a3401ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d989b70e5a98abd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a3effd640 1 -- 172.21.15.83:0/2253706535 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340b5070 con 0x7f0a4009c0a0 2024-05-17T22:03:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a3effd640 1 -- 172.21.15.83:0/2253706535 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a340a8a50 con 0x7f0a4009c0a0 2024-05-17T22:03:11.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a3effd640 1 -- 172.21.15.83:0/2253706535 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340a8e20 con 0x7f0a4009c0a0 2024-05-17T22:03:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 -- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a4009c0a0 msgr2=0x7f0a40071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a4009c0a0 0x7f0a40071d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009870 tx=0x7f0a3401ed50 comp rx=0 tx=0).stop 2024-05-17T22:03:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 -- 172.21.15.83:0/2253706535 shutdown_connections 2024-05-17T22:03:11.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a4009c570 0x7f0a400722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 --2- 172.21.15.83:0/2253706535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a4009c0a0 0x7f0a40071d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 -- 172.21.15.83:0/2253706535 >> 172.21.15.83:0/2253706535 conn(0x7f0a400b5120 msgr2=0x7f0a400b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 -- 172.21.15.83:0/2253706535 shutdown_connections 2024-05-17T22:03:11.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.658+0000 7f0a46560640 1 -- 172.21.15.83:0/2253706535 wait complete. 2024-05-17T22:03:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 Processor -- start 2024-05-17T22:03:11.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- start start 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a400cf2e0 0x7f0a400cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 0x7f0a400c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400d1380 con 0x7f0a4009c570 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400d1200 con 0x7f0a400c8040 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0a400b8de0 con 0x7f0a400cf2e0 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35178/0 (socket says 172.21.15.83:35178) 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 -- 172.21.15.83:0/2004144308 learned_addr learned my addr 172.21.15.83:0/2004144308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a44afd640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 0x7f0a400c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3f7fe640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a400cf2e0 0x7f0a400cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 -- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a400cf2e0 msgr2=0x7f0a400cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a400cf2e0 0x7f0a400cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 -- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 msgr2=0x7f0a400c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 0x7f0a400c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 -- 172.21.15.83:0/2004144308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a400c8b00 con 0x7f0a4009c570 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a44afd640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 0x7f0a400c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3ffff640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0a30001ec0 tx=0x7f0a3000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30017070 con 0x7f0a4009c570 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a30003600 con 0x7f0a4009c570 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a400c8de0 con 0x7f0a4009c570 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a300039f0 con 0x7f0a4009c570 2024-05-17T22:03:11.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.662+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a400c93a0 con 0x7f0a4009c570 2024-05-17T22:03:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4009b990 con 0x7f0a4009c570 2024-05-17T22:03:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a3001a070 con 0x7f0a4009c570 2024-05-17T22:03:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a3cff9640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 0x7f0a2c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:11.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f0a3008e650 con 0x7f0a4009c570 2024-05-17T22:03:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a3f7fe640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 0x7f0a2c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:11.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.666+0000 7f0a3f7fe640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 0x7f0a2c071940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0a340b2000 tx=0x7f0a340aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:11.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.670+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0a3005b4c0 con 0x7f0a4009c570 2024-05-17T22:03:11.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a400b6ba0 con 0x7f0a4009c570 2024-05-17T22:03:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a3cff9640 1 -- 172.21.15.83:0/2004144308 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5189 (secure 0 0 0) 0x7f0a3005adc0 con 0x7f0a4009c570 2024-05-17T22:03:11.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:11.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:02:57.824521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6618},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6618":{"gid":6618,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/3213750564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3213750564},{"type":"v1","addr":"172.21.15.83:6836","nonce":3213750564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6618,"qdb_cluster":[6618]},"id":6}]} 2024-05-17T22:03:11.855 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 59 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 msgr2=0x7f0a2c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 0x7f0a2c071940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0a340b2000 tx=0x7f0a340aa070 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 msgr2=0x7f0a400ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0a30001ec0 tx=0x7f0a3000c890 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 shutdown_connections 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0a2c06f430 0x7f0a2c071940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400c8040 0x7f0a400c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0a400cf2e0 0x7f0a400cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 --2- 172.21.15.83:0/2004144308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a4009c570 0x7f0a400ca7d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 >> 172.21.15.83:0/2004144308 conn(0x7f0a400b5120 msgr2=0x7f0a400ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 shutdown_connections 2024-05-17T22:03:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:11.862+0000 7f0a46560640 1 -- 172.21.15.83:0/2004144308 wait complete. 2024-05-17T22:03:11.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T22:03:12.060 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:03:12.060 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:03:12.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:03:12.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.214+0000 7f9c265a2640 1 Processor -- start 2024-05-17T22:03:12.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.214+0000 7f9c265a2640 1 -- start start 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200b88b0 0x7f9c200d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9c2009ba00 0x7f9c2009be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c200ce130 0x7f9c200ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9c200d1a40 con 0x7f9c200ce130 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2009d040 con 0x7f9c2009ba00 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c265a2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2009c350 con 0x7f9c200b88b0 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c200ce130 0x7f9c200ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c200ce130 0x7f9c200ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35194/0 (socket says 172.21.15.83:35194) 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 -- 172.21.15.83:0/944046426 learned_addr learned my addr 172.21.15.83:0/944046426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c1ffff640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200b88b0 0x7f9c200d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 -- 172.21.15.83:0/944046426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200b88b0 msgr2=0x7f9c200d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200b88b0 0x7f9c200d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 -- 172.21.15.83:0/944046426 >> v2:172.21.15.155:3300/0 conn(0x7f9c2009ba00 msgr2=0x7f9c2009be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c1f7fe640 1 --2- 172.21.15.83:0/944046426 >> v2:172.21.15.155:3300/0 conn(0x7f9c2009ba00 0x7f9c2009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 --2- 172.21.15.83:0/944046426 >> v2:172.21.15.155:3300/0 conn(0x7f9c2009ba00 0x7f9c2009be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 -- 172.21.15.83:0/944046426 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2009c710 con 0x7f9c200ce130 2024-05-17T22:03:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.218+0000 7f9c24b3f640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200ce130 0x7f9c200ce540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009e50 tx=0x7f9c1801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb09736646cc50b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c1effd640 1 -- 172.21.15.83:0/944046426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c180a5660 con 0x7f9c200ce130 2024-05-17T22:03:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c1effd640 1 -- 172.21.15.83:0/944046426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c180a5800 con 0x7f9c200ce130 2024-05-17T22:03:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c1effd640 1 -- 172.21.15.83:0/944046426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c180a5bd0 con 0x7f9c200ce130 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- 172.21.15.83:0/944046426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200ce130 msgr2=0x7f9c200ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200ce130 0x7f9c200ce540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009e50 tx=0x7f9c1801c8d0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- 172.21.15.83:0/944046426 shutdown_connections 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200ce130 0x7f9c200ce540 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 --2- 172.21.15.83:0/944046426 >> v2:172.21.15.155:3300/0 conn(0x7f9c2009ba00 0x7f9c2009be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 --2- 172.21.15.83:0/944046426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200b88b0 0x7f9c200d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- 172.21.15.83:0/944046426 >> 172.21.15.83:0/944046426 conn(0x7f9c200d3620 msgr2=0x7f9c200d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- 172.21.15.83:0/944046426 shutdown_connections 2024-05-17T22:03:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- 172.21.15.83:0/944046426 wait complete. 2024-05-17T22:03:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 Processor -- start 2024-05-17T22:03:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.222+0000 7f9c265a2640 1 -- start start 2024-05-17T22:03:12.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c2009ba00 0x7f9c20153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200ce130 0x7f9c2015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200b2f40 con 0x7f9c200b88b0 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200b2dc0 con 0x7f9c2009ba00 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c200cee90 con 0x7f9c200ce130 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35196/0 (socket says 172.21.15.83:35196) 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 -- 172.21.15.83:0/2805311766 learned_addr learned my addr 172.21.15.83:0/2805311766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1ffff640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c2009ba00 0x7f9c20153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c24b3f640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200ce130 0x7f9c2015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 -- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200ce130 msgr2=0x7f9c2015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200ce130 0x7f9c2015e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 -- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c2009ba00 msgr2=0x7f9c20153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c2009ba00 0x7f9c20153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 -- 172.21.15.83:0/2805311766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2015c510 con 0x7f9c200b88b0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1f7fe640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c4f0 tx=0x7f9c1400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14015070 con 0x7f9c200b88b0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14011800 con 0x7f9c200b88b0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2015ea50 con 0x7f9c200b88b0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14011bf0 con 0x7f9c200b88b0 2024-05-17T22:03:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c20154a20 con 0x7f9c200b88b0 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c200d3400 con 0x7f9c200b88b0 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c14018070 con 0x7f9c200b88b0 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1cff9640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 0x7f9c0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.226+0000 7f9c1ffff640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 0x7f9c0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.230+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5767+0+0 (secure 0 0 0) 0x7f9c1408eff0 con 0x7f9c200b88b0 2024-05-17T22:03:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.230+0000 7f9c1ffff640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 0x7f9c0c071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000a8d0 tx=0x7f9c10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.234+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9c1405bf30 con 0x7f9c200b88b0 2024-05-17T22:03:12.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.430+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c200d11d0 con 0x7f9c200b88b0 2024-05-17T22:03:12.422 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:03:12.430+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:03:12.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.434+0000 7f9c1cff9640 1 -- 172.21.15.83:0/2805311766 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f9c1405b830 con 0x7f9c200b88b0 2024-05-17T22:03:12.426 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 msgr2=0x7f9c0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 0x7f9c0c071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000a8d0 tx=0x7f9c10008040 comp rx=0 tx=0).stop 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 msgr2=0x7f9c2015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c4f0 tx=0x7f9c1400c9b0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 shutdown_connections 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c200ce130 0x7f9c2015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c200b88b0 0x7f9c2015dc40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0c06f400 0x7f9c0c071910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 --2- 172.21.15.83:0/2805311766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c2009ba00 0x7f9c20153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 >> 172.21.15.83:0/2805311766 conn(0x7f9c200d3620 msgr2=0x7f9c200d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 shutdown_connections 2024-05-17T22:03:12.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.438+0000 7f9c265a2640 1 -- 172.21.15.83:0/2805311766 wait complete. 2024-05-17T22:03:12.434 INFO:tasks.ceph.mds.a.smithi083.stderr: -4047> 2024-05-17T22:02:57.358+0000 7ffa3e3b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:03:12.459 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-17T22:03:12.495 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:03:12.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 Processor -- start 2024-05-17T22:03:12.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- start start 2024-05-17T22:03:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea36a0 0x5595cfee90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cff5cc60 0x5595cff5b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595cfdef1b0 con 0x5595cfea2c00 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595cfe4aef0 con 0x5595cff5cc60 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5595cff74f50 con 0x5595cfea36a0 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35240/0 (socket says 172.21.15.83:35240) 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 -- 172.21.15.83:0/1322398445 learned_addr learned my addr 172.21.15.83:0/1322398445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 -- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea36a0 msgr2=0x5595cfee90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea36a0 0x5595cfee90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 -- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cff5cc60 msgr2=0x5595cff5b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cff5cc60 0x5595cff5b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 -- 172.21.15.83:0/1322398445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cfdc9c10 con 0x5595cfea2c00 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f9973fcf640 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9964009580 tx=0x7f9964009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f042d6fba76ead9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f99727cc640 1 -- 172.21.15.83:0/1322398445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964006610 con 0x5595cfea2c00 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f99727cc640 1 -- 172.21.15.83:0/1322398445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99640067b0 con 0x5595cfea2c00 2024-05-17T22:03:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f99727cc640 1 -- 172.21.15.83:0/1322398445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964006ba0 con 0x5595cfea2c00 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- 172.21.15.83:0/1322398445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 msgr2=0x5595cff59730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9964009580 tx=0x7f9964009a40 comp rx=0 tx=0).stop 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- 172.21.15.83:0/1322398445 shutdown_connections 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cff5cc60 0x5595cff5b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea36a0 0x5595cfee90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 --2- 172.21.15.83:0/1322398445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cfea2c00 0x5595cff59730 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- 172.21.15.83:0/1322398445 >> 172.21.15.83:0/1322398445 conn(0x5595cfe8be40 msgr2=0x5595cff5c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- 172.21.15.83:0/1322398445 shutdown_connections 2024-05-17T22:03:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.530+0000 7f997a4ed080 1 -- 172.21.15.83:0/1322398445 wait complete. 2024-05-17T22:03:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 Processor -- start 2024-05-17T22:03:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- start start 2024-05-17T22:03:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea2c00 0x5595cfee5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cfea36a0 0x5595d0009550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 0x5595cfee0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595cfee86f0 con 0x5595cff5cc60 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5595cff745d0 con 0x5595cfea36a0 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5595cfee8990 con 0x5595cfea2c00 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 0x5595cfee0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 0x5595cfee0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35250/0 (socket says 172.21.15.83:35250) 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 -- 172.21.15.83:0/3373275895 learned_addr learned my addr 172.21.15.83:0/3373275895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f9973fcf640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea2c00 0x5595cfee5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99737ce640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cfea36a0 0x5595d0009550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 -- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea2c00 msgr2=0x5595cfee5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea2c00 0x5595cfee5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 -- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cfea36a0 msgr2=0x5595d0009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595cfea36a0 0x5595d0009550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 -- 172.21.15.83:0/3373275895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d0009df0 con 0x5595cff5cc60 2024-05-17T22:03:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f9973fcf640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5595cfea2c00 0x5595cfee5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99747d0640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 0x5595cfee0f80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f996c006f80 tx=0x7f996c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99717ca640 1 -- 172.21.15.83:0/3373275895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0062d0 con 0x5595cff5cc60 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99717ca640 1 -- 172.21.15.83:0/3373275895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c003460 con 0x5595cff5cc60 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5595d000a090 con 0x5595cff5cc60 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f99717ca640 1 -- 172.21.15.83:0/3373275895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0038d0 con 0x5595cff5cc60 2024-05-17T22:03:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.534+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d000a5d0 con 0x5595cff5cc60 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99717ca640 1 -- 172.21.15.83:0/3373275895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f996c010070 con 0x5595cff5cc60 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99717ca640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f995406dde0 0x7f99540702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f9973fcf640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f995406dde0 0x7f99540702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99717ca640 1 -- 172.21.15.83:0/3373275895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5853+0+0 (secure 0 0 0) 0x7f996c08d9c0 con 0x5595cff5cc60 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5595cfee54b0 0x5595d000ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e83) -- 0x5595d000b250 con 0x5595cfee54b0 2024-05-17T22:03:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99737ce640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5595cfee54b0 0x5595d000ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f9973fcf640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f995406dde0 0x7f99540702f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9964014040 tx=0x7f996400f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99737ce640 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5595cfee54b0 0x5595d000ad10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99737ce640 1 -- 172.21.15.83:0/3373275895 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v83'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f995c00a230 con 0x5595cfee54b0 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5595cfee54b0 msgr2=0x5595d000ad10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5595cfee54b0 0x5595d000ad10 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f995406dde0 msgr2=0x7f99540702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f995406dde0 0x7f99540702f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9964014040 tx=0x7f996400f000 comp rx=0 tx=0).stop 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 msgr2=0x5595cfee0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f997a4ed080 1 --2- 172.21.15.83:0/3373275895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595cff5cc60 0x5595cfee0f80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f996c006f80 tx=0x7f996c00b4b0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.538+0000 7f99747d0640 1 -- 172.21.15.83:0/3373275895 reap_dead start 2024-05-17T22:03:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.542+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 shutdown_connections 2024-05-17T22:03:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.542+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 >> 172.21.15.83:0/3373275895 conn(0x5595cfe8be40 msgr2=0x5595cff59180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.542+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 shutdown_connections 2024-05-17T22:03:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.542+0000 7f997a4ed080 1 -- 172.21.15.83:0/3373275895 wait complete. 2024-05-17T22:03:12.539 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-17T22:03:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.646+0000 7ff9fc443400 1 -- start start 2024-05-17T22:03:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.646+0000 7ff9fc443400 1 Processor -- start 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db180 0x556c3eff5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 0x556c3eff4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3f012480 con 0x556c3e2d8d80 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c3f012600 con 0x556c3e2d9200 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556c3f012780 con 0x556c3e2db180 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35254/0 (socket says 172.21.15.83:35254) 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 -- 172.21.15.83:0/4118263090 learned_addr learned my addr 172.21.15.83:0/4118263090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db180 0x556c3eff5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fa401640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 0x556c3eff4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 -- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db180 msgr2=0x556c3eff5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db180 0x556c3eff5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 -- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 msgr2=0x556c3eff4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 0x556c3eff4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 -- 172.21.15.83:0/4118263090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3e352b40 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fa401640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 0x556c3eff4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fb403640 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x556c3f087620 tx=0x556c3e332900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfb328ee17f9ef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9f83fd640 1 -- 172.21.15.83:0/4118263090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e2501a0 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9f83fd640 1 -- 172.21.15.83:0/4118263090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c3e2641c0 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9f83fd640 1 -- 172.21.15.83:0/4118263090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e2504e0 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3e3530e0 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556c3e3532c0 con 0x556c3e2d8d80 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 4 waiting for MDS map... 2024-05-17T22:03:12.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9f83fd640 1 -- 172.21.15.83:0/4118263090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5853+0+0 (secure 0 0 0) 0x556c3e359600 con 0x556c3e2d8d80 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9f83fd640 1 -- 172.21.15.83:0/4118263090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 2885+0+0 (secure 0 0 0) 0x556c3e271180 con 0x556c3e2d8d80 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 4 Got MDS map 60 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 10 main: JournalTool::main 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 4 main: JournalTool: connecting to RADOS... 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 Processor -- start 2024-05-17T22:03:12.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- start start 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2da880 0x556c3eff6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3e27b980 con 0x556c3e2d8000 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c3f012f00 con 0x556c3e2d8480 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556c3f013080 con 0x556c3e2da880 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35262/0 (socket says 172.21.15.83:35262) 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 -- 172.21.15.83:0/2058407203 learned_addr learned my addr 172.21.15.83:0/2058407203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 -- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2da880 msgr2=0x556c3eff6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2da880 0x556c3eff6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fa401640 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 -- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 msgr2=0x556c3eff5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.650+0000 7ff9fac02640 1 -- 172.21.15.83:0/2058407203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3e3521e0 con 0x556c3e2d8000 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fac02640 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x556c3f087320 tx=0x556c3e331f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb9026db9bbcc92 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f63f9640 1 -- 172.21.15.83:0/2058407203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e250b60 con 0x556c3e2d8000 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f63f9640 1 -- 172.21.15.83:0/2058407203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c3e264c40 con 0x556c3e2d8000 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f63f9640 1 -- 172.21.15.83:0/2058407203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e2501a0 con 0x556c3e2d8000 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/2058407203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 msgr2=0x556c3eff4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x556c3f087320 tx=0x556c3e331f20 comp rx=0 tx=0).stop 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/2058407203 shutdown_connections 2024-05-17T22:03:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2da880 0x556c3eff6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8000 0x556c3eff4000 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- 172.21.15.83:0/2058407203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/2058407203 >> 172.21.15.83:0/2058407203 conn(0x556c3e2d8900 msgr2=0x556c3e2fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/2058407203 shutdown_connections 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/2058407203 wait complete. 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 Processor -- start 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- start start 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2d8000 0x556c3eff7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3f013380 con 0x556c3e2da880 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c3f013200 con 0x556c3e2d8480 2024-05-17T22:03:12.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556c3f013500 con 0x556c3e2d8000 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35278/0 (socket says 172.21.15.83:35278) 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 -- 172.21.15.83:0/3263603103 learned_addr learned my addr 172.21.15.83:0/3263603103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 -- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2d8000 msgr2=0x556c3eff7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2d8000 0x556c3eff7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 -- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 msgr2=0x556c3eff6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 -- 172.21.15.83:0/3263603103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3e353c20 con 0x556c3e2da880 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fa401640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x556c3f153c20 tx=0x556c3f15ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f4bf6640 1 -- 172.21.15.83:0/3263603103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e250b60 con 0x556c3e2da880 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f4bf6640 1 -- 172.21.15.83:0/3263603103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c3e265340 con 0x556c3e2da880 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c3e353e00 con 0x556c3e2da880 2024-05-17T22:03:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9f4bf6640 1 -- 172.21.15.83:0/3263603103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e2501a0 con 0x556c3e2da880 2024-05-17T22:03:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3e352960 con 0x556c3e2da880 2024-05-17T22:03:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.654+0000 7ff9fc443400 4 main: JournalTool: resolving pool 12 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9f4bf6640 1 -- 172.21.15.83:0/3263603103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x556c3e550c00 con 0x556c3e2da880 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9f4bf6640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 0x556c3eff7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9f4bf6640 1 -- 172.21.15.83:0/3263603103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5853+0+0 (secure 0 0 0) 0x556c3e560a00 con 0x556c3e2da880 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fac02640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 0x556c3eff7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 4 main: JournalTool: creating IoCtx.. 2024-05-17T22:03:12.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 4 main: Executing for rank 0 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fac02640 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 0x556c3eff7700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556c3f087320 tx=0x556c3e3306c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 -- start start 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 Processor -- start 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db600 0x556c3eff6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e6ac000 0x556c3eff4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556c3f013080 con 0x556c3e2da400 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556c3f012f00 con 0x556c3e6ac000 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fc443400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x556c3f013800 con 0x556c3e2db600 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fac02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fac02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35284/0 (socket says 172.21.15.83:35284) 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.658+0000 7ff9fac02640 1 -- 172.21.15.83:0/1916004770 learned_addr learned my addr 172.21.15.83:0/1916004770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fb403640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db600 0x556c3eff6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e6ac000 0x556c3eff4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db600 msgr2=0x556c3eff6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db600 0x556c3eff6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e6ac000 msgr2=0x556c3eff4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e6ac000 0x556c3eff4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3e6a63c0 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e6ac000 0x556c3eff4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fac02640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x556c3f152b10 tx=0x556c3e3305a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe6e7601951ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9f1bf0640 1 -- 172.21.15.83:0/1916004770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e2504e0 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9f1bf0640 1 -- 172.21.15.83:0/1916004770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c3e264a80 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9f1bf0640 1 -- 172.21.15.83:0/1916004770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c3e250340 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3e6a65a0 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556c3e6a6780 con 0x556c3e2da400 2024-05-17T22:03:12.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 4 waiting for MDS map... 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9f1bf0640 1 -- 172.21.15.83:0/1916004770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5853+0+0 (secure 0 0 0) 0x556c3e6b0200 con 0x556c3e2da400 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9f1bf0640 1 -- 172.21.15.83:0/1916004770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 2885+0+0 (secure 0 0 0) 0x556c3e271500 con 0x556c3e2da400 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 4 Got MDS map 60 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x556c3e6ac480 0x556c3e6c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cc000 con 0x556c3e6ac480 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fb403640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x556c3e6ac480 0x556c3e6c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fb403640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x556c3e6ac480 0x556c3e6c8000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=74772135206f7807 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6ac480 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x556c3e6ac900 0x556c3e6c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cc400 con 0x556c3e6ac900 2024-05-17T22:03:12.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.662+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x556c3e6ac900 0x556c3e6c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x556c3e6ac900 0x556c3e6c8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=382d3e4ee3f72cc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x556c3e31e500 con 0x556c3e6ac900 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9f03ed640 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cc800 con 0x556c3e6ac900 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9f03ed640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x556c3e6af600 0x556c3e6ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9f03ed640 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6ccc00 con 0x556c3e6af600 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9fb403640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x556c3e6af600 0x556c3e6ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x556c3e31e500 con 0x556c3e6ac900 2024-05-17T22:03:12.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.666+0000 7ff9fb403640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x556c3e6af600 0x556c3e6ca680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e9f96a85c39edd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6af600 2024-05-17T22:03:12.665 INFO:teuthology.orchestra.run.smithi083.stdout:old journal was 4210858~892 2024-05-17T22:03:12.665 INFO:teuthology.orchestra.run.smithi083.stdout:new journal start will be 8388608 (4176858 bytes past old end) 2024-05-17T22:03:12.665 INFO:teuthology.orchestra.run.smithi083.stdout:writing journal head 2024-05-17T22:03:12.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cd400 con 0x556c3e6ac900 2024-05-17T22:03:12.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v83'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e500 con 0x556c3e6ac900 2024-05-17T22:03:12.667 INFO:teuthology.orchestra.run.smithi083.stdout:writing EResetJournal entry 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cd800 con 0x556c3e6af600 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556c3e6acd80 0x556c3e6c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cdc00 con 0x556c3e6acd80 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cc000 con 0x556c3e6ac480 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6ccc00 con 0x556c3e6af600 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3f145400 con 0x556c3e6af600 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x556c3e6ad200 0x556c3e6c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3f145000 con 0x556c3e6ad200 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fac02640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556c3e6acd80 0x556c3e6c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x556c3e6ad200 0x556c3e6c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fac02640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556c3e6acd80 0x556c3e6c8b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=f779476118991505 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.674+0000 7ff9fa401640 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x556c3e6ad200 0x556c3e6c9080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=39da1d15d5ef6900 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fac02640 1 -- 172.21.15.83:0/1916004770 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(7 200.00000003 [delete] v83'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e780 con 0x556c3e6acd80 2024-05-17T22:03:12.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(11 200.00000007 [delete] v83'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e500 con 0x556c3e6ad200 2024-05-17T22:03:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(6 200.00000002 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6af600 2024-05-17T22:03:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9f03ed640 1 -- 172.21.15.83:0/1916004770 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x556c3e6cd000 con 0x556c3e6af600 2024-05-17T22:03:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(8 200.00000004 [delete] v83'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6ac480 2024-05-17T22:03:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(9 200.00000005 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6af600 2024-05-17T22:03:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.678+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(10 200.00000006 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6af600 2024-05-17T22:03:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fb403640 1 -- 172.21.15.83:0/1916004770 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v83'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x556c3e31e000 con 0x556c3e6af600 2024-05-17T22:03:12.673 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x556c3e6ad200 msgr2=0x556c3e6c9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x556c3e6ad200 0x556c3e6c9080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556c3e6acd80 msgr2=0x556c3e6c8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x556c3e6acd80 0x556c3e6c8b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x556c3e6af600 msgr2=0x556c3e6ca680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x556c3e6af600 0x556c3e6ca680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x556c3e6ac900 msgr2=0x556c3e6c8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x556c3e6ac900 0x556c3e6c8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 reap_dead start 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x556c3e6ac480 msgr2=0x556c3e6c8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x556c3e6ac480 0x556c3e6c8000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fa401640 1 -- 172.21.15.83:0/1916004770 reap_dead start 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 msgr2=0x556c3eff5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x556c3f152b10 tx=0x556c3e3305a0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 shutdown_connections 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/1916004770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da400 0x556c3eff5600 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 >> 172.21.15.83:0/1916004770 conn(0x556c3e2dad00 msgr2=0x556c3e2fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 shutdown_connections 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/1916004770 wait complete. 2024-05-17T22:03:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 msgr2=0x556c3eff7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 0x556c3eff7700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556c3f087320 tx=0x556c3e3306c0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 msgr2=0x556c3eff6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x556c3f153c20 tx=0x556c3f15ac00 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 shutdown_connections 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2da880 0x556c3eff6680 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x556c3e2dba80 0x556c3eff7700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d8480 0x556c3eff6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/3263603103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2d8000 0x556c3eff7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 >> 172.21.15.83:0/3263603103 conn(0x556c3e2d8900 msgr2=0x556c3e2fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 shutdown_connections 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/3263603103 wait complete. 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 msgr2=0x556c3eff5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x556c3f087620 tx=0x556c3e332900 comp rx=0 tx=0).stop 2024-05-17T22:03:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 shutdown_connections 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556c3e2d8d80 0x556c3eff5b80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c3e2d9200 0x556c3eff4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 --2- 172.21.15.83:0/4118263090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x556c3e2db180 0x556c3eff5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 >> 172.21.15.83:0/4118263090 conn(0x556c3e2d9f80 msgr2=0x556c3e2fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 shutdown_connections 2024-05-17T22:03:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.682+0000 7ff9fc443400 1 -- 172.21.15.83:0/4118263090 wait complete. 2024-05-17T22:03:12.680 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.141606, result: old journal was 4210858~892 new journal start will be 8388608 (4176858 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-17T22:03:12.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:03:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 Processor -- start 2024-05-17T22:03:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 -- start start 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 0x7f8f540b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f540b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f5406f2d0 con 0x7f8f5415be70 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8f5406f5a0 con 0x7f8f5406eec0 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5beaa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5406f880 con 0x7f8f540b44c0 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5a447640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f540b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f59c46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 0x7f8f540b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f59c46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 0x7f8f540b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:60918/0 (socket says 172.21.15.83:60918) 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f59445640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8f5415be70 0x7f8f541601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:59704/0 (socket says 172.21.15.83:59704) 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f59c46640 1 -- 172.21.15.83:0/1043476738 learned_addr learned my addr 172.21.15.83:0/1043476738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5a447640 1 -- 172.21.15.83:0/1043476738 >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 msgr2=0x7f8f540b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5a447640 1 --2- 172.21.15.83:0/1043476738 >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 0x7f8f540b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5a447640 1 -- 172.21.15.83:0/1043476738 >> v1:172.21.15.155:6789/0 conn(0x7f8f5415be70 legacy=0x7f8f541601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.830+0000 7f8f5a447640 1 -- 172.21.15.83:0/1043476738 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5406fb60 con 0x7f8f540b44c0 2024-05-17T22:03:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5a447640 1 --2- 172.21.15.83:0/1043476738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f540b48d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009d90 tx=0x7f8f4c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc3a4e0adee9443 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f58c44640 1 -- 172.21.15.83:0/1043476738 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0b2070 con 0x7f8f540b44c0 2024-05-17T22:03:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f58c44640 1 -- 172.21.15.83:0/1043476738 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0a57a0 con 0x7f8f540b44c0 2024-05-17T22:03:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f58c44640 1 -- 172.21.15.83:0/1043476738 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0a5b70 con 0x7f8f540b44c0 2024-05-17T22:03:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 -- 172.21.15.83:0/1043476738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 msgr2=0x7f8f540b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/1043476738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f540b48d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009d90 tx=0x7f8f4c01c8e0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 -- 172.21.15.83:0/1043476738 shutdown_connections 2024-05-17T22:03:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/1043476738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f540b48d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/1043476738 >> v2:172.21.15.155:3300/0 conn(0x7f8f5406eec0 0x7f8f540b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 -- 172.21.15.83:0/1043476738 >> 172.21.15.83:0/1043476738 conn(0x7f8f5406cd90 msgr2=0x7f8f540b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 -- 172.21.15.83:0/1043476738 shutdown_connections 2024-05-17T22:03:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.834+0000 7f8f5beaa640 1 -- 172.21.15.83:0/1043476738 wait complete. 2024-05-17T22:03:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 Processor -- start 2024-05-17T22:03:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- start start 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f54152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5415be70 0x7f8f54152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54168e00 con 0x7f8f5406eec0 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54168c80 con 0x7f8f5415be70 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f54168f80 con 0x7f8f540b44c0 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35292/0 (socket says 172.21.15.83:35292) 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 -- 172.21.15.83:0/3334389931 learned_addr learned my addr 172.21.15.83:0/3334389931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59445640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f54152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5a447640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5415be70 0x7f8f54152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 -- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 msgr2=0x7f8f54152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f54152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 -- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5415be70 msgr2=0x7f8f54152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5415be70 0x7f8f54152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 -- 172.21.15.83:0/3334389931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f54153510 con 0x7f8f5406eec0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f59c46640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e820 tx=0x7f8f5000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50017070 con 0x7f8f5406eec0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5415aac0 con 0x7f8f5406eec0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5415b080 con 0x7f8f5406eec0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5000c940 con 0x7f8f5406eec0 2024-05-17T22:03:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.838+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50021060 con 0x7f8f5406eec0 2024-05-17T22:03:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c0052a0 con 0x7f8f5406eec0 2024-05-17T22:03:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8f5001a070 con 0x7f8f5406eec0 2024-05-17T22:03:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f467fc640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 0x7f8f38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5853+0+0 (secure 0 0 0) 0x7f8f5008f5d0 con 0x7f8f5406eec0 2024-05-17T22:03:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f59445640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 0x7f8f38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:12.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.842+0000 7f8f59445640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 0x7f8f38071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8f5414d530 tx=0x7f8f48007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:12.846+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8f5005c3b0 con 0x7f8f5406eec0 2024-05-17T22:03:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.046+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8f3c005860 con 0x7f8f5406eec0 2024-05-17T22:03:13.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.438+0000 7f8f467fc640 1 -- 172.21.15.83:0/3334389931 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v61) ==== 162+0+0 (secure 0 0 0) 0x7f8f5005bcb0 con 0x7f8f5406eec0 2024-05-17T22:03:13.430 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:03:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 msgr2=0x7f8f38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 0x7f8f38071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8f5414d530 tx=0x7f8f48007cb0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 msgr2=0x7f8f541521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e820 tx=0x7f8f5000ece0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 shutdown_connections 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3806f400 0x7f8f38071910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5415be70 0x7f8f54152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f540b44c0 0x7f8f54152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 --2- 172.21.15.83:0/3334389931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f5406eec0 0x7f8f541521d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 >> 172.21.15.83:0/3334389931 conn(0x7f8f5406cd90 msgr2=0x7f8f541577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 shutdown_connections 2024-05-17T22:03:13.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.442+0000 7f8f5beaa640 1 -- 172.21.15.83:0/3334389931 wait complete. 2024-05-17T22:03:13.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:13.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 Processor -- start 2024-05-17T22:03:13.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 -- start start 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4df4144080 0x7f4df4072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4df4073370 0x7f4df406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df4073780 con 0x7f4df40d1040 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4df415e790 con 0x7f4df4073370 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfc048640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4df415ea40 con 0x7f4df4144080 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4df4073370 0x7f4df406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4df4073370 0x7f4df406f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35306/0 (socket says 172.21.15.83:35306) 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/184585991 learned_addr learned my addr 172.21.15.83:0/184585991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4df95e3640 1 --2- 172.21.15.83:0/184585991 >> v2:172.21.15.155:3300/0 conn(0x7f4df4144080 0x7f4df4072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/184585991 >> v2:172.21.15.155:3300/0 conn(0x7f4df4144080 msgr2=0x7f4df4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:13.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 --2- 172.21.15.83:0/184585991 >> v2:172.21.15.155:3300/0 conn(0x7f4df4144080 0x7f4df4072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/184585991 >> v1:172.21.15.155:6789/0 conn(0x7f4df40d1040 legacy=0x7f4df415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:13.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.606+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/184585991 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df415ed20 con 0x7f4df4073370 2024-05-17T22:03:13.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfa5e5640 1 --2- 172.21.15.83:0/184585991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4073370 0x7f4df406f730 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4df40b6b10 tx=0x7f4dec01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df73c39ad4a7dc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:13.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4df8de2640 1 -- 172.21.15.83:0/184585991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0a5660 con 0x7f4df4073370 2024-05-17T22:03:13.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4df8de2640 1 -- 172.21.15.83:0/184585991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec0a5800 con 0x7f4df4073370 2024-05-17T22:03:13.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4df8de2640 1 -- 172.21.15.83:0/184585991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0a5bd0 con 0x7f4df4073370 2024-05-17T22:03:13.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 -- 172.21.15.83:0/184585991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4073370 msgr2=0x7f4df406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 --2- 172.21.15.83:0/184585991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4073370 0x7f4df406f730 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4df40b6b10 tx=0x7f4dec01c7e0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 -- 172.21.15.83:0/184585991 shutdown_connections 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 --2- 172.21.15.83:0/184585991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4073370 0x7f4df406f730 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 --2- 172.21.15.83:0/184585991 >> v2:172.21.15.155:3300/0 conn(0x7f4df4144080 0x7f4df4072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 -- 172.21.15.83:0/184585991 >> 172.21.15.83:0/184585991 conn(0x7f4df406cd90 msgr2=0x7f4df40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 -- 172.21.15.83:0/184585991 shutdown_connections 2024-05-17T22:03:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.610+0000 7f4dfc048640 1 -- 172.21.15.83:0/184585991 wait complete. 2024-05-17T22:03:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 Processor -- start 2024-05-17T22:03:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- start start 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4df4073370 0x7f4df415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4df40d1040 0x7f4df413cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4070480 con 0x7f4df4144080 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4df4144490 con 0x7f4df4073370 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4df4070600 con 0x7f4df40d1040 2024-05-17T22:03:13.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35314/0 (socket says 172.21.15.83:35314) 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/959150339 learned_addr learned my addr 172.21.15.83:0/959150339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4df9de4640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4df4073370 0x7f4df415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4df95e3640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4df40d1040 0x7f4df413cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4df40d1040 msgr2=0x7f4df413cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4df40d1040 0x7f4df413cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4df4073370 msgr2=0x7f4df415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4df4073370 0x7f4df415d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 -- 172.21.15.83:0/959150339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df40b4810 con 0x7f4df4144080 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfa5e5640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4dec01ae90 tx=0x7f4dec019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec009510 con 0x7f4df4144080 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dec0be070 con 0x7f4df4144080 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df413b280 con 0x7f4df4144080 2024-05-17T22:03:13.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0b88b0 con 0x7f4df4144080 2024-05-17T22:03:13.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.614+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df413b840 con 0x7f4df4144080 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df413ba10 con 0x7f4df4144080 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4dec0b8a50 con 0x7f4df4144080 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4dea7fc640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 0x7f4dd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f4dec129ec0 con 0x7f4df4144080 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4df9de4640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 0x7f4dd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:13.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.618+0000 7f4df9de4640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 0x7f4dd8071940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4df000b810 tx=0x7f4df0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:13.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.622+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4dec0c7020 con 0x7f4df4144080 2024-05-17T22:03:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.802+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4df4070990 con 0x7f4df4144080 2024-05-17T22:03:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.802+0000 7f4dea7fc640 1 -- 172.21.15.83:0/959150339 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5180 (secure 0 0 0) 0x7f4df4073780 con 0x7f4df4144080 2024-05-17T22:03:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:13.795 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:13.447787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-17T22:03:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2024-05-17T22:03:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 msgr2=0x7f4dd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 0x7f4dd8071940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4df000b810 tx=0x7f4df0008040 comp rx=0 tx=0).stop 2024-05-17T22:03:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 msgr2=0x7f4df413d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4dec01ae90 tx=0x7f4dec019580 comp rx=0 tx=0).stop 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 shutdown_connections 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4df4144080 0x7f4df413d4c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4df40d1040 0x7f4df413cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dd806f430 0x7f4dd8071940 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 --2- 172.21.15.83:0/959150339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4df4073370 0x7f4df415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 >> 172.21.15.83:0/959150339 conn(0x7f4df406cd90 msgr2=0x7f4df41466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 shutdown_connections 2024-05-17T22:03:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:13.806+0000 7f4dfc048640 1 -- 172.21.15.83:0/959150339 wait complete. 2024-05-17T22:03:14.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:03:13.447787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6801': {'gid': 6801, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6836/165849096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 165849096}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 165849096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:03:14.218 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6801:up:replay 2024-05-17T22:03:15.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:15.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 Processor -- start 2024-05-17T22:03:15.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 -- start start 2024-05-17T22:03:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13b415e9e0 0x7f13b4165230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b40b5b10 0x7f13b40b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b4167da0 con 0x7f13b40b4810 2024-05-17T22:03:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f13b4168990 con 0x7f13b415e9e0 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b8f9e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13b4168060 con 0x7f13b40b5b10 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b1d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f13b40b4810 0x7f13b40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35698/0 (socket says 172.21.15.83:35698) 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b1d9b640 1 -- 172.21.15.83:0/737390309 learned_addr learned my addr 172.21.15.83:0/737390309 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586450446 0 0) 0x7f13b4167da0 con 0x7f13b40b4810 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b40b5b10 0x7f13b40b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a0003530 con 0x7f13b40b4810 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.386+0000 7f13b259c640 1 --2- 172.21.15.83:0/737390309 >> v2:172.21.15.155:3300/0 conn(0x7f13b415e9e0 0x7f13b4165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 547795309 0 0) 0x7f13a0003530 con 0x7f13b40b4810 2024-05-17T22:03:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b40b5b10 msgr2=0x7f13b40b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b40b5b10 0x7f13b40b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 >> v2:172.21.15.155:3300/0 conn(0x7f13b415e9e0 msgr2=0x7f13b4165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 --2- 172.21.15.83:0/737390309 >> v2:172.21.15.155:3300/0 conn(0x7f13b415e9e0 0x7f13b4165230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4168340 con 0x7f13b40b4810 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f13ac001a80 con 0x7f13b40b4810 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 >> v1:172.21.15.83:6789/0 conn(0x7f13b40b4810 legacy=0x7f13b40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0003a10 0x7f13a0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a0004810 0x7f13a0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0005160 0x7f13a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4168990 con 0x7f13a0005160 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4168060 con 0x7f13a0003a10 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13a000bf10 con 0x7f13a0004810 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0005160 0x7f13a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b259c640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0003a10 0x7f13a0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b1d9b640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a0004810 0x7f13a0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 -- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a0004810 msgr2=0x7f13a0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13a0004810 0x7f13a0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 -- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0003a10 msgr2=0x7f13a0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13a0003a10 0x7f13a0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 -- 172.21.15.83:0/737390309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a0003530 con 0x7f13a0005160 2024-05-17T22:03:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0005160 0x7f13a000b9d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f13b40b1af0 tx=0x7f13a401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9125d56cd128d8f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40b2070 con 0x7f13a0005160 2024-05-17T22:03:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a40a5620 con 0x7f13a0005160 2024-05-17T22:03:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b159a640 1 -- 172.21.15.83:0/737390309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40096b0 con 0x7f13a0005160 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b8f9e640 1 -- 172.21.15.83:0/737390309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0005160 msgr2=0x7f13a000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/737390309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0005160 0x7f13a000b9d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f13b40b1af0 tx=0x7f13a401ca30 comp rx=0 tx=0).stop 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.390+0000 7f13b2d9d640 1 -- 172.21.15.83:0/737390309 reap_dead start 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- 172.21.15.83:0/737390309 shutdown_connections 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- 172.21.15.83:0/737390309 >> 172.21.15.83:0/737390309 conn(0x7f13b409b8b0 msgr2=0x7f13b40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- 172.21.15.83:0/737390309 shutdown_connections 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- 172.21.15.83:0/737390309 wait complete. 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 Processor -- start 2024-05-17T22:03:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- start start 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b4153d70 0x7f13b414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b4155e50 0x7f13b40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13b40b5c90 con 0x7f13b4156260 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13b40b5b10 con 0x7f13b4153d70 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b8f9e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f13b40b1f70 con 0x7f13b4155e50 2024-05-17T22:03:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35326/0 (socket says 172.21.15.83:35326) 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 -- 172.21.15.83:0/413542909 learned_addr learned my addr 172.21.15.83:0/413542909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b259c640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b4153d70 0x7f13b414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b1d9b640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b4155e50 0x7f13b40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 -- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b4155e50 msgr2=0x7f13b40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b4155e50 0x7f13b40b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 -- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b4153d70 msgr2=0x7f13b414fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b4153d70 0x7f13b414fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 -- 172.21.15.83:0/413542909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4154330 con 0x7f13b4156260 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.394+0000 7f13b2d9d640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f13a40b0040 tx=0x7f13a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40b2040 con 0x7f13b4156260 2024-05-17T22:03:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a40bb070 con 0x7f13b4156260 2024-05-17T22:03:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b4154610 con 0x7f13b4156260 2024-05-17T22:03:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40096b0 con 0x7f13b4156260 2024-05-17T22:03:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b4157630 con 0x7f13b4156260 2024-05-17T22:03:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b4165230 con 0x7f13b4156260 2024-05-17T22:03:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13a40ad640 con 0x7f13b4156260 2024-05-17T22:03:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13b259c640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f13a4129290 con 0x7f13b4156260 2024-05-17T22:03:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.398+0000 7f13b259c640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 0x7f1394071910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f139c009a30 tx=0x7f139c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:15.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.402+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f13a40c4020 con 0x7f13b4156260 2024-05-17T22:03:15.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.602+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13b40b1d50 con 0x7f13b4156260 2024-05-17T22:03:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.602+0000 7f13aaffd640 1 -- 172.21.15.83:0/413542909 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5180 (secure 0 0 0) 0x7f13a4016740 con 0x7f13b4156260 2024-05-17T22:03:15.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:15.597 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:14.458347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-17T22:03:15.597 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 64 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 msgr2=0x7f1394071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 0x7f1394071910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f139c009a30 tx=0x7f139c008040 comp rx=0 tx=0).stop 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 msgr2=0x7f13b40b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f13a40b0040 tx=0x7f13a0005840 comp rx=0 tx=0).stop 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 shutdown_connections 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b4156260 0x7f13b40b0970 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b4155e50 0x7f13b40b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f139406f400 0x7f1394071910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 --2- 172.21.15.83:0/413542909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b4153d70 0x7f13b414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 >> 172.21.15.83:0/413542909 conn(0x7f13b409b8b0 msgr2=0x7f13b40b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 shutdown_connections 2024-05-17T22:03:15.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:15.606+0000 7f13b8f9e640 1 -- 172.21.15.83:0/413542909 wait complete. 2024-05-17T22:03:16.022 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:03:14.458347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6801': {'gid': 6801, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6836/165849096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 165849096}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 165849096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:03:16.022 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6801:up:rejoin 2024-05-17T22:03:17.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:17.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 Processor -- start 2024-05-17T22:03:17.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 -- start start 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4bc15ffa0 0x7ff4bc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc06e200 0x7ff4bc163a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc06e610 con 0x7ff4bc0d1150 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc0d34e0 con 0x7ff4bc15ffa0 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c3d2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc0d3780 con 0x7ff4bc06e200 2024-05-17T22:03:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50740/0 (socket says 172.21.15.83:50740) 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 -- 172.21.15.83:0/2034874431 learned_addr learned my addr 172.21.15.83:0/2034874431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c1ac9640 1 --2- 172.21.15.83:0/2034874431 >> v2:172.21.15.155:3300/0 conn(0x7ff4bc15ffa0 0x7ff4bc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c22ca640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc06e200 0x7ff4bc163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 -- 172.21.15.83:0/2034874431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc06e200 msgr2=0x7ff4bc163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc06e200 0x7ff4bc163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 -- 172.21.15.83:0/2034874431 >> v2:172.21.15.155:3300/0 conn(0x7ff4bc15ffa0 msgr2=0x7ff4bc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/2034874431 >> v2:172.21.15.155:3300/0 conn(0x7ff4bc15ffa0 0x7ff4bc0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.198+0000 7ff4c12c8640 1 -- 172.21.15.83:0/2034874431 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc0d3a60 con 0x7ff4bc0d1150 2024-05-17T22:03:17.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc08e380 tx=0x7ff4b001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda8d647387756e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:17.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c0ac7640 1 -- 172.21.15.83:0/2034874431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00b0070 con 0x7ff4bc0d1150 2024-05-17T22:03:17.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c0ac7640 1 -- 172.21.15.83:0/2034874431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b00ab600 con 0x7ff4bc0d1150 2024-05-17T22:03:17.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c0ac7640 1 -- 172.21.15.83:0/2034874431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00ab9d0 con 0x7ff4bc0d1150 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/2034874431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc0d1150 msgr2=0x7ff4bc06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc08e380 tx=0x7ff4b001c260 comp rx=0 tx=0).stop 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/2034874431 shutdown_connections 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc06e200 0x7ff4bc163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/2034874431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc06dcc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/2034874431 >> v2:172.21.15.155:3300/0 conn(0x7ff4bc15ffa0 0x7ff4bc0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/2034874431 >> 172.21.15.83:0/2034874431 conn(0x7ff4bc0d28f0 msgr2=0x7ff4bc0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/2034874431 shutdown_connections 2024-05-17T22:03:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/2034874431 wait complete. 2024-05-17T22:03:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 Processor -- start 2024-05-17T22:03:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.202+0000 7ff4c3d2d640 1 -- start start 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc162b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc15ffa0 0x7ff4bc0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc0d43e0 con 0x7ff4bc06e200 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc0d4260 con 0x7ff4bc0d1150 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc0d4560 con 0x7ff4bc15ffa0 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50748/0 (socket says 172.21.15.83:50748) 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 -- 172.21.15.83:0/3282553272 learned_addr learned my addr 172.21.15.83:0/3282553272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c22ca640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc15ffa0 0x7ff4bc0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 -- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc15ffa0 msgr2=0x7ff4bc0b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc15ffa0 0x7ff4bc0b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 -- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 msgr2=0x7ff4bc162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc162b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 -- 172.21.15.83:0/3282553272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc1572f0 con 0x7ff4bc06e200 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c1ac9640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac007e70 tx=0x7ff4ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac014620 con 0x7ff4bc06e200 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac019070 con 0x7ff4bc06e200 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc1575d0 con 0x7ff4bc06e200 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac014a00 con 0x7ff4bc06e200 2024-05-17T22:03:17.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc157b90 con 0x7ff4bc06e200 2024-05-17T22:03:17.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc06dcc0 con 0x7ff4bc06e200 2024-05-17T22:03:17.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.206+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4ac014ba0 con 0x7ff4bc06e200 2024-05-17T22:03:17.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.210+0000 7ff4ba7fc640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 0x7ff4a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.210+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7ff4ac08fbe0 con 0x7ff4bc06e200 2024-05-17T22:03:17.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.210+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 0x7ff4a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.210+0000 7ff4c12c8640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 0x7ff4a8071910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00ad000 tx=0x7ff4b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:17.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.214+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff4ac05ca40 con 0x7ff4bc06e200 2024-05-17T22:03:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.410+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4bc15f420 con 0x7ff4bc06e200 2024-05-17T22:03:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.410+0000 7ff4ba7fc640 1 -- 172.21.15.83:0/3282553272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5187 (secure 0 0 0) 0x7ff4ac05c340 con 0x7ff4bc06e200 2024-05-17T22:03:17.401 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:17.401 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:16.463193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":6}]} 2024-05-17T22:03:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 65 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 msgr2=0x7ff4a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 0x7ff4a8071910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00ad000 tx=0x7ff4b0008040 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 msgr2=0x7ff4bc1489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac007e70 tx=0x7ff4ac00e4b0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 shutdown_connections 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4bc15ffa0 0x7ff4bc0b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff4a806f400 0x7ff4a8071910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4bc0d1150 0x7ff4bc162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 --2- 172.21.15.83:0/3282553272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4bc06e200 0x7ff4bc1489e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 >> 172.21.15.83:0/3282553272 conn(0x7ff4bc0d28f0 msgr2=0x7ff4bc0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 shutdown_connections 2024-05-17T22:03:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:17.414+0000 7ff4c3d2d640 1 -- 172.21.15.83:0/3282553272 wait complete. 2024-05-17T22:03:17.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:03:16.463193+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6801}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6801': {'gid': 6801, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6836/165849096', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 165849096}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 165849096}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6801, 'qdb_cluster': [6801]} 2024-05-17T22:03:17.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:03:17.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:03:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 Processor -- start 2024-05-17T22:03:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 -- start start 2024-05-17T22:03:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 0x7f52c80d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 0x7f52c815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52c80d2000 0x7f52c80d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f52c80d4170 con 0x7f52c80d2000 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f52c80d4450 con 0x7f52c80b4810 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cfcc5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52c80d4730 con 0x7f52c80d0290 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52c80d2000 0x7f52c80d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52c80d2000 0x7f52c80d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50750/0 (socket says 172.21.15.83:50750) 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 -- 172.21.15.83:0/526482809 learned_addr learned my addr 172.21.15.83:0/526482809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cd260640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 0x7f52c815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cda61640 1 --2- 172.21.15.83:0/526482809 >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 0x7f52c80d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 -- 172.21.15.83:0/526482809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 msgr2=0x7f52c815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 0x7f52c815fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 -- 172.21.15.83:0/526482809 >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 msgr2=0x7f52c80d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:17.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 --2- 172.21.15.83:0/526482809 >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 0x7f52c80d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52ce262640 1 -- 172.21.15.83:0/526482809 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c80d4a10 con 0x7f52c80d2000 2024-05-17T22:03:17.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cd260640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 0x7f52c815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:17.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.002+0000 7f52cda61640 1 --2- 172.21.15.83:0/526482809 >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 0x7f52c80d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52ce262640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80d2000 0x7f52c80d2410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52c808e620 tx=0x7f52c001d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910b29822379e5e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cca5f640 1 -- 172.21.15.83:0/526482809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00a5890 con 0x7f52c80d2000 2024-05-17T22:03:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cca5f640 1 -- 172.21.15.83:0/526482809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00a5a30 con 0x7f52c80d2000 2024-05-17T22:03:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cca5f640 1 -- 172.21.15.83:0/526482809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00a5e00 con 0x7f52c80d2000 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 -- 172.21.15.83:0/526482809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80d2000 msgr2=0x7f52c80d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80d2000 0x7f52c80d2410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52c808e620 tx=0x7f52c001d450 comp rx=0 tx=0).stop 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 -- 172.21.15.83:0/526482809 shutdown_connections 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80d2000 0x7f52c80d2410 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/526482809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d0290 0x7f52c815fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/526482809 >> v2:172.21.15.155:3300/0 conn(0x7f52c80b4810 0x7f52c80d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 -- 172.21.15.83:0/526482809 >> 172.21.15.83:0/526482809 conn(0x7f52c806e010 msgr2=0x7f52c806e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 -- 172.21.15.83:0/526482809 shutdown_connections 2024-05-17T22:03:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.006+0000 7f52cfcc5640 1 -- 172.21.15.83:0/526482809 wait complete. 2024-05-17T22:03:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 Processor -- start 2024-05-17T22:03:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- start start 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 0x7f52c815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 0x7f52c8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52c80b7aa0 con 0x7f52c80b4810 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8160050 con 0x7f52c80d0290 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52c80b7c20 con 0x7f52c80d2000 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50762/0 (socket says 172.21.15.83:50762) 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 -- 172.21.15.83:0/1651384469 learned_addr learned my addr 172.21.15.83:0/1651384469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cd260640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 0x7f52c815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52ce262640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 0x7f52c8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 -- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 msgr2=0x7f52c8156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 0x7f52c8156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 -- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 msgr2=0x7f52c815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 0x7f52c815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 -- 172.21.15.83:0/1651384469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c80cfcf0 con 0x7f52c80b4810 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cd260640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 0x7f52c815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52ce262640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 0x7f52c8156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cda61640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c820 tx=0x7f52c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52be7fc640 1 -- 172.21.15.83:0/1651384469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4015070 con 0x7f52c80b4810 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52be7fc640 1 -- 172.21.15.83:0/1651384469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c40079d0 con 0x7f52c80b4810 2024-05-17T22:03:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c8162a50 con 0x7f52c80b4810 2024-05-17T22:03:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52be7fc640 1 -- 172.21.15.83:0/1651384469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4007da0 con 0x7f52c80b4810 2024-05-17T22:03:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.010+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c8161c90 con 0x7f52c80b4810 2024-05-17T22:03:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52be7fc640 1 -- 172.21.15.83:0/1651384469 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52c4018070 con 0x7f52c80b4810 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52be7fc640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 0x7f52ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52be7fc640 1 -- 172.21.15.83:0/1651384469 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f52c408f1f0 con 0x7f52c80b4810 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52cd260640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 0x7f52ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52cd260640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 0x7f52ac071940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f52c806e6d0 tx=0x7f52b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 Processor -- start 2024-05-17T22:03:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 -- start start 2024-05-17T22:03:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e112adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e112b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628e0fc8ca0 con 0x5628e0fe4fa0 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f3b110 con 0x5628e0fda600 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.014+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5628e0f2acb0 con 0x5628e0fc3d30 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50764/0 (socket says 172.21.15.83:50764) 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 -- 172.21.15.83:0/832298188 learned_addr learned my addr 172.21.15.83:0/832298188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52ce262640 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e112b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cd260640 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e112adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 -- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 msgr2=0x5628e112b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e112b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 -- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 msgr2=0x5628e112adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e112adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 -- 172.21.15.83:0/832298188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e0eef370 con 0x5628e0fe4fa0 2024-05-17T22:03:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52cda61640 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f52c4000c10 tx=0x7f52c405b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c67c90911e9f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52ab7fe640 1 -- 172.21.15.83:0/832298188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c401f040 con 0x5628e0fe4fa0 2024-05-17T22:03:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52ab7fe640 1 -- 172.21.15.83:0/832298188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c4064100 con 0x5628e0fe4fa0 2024-05-17T22:03:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52ab7fe640 1 -- 172.21.15.83:0/832298188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c405adb0 con 0x5628e0fe4fa0 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- 172.21.15.83:0/832298188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 msgr2=0x5628e112a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f52c4000c10 tx=0x7f52c405b850 comp rx=0 tx=0).stop 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- 172.21.15.83:0/832298188 shutdown_connections 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e112b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e112adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/832298188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e112a8b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- 172.21.15.83:0/832298188 >> 172.21.15.83:0/832298188 conn(0x5628e0bec460 msgr2=0x5628e10dfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- 172.21.15.83:0/832298188 shutdown_connections 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- 172.21.15.83:0/832298188 wait complete. 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 -- start start 2024-05-17T22:03:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 Processor -- start 2024-05-17T22:03:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.018+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fe4fa0 0x5628e10ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628e0ee6d40 con 0x5628e0fda600 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f2ecf0 con 0x5628e0fe4fa0 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5628e0ed3e90 con 0x5628e0fc3d30 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 10 client.0 _ll_register_callbacks cb 0x5628e0f08fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50780/0 (socket says 172.21.15.83:50780) 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 -- 172.21.15.83:0/4283954343 learned_addr learned my addr 172.21.15.83:0/4283954343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52ce262640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fe4fa0 0x5628e10ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cda61640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 -- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 msgr2=0x5628e10e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 -- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fe4fa0 msgr2=0x5628e10ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fe4fa0 0x5628e10ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 -- 172.21.15.83:0/4283954343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e0ef96e0 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cd260640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f52b80126f0 tx=0x7f52b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfa63894150e418 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8023020 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f52b801f830 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52c4064100 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f52b802d050 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 10 client.6861 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x5628e0ee9690 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c4064100 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f52b8030020 con 0x5628e0fda600 2024-05-17T22:03:18.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 10 client.6861 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 10 client.6861 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 20 client.6861 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bd17d0 0x5628e1131920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 4 client.6861 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e0ec6a00 con 0x5628e0bd17d0 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cda61640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bd17d0 0x5628e1131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52cda61640 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bd17d0 0x5628e1131920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd01ca38ad02c09b server_cookie=83ac52993e167dd5 in_seq=0 out_seq=0 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 10 client.6861 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 1 -- 172.21.15.83:0/4283954343 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628e0ec6a00 con 0x5628e0bd17d0 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 10 client.6861 handle_command_reply: tid=0 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 client.6861 shutdown 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 msgr2=0x5628e10e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f52b80126f0 tx=0x7f52b8004740 comp rx=0 tx=0).stop 2024-05-17T22:03:18.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 shutdown_connections 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bd17d0 0x5628e1131920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fe4fa0 0x5628e10ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52a8ff9640 0 client.6861 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fda600 0x5628e10e77c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/4283954343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 >> 172.21.15.83:0/4283954343 conn(0x5628e0bec460 msgr2=0x5628e10dfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 shutdown_connections 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 1 -- 172.21.15.83:0/4283954343 wait complete. 2024-05-17T22:03:18.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.022+0000 7f52d6f3f000 20 client.6861 trim_cache size 0 max 16384 2024-05-17T22:03:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 Processor -- start 2024-05-17T22:03:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 -- start start 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e10e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8204620 con 0x5628e0fe4fa0 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52c814fac0 con 0x5628e0fda600 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52c8150110 con 0x5628e0fc3d30 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50788/0 (socket says 172.21.15.83:50788) 2024-05-17T22:03:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 -- 172.21.15.83:0/1264665155 learned_addr learned my addr 172.21.15.83:0/1264665155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52cd260640 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e10e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 -- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 msgr2=0x5628e10e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 -- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 msgr2=0x5628e10e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e10e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 -- 172.21.15.83:0/1264665155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e10e3e90 con 0x5628e0fe4fa0 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.042+0000 7f52ce262640 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f52c001d290 tx=0x7f52c0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6b6d2519e82d95 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52aaffd640 1 -- 172.21.15.83:0/1264665155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0009150 con 0x5628e0fe4fa0 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52aaffd640 1 -- 172.21.15.83:0/1264665155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00a5890 con 0x5628e0fe4fa0 2024-05-17T22:03:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52aaffd640 1 -- 172.21.15.83:0/1264665155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00a5c80 con 0x5628e0fe4fa0 2024-05-17T22:03:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- 172.21.15.83:0/1264665155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 msgr2=0x5628e10e6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f52c001d290 tx=0x7f52c0004ac0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- 172.21.15.83:0/1264665155 shutdown_connections 2024-05-17T22:03:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fe4fa0 0x5628e10e6770 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e10e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/1264665155 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fc3d30 0x5628e10e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- 172.21.15.83:0/1264665155 >> 172.21.15.83:0/1264665155 conn(0x5628e0bec460 msgr2=0x5628e1140ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- 172.21.15.83:0/1264665155 shutdown_connections 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- 172.21.15.83:0/1264665155 wait complete. 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- start start 2024-05-17T22:03:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 Processor -- start 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e119d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 0x5628e119def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628e0ed5900 con 0x5628e0fc3d30 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5628e0fb1820 con 0x5628e0fda600 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5628e0ef9c10 con 0x5628e0fe4fa0 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52d6f3f000 10 client.0 _ll_register_callbacks cb 0x7f52b800b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50802/0 (socket says 172.21.15.83:50802) 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 -- 172.21.15.83:0/2865026934 learned_addr learned my addr 172.21.15.83:0/2865026934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cd260640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e119d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52ce262640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 0x5628e119def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 -- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 msgr2=0x5628e119def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 0x5628e119def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 -- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 msgr2=0x5628e119d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e119d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cda61640 1 -- 172.21.15.83:0/2865026934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e0fb1cb0 con 0x5628e0fc3d30 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52cd260640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e119d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.046+0000 7f52ce262640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 0x5628e119def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cda61640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f52c4058bc0 tx=0x7f52c408d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f2bdc1fb478361 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c401f040 con 0x5628e0fc3d30 2024-05-17T22:03:18.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f52c405abb0 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52c00a5890 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f52c4064100 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 10 client.6867 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x5628e0eea4f0 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00a5890 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f52c40572c0 con 0x5628e0fc3d30 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 10 client.6867 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 10 client.6867 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 20 client.6867 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bbf830 0x5628e10e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 4 client.6867 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "status"}) -- 0x5628e0f3d060 con 0x5628e0bbf830 2024-05-17T22:03:18.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cd260640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bbf830 0x5628e10e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 10 client.6867 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cd260640 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bbf830 0x5628e10e18a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3134395f727529e0 server_cookie=c5f0dcc4f89bfb79 in_seq=0 out_seq=0 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 1 -- 172.21.15.83:0/2865026934 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5628e0f3d060 con 0x5628e0bbf830 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 10 client.6867 handle_command_reply: tid=0 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 client.6867 shutdown 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 msgr2=0x5628e119d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f52c4058bc0 tx=0x7f52c408d560 comp rx=0 tx=0).stop 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 shutdown_connections 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5628e0bbf830 0x5628e10e18a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5628e0fe4fa0 0x5628e119def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628e0fda600 0x5628e119d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52a8ff9640 0 client.6867 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:18.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 --2- 172.21.15.83:0/2865026934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628e0fc3d30 0x5628e119d470 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 >> 172.21.15.83:0/2865026934 conn(0x5628e0bec460 msgr2=0x5628e112a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 shutdown_connections 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 1 -- 172.21.15.83:0/2865026934 wait complete. 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52d6f3f000 20 client.6867 trim_cache size 0 max 16384 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6801, 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:03:18.044 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.5981061409999997, 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 65, 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 84, 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 84, 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.5415183130000001 2024-05-17T22:03:18.045 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 msgr2=0x7f52ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 0x7f52ac071940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f52c806e6d0 tx=0x7f52b8008040 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 msgr2=0x7f52c80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c820 tx=0x7f52c400cce0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 shutdown_connections 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52c80d2000 0x7f52c8156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f52ac06f430 0x7f52ac071940 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52c80d0290 0x7f52c815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 --2- 172.21.15.83:0/1651384469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52c80b4810 0x7f52c80cf7b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 >> 172.21.15.83:0/1651384469 conn(0x7f52c806e010 msgr2=0x7f52c80b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 shutdown_connections 2024-05-17T22:03:18.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.050+0000 7f52cfcc5640 1 -- 172.21.15.83:0/1651384469 wait complete. 2024-05-17T22:03:18.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:03:18.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 Processor -- start 2024-05-17T22:03:18.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 -- start start 2024-05-17T22:03:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21c809c0a0 0x7f21c8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 0x7f21c80722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c80d1950 con 0x7f21c80cf2e0 2024-05-17T22:03:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80d1670 con 0x7f21c809c0a0 2024-05-17T22:03:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cecce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21c8071280 con 0x7f21c809c570 2024-05-17T22:03:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cd26b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 0x7f21c80722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21c809c0a0 0x7f21c8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cd26b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 0x7f21c80722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:53896/0 (socket says 172.21.15.83:53896) 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21c809c0a0 0x7f21c8071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:51044/0 (socket says 172.21.15.83:51044) 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21cd26b640 1 -- 172.21.15.83:0/1628735886 learned_addr learned my addr 172.21.15.83:0/1628735886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 -- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 msgr2=0x7f21c80722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 --2- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 0x7f21c80722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 -- 172.21.15.83:0/1628735886 >> v1:172.21.15.155:6789/0 conn(0x7f21c80cf2e0 legacy=0x7f21c809b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.210+0000 7f21c7fff640 1 -- 172.21.15.83:0/1628735886 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c8071560 con 0x7f21c809c0a0 2024-05-17T22:03:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21c7fff640 1 --2- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c0a0 0x7f21c8071d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009d90 tx=0x7f21b001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8c5f8dd57a0a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21c77fe640 1 -- 172.21.15.83:0/1628735886 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b00b4070 con 0x7f21c809c0a0 2024-05-17T22:03:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21c77fe640 1 -- 172.21.15.83:0/1628735886 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b00af600 con 0x7f21c809c0a0 2024-05-17T22:03:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21c77fe640 1 -- 172.21.15.83:0/1628735886 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b00af9d0 con 0x7f21c809c0a0 2024-05-17T22:03:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 -- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c0a0 msgr2=0x7f21c8071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 --2- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c0a0 0x7f21c8071d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009d90 tx=0x7f21b001f150 comp rx=0 tx=0).stop 2024-05-17T22:03:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 -- 172.21.15.83:0/1628735886 shutdown_connections 2024-05-17T22:03:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 --2- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c809c570 0x7f21c80722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 --2- 172.21.15.83:0/1628735886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c0a0 0x7f21c8071d60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 -- 172.21.15.83:0/1628735886 >> 172.21.15.83:0/1628735886 conn(0x7f21c80b5120 msgr2=0x7f21c80b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 -- 172.21.15.83:0/1628735886 shutdown_connections 2024-05-17T22:03:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.214+0000 7f21cecce640 1 -- 172.21.15.83:0/1628735886 wait complete. 2024-05-17T22:03:18.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 Processor -- start 2024-05-17T22:03:18.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 -- start start 2024-05-17T22:03:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c570 0x7f21c814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c8160610 0x7f21c815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80d1380 con 0x7f21c80cf2e0 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21c80d1200 con 0x7f21c809c570 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cecce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21c80b8de0 con 0x7f21c8160610 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50814/0 (socket says 172.21.15.83:50814) 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21c7fff640 1 -- 172.21.15.83:0/3532238966 learned_addr learned my addr 172.21.15.83:0/3532238966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.218+0000 7f21cca6a640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c570 0x7f21c814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cd26b640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c8160610 0x7f21c815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 -- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c8160610 msgr2=0x7f21c815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c8160610 0x7f21c815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 -- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c570 msgr2=0x7f21c814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c570 0x7f21c814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 -- 172.21.15.83:0/3532238966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80c8f70 con 0x7f21c80cf2e0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c7fff640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f21b001cd80 tx=0x7f21b001bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b00b4040 con 0x7f21c80cf2e0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b00c1070 con 0x7f21c80cf2e0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c80c9250 con 0x7f21c80cf2e0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0009450 con 0x7f21c80cf2e0 2024-05-17T22:03:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c80c9810 con 0x7f21c80cf2e0 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c809b990 con 0x7f21c80cf2e0 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f21b00ba690 con 0x7f21c80cf2e0 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 0x7f21a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f21b012e5d0 con 0x7f21c80cf2e0 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cca6a640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 0x7f21a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.222+0000 7f21cca6a640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 0x7f21a8071910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001650 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.226+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f21b0018cb0 con 0x7f21c80cf2e0 2024-05-17T22:03:18.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.426+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f21c80c8d00 con 0x7f21c80cf2e0 2024-05-17T22:03:18.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.426+0000 7f21c57fa640 1 -- 172.21.15.83:0/3532238966 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7f21b00fb5a0 con 0x7f21c80cf2e0 2024-05-17T22:03:18.416 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 msgr2=0x7f21a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 0x7f21a8071910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001650 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 msgr2=0x7f21c80ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f21b001cd80 tx=0x7f21b001bef0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 shutdown_connections 2024-05-17T22:03:18.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21c8160610 0x7f21c815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21c80cf2e0 0x7f21c80ca010 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a806f400 0x7f21a8071910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 --2- 172.21.15.83:0/3532238966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21c809c570 0x7f21c814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 >> 172.21.15.83:0/3532238966 conn(0x7f21c80b5120 msgr2=0x7f21c80b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 shutdown_connections 2024-05-17T22:03:18.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18.430+0000 7f21cecce640 1 -- 172.21.15.83:0/3532238966 wait complete. 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:03:18.436 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:03:18.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:03:18.437 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:18.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.437 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:03:18.442 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:03:18.442 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.443 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:03:18.444 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:03:18.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.445 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:03:18.495 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:03:18.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.497 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:03:18.548 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:03:18.549 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:18.549 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:18.549 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:03:18.549 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:03:18.549 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:03:18.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:03:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:03:18.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.667 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:03:18.713 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:03:18.713 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:03:18.713 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:18.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:03:18.766 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:03:18.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:03:18.766 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:18.813 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:03:18.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:03:18.814 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:18.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:18.917 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:18.967 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:18.967 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:18.967 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:18.967 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:18.968 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:18.969 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:18.970 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:18.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:18.971 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:19.017 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:03:19.017 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:03:19.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:19.078 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:19.125 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:19.125 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:19.125 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:19.125 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:19.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:19.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:19.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:19.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:19.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:19.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:19.129 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:19.171 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:03:19.182+0000 7fe0e746a580 -1 init, newargv = 0x55735595a080 newargc=15 2024-05-17T22:03:19.171 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43967]: starting ceph client 2024-05-17T22:03:19.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43967]: starting fuse 2024-05-17T22:03:19.441 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:03:19.450+0000 7f5dfbecc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:03:20.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:03:20.293 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:03:20.346 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:03:20.346 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:03:20.346 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:03:20.347 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:20.348 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:03:20.349 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:20.349 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:03:20.349 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:20.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.350 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:03:20.398 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:03:20.398 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:03:20.398 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:03:20.399 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:03:20.400 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:03:20.522 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.43967.asok 2024-05-17T22:03:20.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:03:20.534 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43967.asok 2024-05-17T22:03:20.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43967.asok status 2024-05-17T22:03:20.716 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "43967", 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6876, 2024-05-17T22:03:20.717 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 6876 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1027432508 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:03:20.718 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1027432508 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.6876 192.168.0.1:0/1027432508", 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1027432508", 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 65, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 84, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:03:20.719 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:20.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.733 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:03:20.733 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:20.739 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:03:20.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:20.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:03:20.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.792 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-17T22:03:20.844 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:03:20.854+0000 7f5df7ec4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-17T22:03:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-17T22:03:20.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:03:20.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:20.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-17T22:03:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.058+0000 7fe418765640 1 Processor -- start 2024-05-17T22:03:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.058+0000 7fe418765640 1 -- start start 2024-05-17T22:03:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe418765640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe410159da0 0x7fe41015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe418765640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe410073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe418765640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4100737a0 con 0x7fe4100b38d0 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe418765640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe4100b0e90 con 0x7fe410159da0 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe418765640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100cf050 con 0x7fe410072e50 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe410159da0 0x7fe41015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe410159da0 0x7fe41015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50820/0 (socket says 172.21.15.83:50820) 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 -- 172.21.15.83:0/462762300 learned_addr learned my addr 172.21.15.83:0/462762300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe415d00640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe410073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 -- 172.21.15.83:0/462762300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 msgr2=0x7fe410073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe410073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 -- 172.21.15.83:0/462762300 >> v1:172.21.15.155:6789/0 conn(0x7fe4100b38d0 legacy=0x7fe41006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.062+0000 7fe416501640 1 -- 172.21.15.83:0/462762300 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4100cf2f0 con 0x7fe410159da0 2024-05-17T22:03:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe416501640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe410159da0 0x7fe41015cfc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009870 tx=0x7fe40c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b1227c3fdd288f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe4154ff640 1 -- 172.21.15.83:0/462762300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0a9400 con 0x7fe410159da0 2024-05-17T22:03:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe4154ff640 1 -- 172.21.15.83:0/462762300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c0a95a0 con 0x7fe410159da0 2024-05-17T22:03:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe4154ff640 1 -- 172.21.15.83:0/462762300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0a9970 con 0x7fe410159da0 2024-05-17T22:03:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- 172.21.15.83:0/462762300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe410159da0 msgr2=0x7fe41015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe410159da0 0x7fe41015cfc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009870 tx=0x7fe40c01c540 comp rx=0 tx=0).stop 2024-05-17T22:03:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- 172.21.15.83:0/462762300 shutdown_connections 2024-05-17T22:03:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe410073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 --2- 172.21.15.83:0/462762300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe410159da0 0x7fe41015cfc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- 172.21.15.83:0/462762300 >> 172.21.15.83:0/462762300 conn(0x7fe41006cd90 msgr2=0x7fe4100b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- 172.21.15.83:0/462762300 shutdown_connections 2024-05-17T22:03:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- 172.21.15.83:0/462762300 wait complete. 2024-05-17T22:03:21.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 Processor -- start 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.066+0000 7fe418765640 1 -- start start 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe41013dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410159da0 0x7fe410152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe410070520 con 0x7fe4100b38d0 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100cf870 con 0x7fe410159da0 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100b0810 con 0x7fe410072e50 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50836/0 (socket says 172.21.15.83:50836) 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 -- 172.21.15.83:0/2127655347 learned_addr learned my addr 172.21.15.83:0/2127655347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe416501640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe41013dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe416d02640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410159da0 0x7fe410152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 -- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 msgr2=0x7fe41013dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe41013dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 -- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410159da0 msgr2=0x7fe410152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410159da0 0x7fe410152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 -- 172.21.15.83:0/2127655347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41014e6f0 con 0x7fe4100b38d0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe416501640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe41013dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe415d00640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe40000d5d0 tx=0x7fe40000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 -- 172.21.15.83:0/2127655347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000125f0 con 0x7fe4100b38d0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 -- 172.21.15.83:0/2127655347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe400019070 con 0x7fe4100b38d0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe410151560 con 0x7fe4100b38d0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 -- 172.21.15.83:0/2127655347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000129b0 con 0x7fe4100b38d0 2024-05-17T22:03:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe410150a80 con 0x7fe4100b38d0 2024-05-17T22:03:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 -- 172.21.15.83:0/2127655347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe400012b50 con 0x7fe4100b38d0 2024-05-17T22:03:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 0x7fe3f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe406ffd640 1 -- 172.21.15.83:0/2127655347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fe40008f610 con 0x7fe4100b38d0 2024-05-17T22:03:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.070+0000 7fe416501640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 0x7fe3f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe416501640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 0x7fe3f4071910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0ae000 tx=0x7fe40c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 Processor -- start 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- start start 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068a4c10 0x5612069f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x5612069f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561206807300 con 0x561206896450 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5612067f6ea0 con 0x5612068826d0 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5612067faee0 con 0x5612068a4c10 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50840/0 (socket says 172.21.15.83:50840) 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 -- 172.21.15.83:0/1634181092 learned_addr learned my addr 172.21.15.83:0/1634181092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 -- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068a4c10 msgr2=0x5612069f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068a4c10 0x5612069f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 -- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 msgr2=0x5612069f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x5612069f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 -- 172.21.15.83:0/1634181092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612067d1e80 con 0x561206896450 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe415d00640 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe40008da50 tx=0x7fe40005b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdf32370f395700 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe3f3fff640 1 -- 172.21.15.83:0/1634181092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40005be00 con 0x561206896450 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe3f3fff640 1 -- 172.21.15.83:0/1634181092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4000601b0 con 0x561206896450 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe3f3fff640 1 -- 172.21.15.83:0/1634181092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400064bd0 con 0x561206896450 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- 172.21.15.83:0/1634181092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 msgr2=0x5612069f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe40008da50 tx=0x7fe40005b880 comp rx=0 tx=0).stop 2024-05-17T22:03:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- 172.21.15.83:0/1634181092 shutdown_connections 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x5612069f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069f5ab0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 --2- 172.21.15.83:0/1634181092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068a4c10 0x5612069f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- 172.21.15.83:0/1634181092 >> 172.21.15.83:0/1634181092 conn(0x5612064b9590 msgr2=0x5612069aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- 172.21.15.83:0/1634181092 shutdown_connections 2024-05-17T22:03:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.074+0000 7fe41f9df000 1 -- 172.21.15.83:0/1634181092 wait complete. 2024-05-17T22:03:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- start start 2024-05-17T22:03:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 Processor -- start 2024-05-17T22:03:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x5612069e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068a4c10 0x5612069e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612067bebf0 con 0x561206896450 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5612068b6a10 con 0x5612068a4c10 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5612067f3410 con 0x5612068826d0 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 10 client.0 _ll_register_callbacks cb 0x561206997a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50848/0 (socket says 172.21.15.83:50848) 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 -- 172.21.15.83:0/3747881540 learned_addr learned my addr 172.21.15.83:0/3747881540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe416501640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x5612069e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe416d02640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068a4c10 0x5612069e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 -- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 msgr2=0x5612069e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x5612069e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 -- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068a4c10 msgr2=0x5612069e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068a4c10 0x5612069e4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 -- 172.21.15.83:0/3747881540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612067ac8f0 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe415d00640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe40005bb00 tx=0x7fe400020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e0b5a991ebed2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40007b060 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4000601b0 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fe40005b060 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe400081050 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 10 client.6888 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x5612067b74e0 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4000601b0 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7fe400084020 con 0x561206896450 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 10 client.6888 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:21.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 10 client.6888 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 20 client.6888 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120649dac0 0x561206a06b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 4 client.6888 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561206792d70 con 0x56120649dac0 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.078+0000 7fe416501640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120649dac0 0x561206a06b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe3f17fa640 10 client.6888 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe416501640 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120649dac0 0x561206a06b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e2adfc5c924fde server_cookie=e038810f3a63b377 in_seq=0 out_seq=0 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe3f17fa640 1 -- 172.21.15.83:0/3747881540 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561206792d70 con 0x56120649dac0 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe3f17fa640 10 client.6888 handle_command_reply: tid=0 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 client.6888 shutdown 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 msgr2=0x5612069e4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe40005bb00 tx=0x7fe400020070 comp rx=0 tx=0).stop 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 shutdown_connections 2024-05-17T22:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120649dac0 0x561206a06b30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068a4c10 0x5612069e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561206896450 0x5612069e4450 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe3f17fa640 0 client.6888 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 --2- 172.21.15.83:0/3747881540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x5612069e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 >> 172.21.15.83:0/3747881540 conn(0x5612064b9590 msgr2=0x5612069ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 shutdown_connections 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 1 -- 172.21.15.83:0/3747881540 wait complete. 2024-05-17T22:03:21.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.082+0000 7fe41f9df000 20 client.6888 trim_cache size 0 max 16384 2024-05-17T22:03:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.098+0000 7fe41f9df000 1 Processor -- start 2024-05-17T22:03:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.098+0000 7fe41f9df000 1 -- start start 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x561206a68f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 0x561206a694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe410200890 con 0x5612068a4c10 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100b59c0 con 0x561206896450 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100b6010 con 0x5612068826d0 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50864/0 (socket says 172.21.15.83:50864) 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 -- 172.21.15.83:0/529220457 learned_addr learned my addr 172.21.15.83:0/529220457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe415d00640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 0x561206a694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416501640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x561206a68f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 -- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 msgr2=0x561206a68f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x561206a68f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 -- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 msgr2=0x561206a694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 0x561206a694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 -- 172.21.15.83:0/529220457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120687bbf0 con 0x5612068a4c10 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe415d00640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 0x561206a694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe416d02640 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe408001c00 tx=0x7fe4080020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9617daae17cea5cc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe3f37fe640 1 -- 172.21.15.83:0/529220457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408003b90 con 0x5612068a4c10 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe3f37fe640 1 -- 172.21.15.83:0/529220457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe408018070 con 0x5612068a4c10 2024-05-17T22:03:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe3f37fe640 1 -- 172.21.15.83:0/529220457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40800f850 con 0x5612068a4c10 2024-05-17T22:03:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- 172.21.15.83:0/529220457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 msgr2=0x561206a69a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe408001c00 tx=0x7fe4080020c0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- 172.21.15.83:0/529220457 shutdown_connections 2024-05-17T22:03:21.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a69a10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561206896450 0x561206a694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 --2- 172.21.15.83:0/529220457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612068826d0 0x561206a68f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- 172.21.15.83:0/529220457 >> 172.21.15.83:0/529220457 conn(0x5612064b9590 msgr2=0x5612069ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- 172.21.15.83:0/529220457 shutdown_connections 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- 172.21.15.83:0/529220457 wait complete. 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 -- start start 2024-05-17T22:03:21.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.102+0000 7fe41f9df000 1 Processor -- start 2024-05-17T22:03:21.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x561206a6d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561206896450 0x561206a6dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612067c4460 con 0x5612068a4c10 2024-05-17T22:03:21.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5612068060d0 con 0x5612068826d0 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5612067fa1f0 con 0x561206896450 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 10 client.0 _ll_register_callbacks cb 0x7fe40005b750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50876/0 (socket says 172.21.15.83:50876) 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 -- 172.21.15.83:0/2508437872 learned_addr learned my addr 172.21.15.83:0/2508437872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416501640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x561206a6d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 -- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561206896450 msgr2=0x561206a6dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe415d00640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561206896450 0x561206a6dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561206896450 0x561206a6dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 -- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 msgr2=0x561206a6d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x561206a6d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 -- 172.21.15.83:0/2508437872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612067b3df0 con 0x5612068a4c10 2024-05-17T22:03:21.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416d02640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x56120688c7c0 tx=0x7fe408001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de44328b25b2e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408010da0 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fe4080050d0 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe408018070 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe40802f050 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 10 client.6894 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x56120679e4d0 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe408018070 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7fe408032020 con 0x5612068a4c10 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 10 client.6894 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 10 client.6894 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 20 client.6894 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120648bb20 0x561206a29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 4 client.6894 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x561206809250 con 0x56120648bb20 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416501640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120648bb20 0x561206a29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 10 client.6894 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe416501640 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120648bb20 0x561206a29030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436a1c2d491877c1 server_cookie=e5b2d0bae6838328 in_seq=0 out_seq=0 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 1 -- 172.21.15.83:0/2508437872 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561206809250 con 0x56120648bb20 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe3f17fa640 10 client.6894 handle_command_reply: tid=0 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.106+0000 7fe41f9df000 1 client.6894 shutdown 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 msgr2=0x561206a28af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x56120688c7c0 tx=0x7fe408001260 comp rx=0 tx=0).stop 2024-05-17T22:03:21.100 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:03:21.110+0000 7f5df7ec4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0) n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x564d2e646100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-17T22:03:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 shutdown_connections 2024-05-17T22:03:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x56120648bb20 0x561206a29030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612068a4c10 0x561206a28af0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561206896450 0x561206a6dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe3f17fa640 0 client.6894 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 --2- 172.21.15.83:0/2508437872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612068826d0 0x561206a6d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 >> 172.21.15.83:0/2508437872 conn(0x5612064b9590 msgr2=0x5612069f48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 shutdown_connections 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 1 -- 172.21.15.83:0/2508437872 wait complete. 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe41f9df000 20 client.6894 trim_cache size 0 max 16384 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "411feeb5-1092-4f6a-a568-97476aa9d75c", 2024-05-17T22:03:21.102 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:03:21.103 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 msgr2=0x7fe3f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 0x7fe3f4071910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0ae000 tx=0x7fe40c008040 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 msgr2=0x7fe4101523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe40000d5d0 tx=0x7fe40000da90 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 shutdown_connections 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3f406f400 0x7fe3f4071910 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe410159da0 0x7fe410152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4100b38d0 0x7fe4101523e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 --2- 172.21.15.83:0/2127655347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe410072e50 0x7fe41013dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 >> 172.21.15.83:0/2127655347 conn(0x7fe41006cd90 msgr2=0x7fe410070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 shutdown_connections 2024-05-17T22:03:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.110+0000 7fe418765640 1 -- 172.21.15.83:0/2127655347 wait complete. 2024-05-17T22:03:21.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:21.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:03:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.270+0000 7f775fc19640 1 Processor -- start 2024-05-17T22:03:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.270+0000 7f775fc19640 1 -- start start 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775fc19640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580b3d00 0x7f77580d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775fc19640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7758143f20 0x7f7758072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775fc19640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77580737d0 con 0x7f77580733c0 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775fc19640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f77580b4110 con 0x7f7758143f20 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775fc19640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7758070770 con 0x7f77580b3d00 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7758143f20 0x7f7758072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7758143f20 0x7f7758072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50880/0 (socket says 172.21.15.83:50880) 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 -- 172.21.15.83:0/2784801505 learned_addr learned my addr 172.21.15.83:0/2784801505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d9b5640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580b3d00 0x7f77580d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 -- 172.21.15.83:0/2784801505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580b3d00 msgr2=0x7f77580d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580b3d00 0x7f77580d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 -- 172.21.15.83:0/2784801505 >> v1:172.21.15.83:6789/0 conn(0x7f77580733c0 legacy=0x7f775806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 -- 172.21.15.83:0/2784801505 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77580d10d0 con 0x7f7758143f20 2024-05-17T22:03:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775d1b4640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f7758072e80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d80b576e411bfe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775c9b3640 1 -- 172.21.15.83:0/2784801505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480ab650 con 0x7f7758143f20 2024-05-17T22:03:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775c9b3640 1 -- 172.21.15.83:0/2784801505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77480b0070 con 0x7f7758143f20 2024-05-17T22:03:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.274+0000 7f775c9b3640 1 -- 172.21.15.83:0/2784801505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480aba10 con 0x7f7758143f20 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- 172.21.15.83:0/2784801505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 msgr2=0x7f7758072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f7758072e80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774801c560 comp rx=0 tx=0).stop 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- 172.21.15.83:0/2784801505 shutdown_connections 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f7758072e80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 --2- 172.21.15.83:0/2784801505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580b3d00 0x7f77580d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- 172.21.15.83:0/2784801505 >> 172.21.15.83:0/2784801505 conn(0x7f775806cd90 msgr2=0x7f77580b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- 172.21.15.83:0/2784801505 shutdown_connections 2024-05-17T22:03:21.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- 172.21.15.83:0/2784801505 wait complete. 2024-05-17T22:03:21.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 Processor -- start 2024-05-17T22:03:21.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.278+0000 7f775fc19640 1 -- start start 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580733c0 0x7f775815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77580b3d00 0x7f775813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7758070c00 con 0x7f7758143f20 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7758144330 con 0x7f77580b3d00 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77580af830 con 0x7f77580733c0 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50886/0 (socket says 172.21.15.83:50886) 2024-05-17T22:03:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 -- 172.21.15.83:0/216162706 learned_addr learned my addr 172.21.15.83:0/216162706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775d9b5640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580733c0 0x7f775815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775d1b4640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77580b3d00 0x7f775813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 -- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580733c0 msgr2=0x7f775815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580733c0 0x7f775815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 -- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77580b3d00 msgr2=0x7f775813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77580b3d00 0x7f775813b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 -- 172.21.15.83:0/216162706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77580b54d0 con 0x7f7758143f20 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775e1b6640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f775000c730 tx=0x7f775000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 -- 172.21.15.83:0/216162706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750003b80 con 0x7f7758143f20 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 -- 172.21.15.83:0/216162706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750003d20 con 0x7f7758143f20 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77580b6570 con 0x7f7758143f20 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 -- 172.21.15.83:0/216162706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775000aa70 con 0x7f7758143f20 2024-05-17T22:03:21.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77580b5a90 con 0x7f7758143f20 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 -- 172.21.15.83:0/216162706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f775000ac10 con 0x7f7758143f20 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 0x7f773c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f774e7fc640 1 -- 172.21.15.83:0/216162706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f775008f240 con 0x7f7758143f20 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.282+0000 7f775d9b5640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 0x7f773c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 0x7f773c071910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f77540096f0 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 Processor -- start 2024-05-17T22:03:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- start start 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19be5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6c090 0x55dd19be58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd19a83a90 con 0x55dd19a7ef80 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd199f7120 con 0x55dd19a6c090 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dd199e6cc0 con 0x55dd19a6b880 2024-05-17T22:03:21.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50890/0 (socket says 172.21.15.83:50890) 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 -- 172.21.15.83:0/1540221507 learned_addr learned my addr 172.21.15.83:0/1540221507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d1b4640 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19be5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775e1b6640 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6c090 0x55dd19be58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 -- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 msgr2=0x55dd19be5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19be5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 -- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6c090 msgr2=0x55dd19be58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6c090 0x55dd19be58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 -- 172.21.15.83:0/1540221507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd199a3110 con 0x55dd19a7ef80 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f775d9b5640 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f775400e850 tx=0x7f7754011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7443dfa25b87d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f773b7fe640 1 -- 172.21.15.83:0/1540221507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754025070 con 0x55dd19a7ef80 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f773b7fe640 1 -- 172.21.15.83:0/1540221507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754015640 con 0x55dd19a7ef80 2024-05-17T22:03:21.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f773b7fe640 1 -- 172.21.15.83:0/1540221507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400f040 con 0x55dd19a7ef80 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- 172.21.15.83:0/1540221507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 msgr2=0x55dd19be4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f775400e850 tx=0x7f7754011000 comp rx=0 tx=0).stop 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- 172.21.15.83:0/1540221507 shutdown_connections 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6c090 0x55dd19be58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19be5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 --2- 172.21.15.83:0/1540221507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a7ef80 0x55dd19be4e20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- 172.21.15.83:0/1540221507 >> 172.21.15.83:0/1540221507 conn(0x55dd19aba010 msgr2=0x55dd19b9a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- 172.21.15.83:0/1540221507 shutdown_connections 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.286+0000 7f7766e93000 1 -- 172.21.15.83:0/1540221507 wait complete. 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- start start 2024-05-17T22:03:21.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 Processor -- start 2024-05-17T22:03:21.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6c090 0x55dd19ba1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a7ef80 0x55dd19ba6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd199a2ec0 con 0x55dd19a6b880 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55dd199ead00 con 0x55dd19a7ef80 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dd196b0680 con 0x55dd19a6c090 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 10 client.0 _ll_register_callbacks cb 0x55dd19a9e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50902/0 (socket says 172.21.15.83:50902) 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 -- 172.21.15.83:0/3253056973 learned_addr learned my addr 172.21.15.83:0/3253056973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775e1b6640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a7ef80 0x55dd19ba6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 -- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6c090 msgr2=0x55dd19ba1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6c090 0x55dd19ba1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 -- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a7ef80 msgr2=0x55dd19ba6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a7ef80 0x55dd19ba6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 -- 172.21.15.83:0/3253056973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd199a5890 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d9b5640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7754009120 tx=0x7f7754004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd21fb058f73c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754025040 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7754015640 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f7754029430 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7754036050 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 10 client.6909 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55dd199a7650 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754015640 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f7754039020 con 0x55dd19a6b880 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 10 client.6909 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 10 client.6909 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 20 client.6909 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd1968d930 0x55dd19bebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 4 client.6909 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd19982c40 con 0x55dd1968d930 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d1b4640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd1968d930 0x55dd19bebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f7738ff9640 10 client.6909 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.290+0000 7f775d1b4640 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd1968d930 0x55dd19bebe90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f582cb48e9376 server_cookie=bfa3ab1fa5e3a904 in_seq=0 out_seq=0 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7738ff9640 1 -- 172.21.15.83:0/3253056973 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd19982c40 con 0x55dd1968d930 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7738ff9640 10 client.6909 handle_command_reply: tid=0 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 client.6909 shutdown 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 msgr2=0x55dd19ba1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7754009120 tx=0x7f7754004740 comp rx=0 tx=0).stop 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 shutdown_connections 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd1968d930 0x55dd19bebe90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a7ef80 0x55dd19ba6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7738ff9640 0 client.6909 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6c090 0x55dd19ba1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 --2- 172.21.15.83:0/3253056973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6b880 0x55dd19ba1a30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 >> 172.21.15.83:0/3253056973 conn(0x55dd19aba010 msgr2=0x55dd19b9a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 shutdown_connections 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 1 -- 172.21.15.83:0/3253056973 wait complete. 2024-05-17T22:03:21.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.294+0000 7f7766e93000 20 client.6909 trim_cache size 0 max 16384 2024-05-17T22:03:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 Processor -- start 2024-05-17T22:03:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 -- start start 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19aba010 0x55dd19bfb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6b880 0x55dd19bfb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7758205940 con 0x55dd19a6c090 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77580af560 con 0x55dd19a6b880 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77580ae8c0 con 0x55dd19aba010 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50908/0 (socket says 172.21.15.83:50908) 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 -- 172.21.15.83:0/409873541 learned_addr learned my addr 172.21.15.83:0/409873541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775d1b4640 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6b880 0x55dd19bfb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 -- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19aba010 msgr2=0x55dd19bfb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19aba010 0x55dd19bfb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 -- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6b880 msgr2=0x55dd19bfb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.310+0000 7f775e1b6640 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6b880 0x55dd19bfb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775e1b6640 1 -- 172.21.15.83:0/409873541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd199868c0 con 0x55dd19a6c090 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775e1b6640 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f77500012f0 tx=0x7f7750001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5454f64d5a9a3801 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f773affd640 1 -- 172.21.15.83:0/409873541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775000ce40 con 0x55dd19a6c090 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f773affd640 1 -- 172.21.15.83:0/409873541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750064140 con 0x55dd19a6c090 2024-05-17T22:03:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f773affd640 1 -- 172.21.15.83:0/409873541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775005b4b0 con 0x55dd19a6c090 2024-05-17T22:03:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- 172.21.15.83:0/409873541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 msgr2=0x55dd19bfbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f77500012f0 tx=0x7f7750001410 comp rx=0 tx=0).stop 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- 172.21.15.83:0/409873541 shutdown_connections 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19bfbe90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19a6b880 0x55dd19bfb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- 172.21.15.83:0/409873541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19aba010 0x55dd19bfb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- 172.21.15.83:0/409873541 >> 172.21.15.83:0/409873541 conn(0x55dd19b9e400 msgr2=0x55dd19ba09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- 172.21.15.83:0/409873541 shutdown_connections 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- 172.21.15.83:0/409873541 wait complete. 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- start start 2024-05-17T22:03:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 Processor -- start 2024-05-17T22:03:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19c581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19aba010 0x55dd19b9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd1998e0f0 con 0x55dd19a6c090 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77580aef10 con 0x55dd19aba010 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55dd199adf10 con 0x55dd19a6b880 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f7766e93000 10 client.0 _ll_register_callbacks cb 0x7f7754023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50914/0 (socket says 172.21.15.83:50914) 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 -- 172.21.15.83:0/3066713274 learned_addr learned my addr 172.21.15.83:0/3066713274 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 -- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 msgr2=0x55dd19c581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19c581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 -- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19aba010 msgr2=0x55dd19b9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19aba010 0x55dd19b9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.314+0000 7f775d1b4640 1 -- 172.21.15.83:0/3066713274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd199a1fa0 con 0x55dd19a6c090 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775d1b4640 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae5c9a79ca00f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480c6020 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f77480bd5f0 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd199f9070 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f77480bd7f0 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 10 client.6915 fetch_fsmap learned FSMap version 65 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55dd196a1220 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55dd199f9070 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f77480ab450 con 0x55dd19a6c090 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 10 client.6915 fetch_fsmap finished waiting for FSMap version 65 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 10 client.6915 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 20 client.6915 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd196a6fe0 0x55dd19b9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 4 client.6915 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:03:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd199f2850 con 0x55dd196a6fe0 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775d9b5640 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd196a6fe0 0x55dd19b9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 10 client.6915 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775d9b5640 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd196a6fe0 0x55dd19b9c220 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154b899dcf41f92b server_cookie=d04200a0f969f7a6 in_seq=0 out_seq=0 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 1 -- 172.21.15.83:0/3066713274 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55dd199f2850 con 0x55dd196a6fe0 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 10 client.6915 handle_command_reply: tid=0 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 client.6915 shutdown 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 msgr2=0x55dd19c58730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7748009870 tx=0x7f774801ada0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 shutdown_connections 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55dd196a6fe0 0x55dd19b9c220 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd19aba010 0x55dd19b9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd19a6c090 0x55dd19c58730 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 --2- 172.21.15.83:0/3066713274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55dd19a6b880 0x55dd19c581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7738ff9640 0 client.6915 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 >> 172.21.15.83:0/3066713274 conn(0x55dd19b9e400 msgr2=0x55dd19be42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 shutdown_connections 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 1 -- 172.21.15.83:0/3066713274 wait complete. 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f7766e93000 20 client.6915 trim_cache size 0 max 16384 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:03:21.311 INFO:teuthology.orchestra.run.smithi083.stdout: "411feeb5-1092-4f6a-a568-97476aa9d75c": { 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir", 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "411feeb5-1092-4f6a-a568-97476aa9d75c", 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive" 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:21.312 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 msgr2=0x7f773c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 0x7f773c071910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f77540096f0 tx=0x7f7754008040 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 msgr2=0x7f775813b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f775000c730 tx=0x7f775000cbf0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 shutdown_connections 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7758143f20 0x7f775813b8e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f773c06f400 0x7f773c071910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77580b3d00 0x7f775813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 --2- 172.21.15.83:0/216162706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77580733c0 0x7f775815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 >> 172.21.15.83:0/216162706 conn(0x7f775806cd90 msgr2=0x7f775813d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 shutdown_connections 2024-05-17T22:03:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:21.318+0000 7f775fc19640 1 -- 172.21.15.83:0/216162706 wait complete. 2024-05-17T22:03:21.324 INFO:tasks.cephfs.filesystem:scrub status for tag:411feeb5-1092-4f6a-a568-97476aa9d75c - {'path': '/dir', 'tag': '411feeb5-1092-4f6a-a568-97476aa9d75c', 'options': 'recursive'} 2024-05-17T22:03:24.140 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:03:24.150+0000 7fb429809640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-17T22:03:30.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:03:30.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:03:51.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:51.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:03:51.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.501+0000 7fab69226640 1 Processor -- start 2024-05-17T22:03:51.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.501+0000 7fab69226640 1 -- start start 2024-05-17T22:03:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab69226640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 0x7fab6415be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab69226640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab69226640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6406f2d0 con 0x7fab6406eec0 2024-05-17T22:03:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab69226640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fab6415c3b0 con 0x7fab640b38d0 2024-05-17T22:03:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab69226640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab6406f5a0 con 0x7fab640b08a0 2024-05-17T22:03:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6359e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 0x7fab6415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab62d9d640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fab6406eec0 0x7fab641601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:49328/0 (socket says 172.21.15.83:49328) 2024-05-17T22:03:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6359e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:49560/0 (socket says 172.21.15.83:49560) 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab62d9d640 1 -- 172.21.15.83:0/76838868 learned_addr learned my addr 172.21.15.83:0/76838868 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6259c640 1 -- 172.21.15.83:0/76838868 >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 msgr2=0x7fab640b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6259c640 1 --2- 172.21.15.83:0/76838868 >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6259c640 1 -- 172.21.15.83:0/76838868 >> v1:172.21.15.155:6789/0 conn(0x7fab6406eec0 legacy=0x7fab641601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6259c640 1 -- 172.21.15.83:0/76838868 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6406f880 con 0x7fab640b08a0 2024-05-17T22:03:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.505+0000 7fab6359e640 1 --2- 172.21.15.83:0/76838868 >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:03:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab6259c640 1 --2- 172.21.15.83:0/76838868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 0x7fab6415be70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab540097c0 tx=0x7fab5401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d4d11a0e8868f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab61d9b640 1 -- 172.21.15.83:0/76838868 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540a7070 con 0x7fab640b08a0 2024-05-17T22:03:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab61d9b640 1 -- 172.21.15.83:0/76838868 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab540b0070 con 0x7fab640b08a0 2024-05-17T22:03:51.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab61d9b640 1 -- 172.21.15.83:0/76838868 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540ab890 con 0x7fab640b08a0 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- 172.21.15.83:0/76838868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 msgr2=0x7fab6415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 --2- 172.21.15.83:0/76838868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 0x7fab6415be70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab540097c0 tx=0x7fab5401c470 comp rx=0 tx=0).stop 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- 172.21.15.83:0/76838868 shutdown_connections 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 --2- 172.21.15.83:0/76838868 >> v2:172.21.15.155:3300/0 conn(0x7fab640b38d0 0x7fab640b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 --2- 172.21.15.83:0/76838868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b08a0 0x7fab6415be70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- 172.21.15.83:0/76838868 >> 172.21.15.83:0/76838868 conn(0x7fab6406cd90 msgr2=0x7fab640b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- 172.21.15.83:0/76838868 shutdown_connections 2024-05-17T22:03:51.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- 172.21.15.83:0/76838868 wait complete. 2024-05-17T22:03:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 Processor -- start 2024-05-17T22:03:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.509+0000 7fab69226640 1 -- start start 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 0x7fab64154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b38d0 0x7fab64159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab64073810 con 0x7fab640b08a0 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab64073690 con 0x7fab6406eec0 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab641608c0 con 0x7fab640b38d0 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55772/0 (socket says 172.21.15.83:55772) 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 -- 172.21.15.83:0/1082899233 learned_addr learned my addr 172.21.15.83:0/1082899233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 0x7fab64154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6359e640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b38d0 0x7fab64159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 -- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b38d0 msgr2=0x7fab64159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b38d0 0x7fab64159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 -- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 msgr2=0x7fab64154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 0x7fab64154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 -- 172.21.15.83:0/1082899233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab64151e50 con 0x7fab640b08a0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 0x7fab64154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab6259c640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fab5401a0a0 tx=0x7fab5401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab5b7fe640 1 -- 172.21.15.83:0/1082899233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54006e20 con 0x7fab640b08a0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab64152130 con 0x7fab640b08a0 2024-05-17T22:03:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6415dc20 con 0x7fab640b08a0 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab5b7fe640 1 -- 172.21.15.83:0/1082899233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab540ab880 con 0x7fab640b08a0 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.513+0000 7fab5b7fe640 1 -- 172.21.15.83:0/1082899233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540abc30 con 0x7fab640b08a0 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab5b7fe640 1 -- 172.21.15.83:0/1082899233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fab540c73a0 con 0x7fab640b08a0 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab5b7fe640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 0x7fab44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab5b7fe640 1 -- 172.21.15.83:0/1082899233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fab5412bc00 con 0x7fab640b08a0 2024-05-17T22:03:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 0x7fab44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 0x7fab44071940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 Processor -- start 2024-05-17T22:03:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 -- start start 2024-05-17T22:03:51.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d2df80 0x561ef0e93e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e94360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561ef0d32a90 con 0x561ef0d1b090 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ef0ca6120 con 0x561ef0d2df80 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ef0c95cc0 con 0x561ef0d1a880 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55782/0 (socket says 172.21.15.83:55782) 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 -- 172.21.15.83:0/370815329 learned_addr learned my addr 172.21.15.83:0/370815329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab62d9d640 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d2df80 0x561ef0e93e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6259c640 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e94360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 -- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 msgr2=0x561ef0e94360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e94360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 -- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d2df80 msgr2=0x561ef0e93e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d2df80 0x561ef0e93e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 -- 172.21.15.83:0/370815329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef0c52110 con 0x561ef0d1b090 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.517+0000 7fab6359e640 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0074f0 tx=0x7fab5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6667433074d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab58ff9640 1 -- 172.21.15.83:0/370815329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c019020 con 0x561ef0d1b090 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab58ff9640 1 -- 172.21.15.83:0/370815329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c015850 con 0x561ef0d1b090 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab58ff9640 1 -- 172.21.15.83:0/370815329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c015c00 con 0x561ef0d1b090 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- 172.21.15.83:0/370815329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 msgr2=0x561ef0e948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0074f0 tx=0x7fab5c009750 comp rx=0 tx=0).stop 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- 172.21.15.83:0/370815329 shutdown_connections 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e948a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e94360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- 172.21.15.83:0/370815329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d2df80 0x561ef0e93e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- 172.21.15.83:0/370815329 >> 172.21.15.83:0/370815329 conn(0x561ef0d69010 msgr2=0x561ef0e495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- 172.21.15.83:0/370815329 shutdown_connections 2024-05-17T22:03:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- 172.21.15.83:0/370815329 wait complete. 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- start start 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 Processor -- start 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1a880 0x561ef0e50a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1b090 0x561ef0e50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561ef0c51ec0 con 0x561ef0d2df80 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561ef0c99d00 con 0x561ef0d1a880 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ef095f680 con 0x561ef0d1b090 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab704a0000 10 client.0 _ll_register_callbacks cb 0x561ef0d4d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55790/0 (socket says 172.21.15.83:55790) 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 -- 172.21.15.83:0/1358694874 learned_addr learned my addr 172.21.15.83:0/1358694874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1a880 0x561ef0e50a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6259c640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1b090 0x561ef0e50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 -- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1b090 msgr2=0x561ef0e50fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1b090 0x561ef0e50fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 -- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1a880 msgr2=0x561ef0e50a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1a880 0x561ef0e50a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.521+0000 7fab6359e640 1 -- 172.21.15.83:0/1358694874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef0c54890 con 0x561ef0d2df80 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab6359e640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009790 tx=0x7fab5c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be59bd2571bdadc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c024070 con 0x561ef0d2df80 2024-05-17T22:03:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fab5c029b40 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c015850 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fab5c019040 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 10 client.6924 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x561ef0c56650 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c015850 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fab5c029d40 con 0x561ef0d2df80 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 10 client.6924 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 10 client.6924 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 20 client.6924 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef093c930 0x561ef0e9ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 4 client.6924 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef0c31c40 con 0x561ef093c930 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef093c930 0x561ef0e9ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 10 client.6924 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab62d9d640 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef093c930 0x561ef0e9ae90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dc59b3279c364c server_cookie=34686cc915db0628 in_seq=0 out_seq=0 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 1 -- 172.21.15.83:0/1358694874 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ef0c31c40 con 0x561ef093c930 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 10 client.6924 handle_command_reply: tid=0 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 client.6924 shutdown 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 msgr2=0x561ef0e55be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009790 tx=0x7fab5c004ac0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 shutdown_connections 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef093c930 0x561ef0e9ae90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d2df80 0x561ef0e55be0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1b090 0x561ef0e50fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 --2- 172.21.15.83:0/1358694874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1a880 0x561ef0e50a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab49ffb640 0 client.6924 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 >> 172.21.15.83:0/1358694874 conn(0x561ef0d69010 msgr2=0x561ef0e495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 shutdown_connections 2024-05-17T22:03:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 1 -- 172.21.15.83:0/1358694874 wait complete. 2024-05-17T22:03:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.525+0000 7fab704a0000 20 client.6924 trim_cache size 0 max 16384 2024-05-17T22:03:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 Processor -- start 2024-05-17T22:03:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- start start 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d69010 0x561ef0e4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e4ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab4000c9b0 con 0x561ef0d1b090 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab4000b990 con 0x561ef0d69010 2024-05-17T22:03:51.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fab4000ac50 con 0x561ef0d1a880 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55794/0 (socket says 172.21.15.83:55794) 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 -- 172.21.15.83:0/2744279563 learned_addr learned my addr 172.21.15.83:0/2744279563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 -- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 msgr2=0x561ef0e4ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e4ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 -- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d69010 msgr2=0x561ef0e4fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d69010 0x561ef0e4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 -- 172.21.15.83:0/2744279563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef0c358c0 con 0x561ef0d1b090 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab6359e640 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002240 tx=0x7fab5c020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d7f2561d71683c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab4bfff640 1 -- 172.21.15.83:0/2744279563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c036070 con 0x561ef0d1b090 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab4bfff640 1 -- 172.21.15.83:0/2744279563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c01e770 con 0x561ef0d1b090 2024-05-17T22:03:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab4bfff640 1 -- 172.21.15.83:0/2744279563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c024040 con 0x561ef0d1b090 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- 172.21.15.83:0/2744279563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 msgr2=0x561ef0e50760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fab5c002240 tx=0x7fab5c020070 comp rx=0 tx=0).stop 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- 172.21.15.83:0/2744279563 shutdown_connections 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d1b090 0x561ef0e50760 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0e4ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 --2- 172.21.15.83:0/2744279563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d69010 0x561ef0e4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- 172.21.15.83:0/2744279563 >> 172.21.15.83:0/2744279563 conn(0x561ef0e93a10 msgr2=0x561ef0e4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- 172.21.15.83:0/2744279563 shutdown_connections 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.545+0000 7fab704a0000 1 -- 172.21.15.83:0/2744279563 wait complete. 2024-05-17T22:03:51.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- start start 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 Processor -- start 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0f0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1b090 0x561ef0f0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561ef0c3d0f0 con 0x561ef0d69010 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab4000b2c0 con 0x561ef0d1b090 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x561ef0c9e330 con 0x561ef0d1a880 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 10 client.0 _ll_register_callbacks cb 0x7fab5c0073c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55798/0 (socket says 172.21.15.83:55798) 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 -- 172.21.15.83:0/2726405678 learned_addr learned my addr 172.21.15.83:0/2726405678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab62d9d640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0f0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6259c640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1b090 0x561ef0f0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 -- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 msgr2=0x561ef0f0b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0f0b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 -- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1b090 msgr2=0x561ef0f0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1b090 0x561ef0f0ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 -- 172.21.15.83:0/2726405678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef0c50fa0 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab6359e640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fab5c008010 tx=0x7fab5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321a8ef812d6f261 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c024070 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fab5c004020 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c01e770 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fab5c04a050 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 10 client.6930 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x561ef0950220 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c01e770 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fab5c04d020 con 0x561ef0d69010 2024-05-17T22:03:51.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 10 client.6930 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 10 client.6930 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 20 client.6930 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef0955fe0 0x561ef0ec7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 4 client.6930 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ef0ca8070 con 0x561ef0955fe0 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab62d9d640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef0955fe0 0x561ef0ec7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 10 client.6930 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab62d9d640 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef0955fe0 0x561ef0ec7320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0e8734cb63f539 server_cookie=419b74e003bfb2a5 in_seq=0 out_seq=0 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 1 -- 172.21.15.83:0/2726405678 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ef0ca8070 con 0x561ef0955fe0 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab49ffb640 10 client.6930 handle_command_reply: tid=0 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.549+0000 7fab704a0000 1 client.6930 shutdown 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 msgr2=0x561ef0ec6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fab5c008010 tx=0x7fab5c00b040 comp rx=0 tx=0).stop 2024-05-17T22:03:51.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 shutdown_connections 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x561ef0955fe0 0x561ef0ec7320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561ef0d69010 0x561ef0ec6de0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561ef0d1b090 0x561ef0f0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab49ffb640 0 client.6930 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 --2- 172.21.15.83:0/2726405678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x561ef0d1a880 0x561ef0f0b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 >> 172.21.15.83:0/2726405678 conn(0x561ef0e93a10 msgr2=0x561ef0e93290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 shutdown_connections 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 1 -- 172.21.15.83:0/2726405678 wait complete. 2024-05-17T22:03:51.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab704a0000 20 client.6930 trim_cache size 0 max 16384 2024-05-17T22:03:51.547 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:03:51.547 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:03:51.547 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:51.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 msgr2=0x7fab44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 0x7fab44071940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c009040 comp rx=0 tx=0).stop 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 msgr2=0x7fab6414d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fab5401a0a0 tx=0x7fab5401acb0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 shutdown_connections 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab640b38d0 0x7fab64159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab640b08a0 0x7fab6414d040 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fab4406f430 0x7fab44071940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 --2- 172.21.15.83:0/1082899233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab6406eec0 0x7fab64154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 >> 172.21.15.83:0/1082899233 conn(0x7fab6406cd90 msgr2=0x7fab640b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 shutdown_connections 2024-05-17T22:03:51.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.553+0000 7fab69226640 1 -- 172.21.15.83:0/1082899233 wait complete. 2024-05-17T22:03:51.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:03:51.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:03:51.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.713+0000 7fee951cd640 1 Processor -- start 2024-05-17T22:03:51.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.713+0000 7fee951cd640 1 -- start start 2024-05-17T22:03:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee951cd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 0x7fee90072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee951cd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee90073370 0x7fee9006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee951cd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee90073780 con 0x7fee900d1040 2024-05-17T22:03:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee951cd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fee9015e790 con 0x7fee90073370 2024-05-17T22:03:51.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee951cd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee9015ea40 con 0x7fee90144080 2024-05-17T22:03:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee90073370 0x7fee9006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8e59c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 0x7fee90072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fee90073370 0x7fee9006f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:49602/0 (socket says 172.21.15.83:49602) 2024-05-17T22:03:51.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 -- 172.21.15.83:0/1992716195 learned_addr learned my addr 172.21.15.83:0/1992716195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 -- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 msgr2=0x7fee90072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 0x7fee90072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 -- 172.21.15.83:0/1992716195 >> v1:172.21.15.155:6789/0 conn(0x7fee900d1040 legacy=0x7fee9015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8f59e640 1 -- 172.21.15.83:0/1992716195 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9015ed20 con 0x7fee90073370 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8dd9b640 1 -- 172.21.15.83:0/1992716195 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3741041548 0 0) 0x7fee80001120 con 0x7fee900d1040 2024-05-17T22:03:51.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.717+0000 7fee8e59c640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 0x7fee90072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:03:51.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee8f59e640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee9006f730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e50 tx=0x7fee8801ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92392d050692f3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee8dd9b640 1 -- 172.21.15.83:0/1992716195 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a58d0 con 0x7fee90073370 2024-05-17T22:03:51.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee8dd9b640 1 -- 172.21.15.83:0/1992716195 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee880a5a70 con 0x7fee90073370 2024-05-17T22:03:51.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee8dd9b640 1 -- 172.21.15.83:0/1992716195 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a5e40 con 0x7fee90073370 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 msgr2=0x7fee9006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee9006f730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e50 tx=0x7fee8801ca00 comp rx=0 tx=0).stop 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- 172.21.15.83:0/1992716195 shutdown_connections 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee9006f730 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 --2- 172.21.15.83:0/1992716195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee90144080 0x7fee90072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- 172.21.15.83:0/1992716195 >> 172.21.15.83:0/1992716195 conn(0x7fee9006cd90 msgr2=0x7fee900b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- 172.21.15.83:0/1992716195 shutdown_connections 2024-05-17T22:03:51.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- 172.21.15.83:0/1992716195 wait complete. 2024-05-17T22:03:51.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 Processor -- start 2024-05-17T22:03:51.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.721+0000 7fee951cd640 1 -- start start 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee900ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee900d1040 0x7fee900c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee90070480 con 0x7fee90144080 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee90144490 con 0x7fee90073370 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fee90070600 con 0x7fee900d1040 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55814/0 (socket says 172.21.15.83:55814) 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 -- 172.21.15.83:0/1199021262 learned_addr learned my addr 172.21.15.83:0/1199021262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8ed9d640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee900ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8e59c640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee900d1040 0x7fee900c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 -- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee900d1040 msgr2=0x7fee900c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee900d1040 0x7fee900c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 -- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 msgr2=0x7fee900ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee900ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 -- 172.21.15.83:0/1199021262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee900ad060 con 0x7fee90144080 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee8f59e640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e50 tx=0x7fee880197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a58d0 con 0x7fee90144080 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88007c30 con 0x7fee90144080 2024-05-17T22:03:51.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88009510 con 0x7fee90144080 2024-05-17T22:03:51.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee900ad340 con 0x7fee90144080 2024-05-17T22:03:51.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.725+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee900aed70 con 0x7fee90144080 2024-05-17T22:03:51.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9006f730 con 0x7fee90144080 2024-05-17T22:03:51.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fee880c0020 con 0x7fee90144080 2024-05-17T22:03:51.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee777fe640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 0x7fee7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:51.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fee88129f70 con 0x7fee90144080 2024-05-17T22:03:51.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee8ed9d640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 0x7fee7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:51.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.729+0000 7fee8ed9d640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 0x7fee7c071910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee80000f70 tx=0x7fee80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.733+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fee880c7020 con 0x7fee90144080 2024-05-17T22:03:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.921+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee90142b80 con 0x7fee90144080 2024-05-17T22:03:51.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee777fe640 1 -- 172.21.15.83:0/1199021262 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7fee88016620 con 0x7fee90144080 2024-05-17T22:03:51.913 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:51.913 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:23.463499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":6}]} 2024-05-17T22:03:51.913 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 66 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 msgr2=0x7fee7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 0x7fee7c071910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee80000f70 tx=0x7fee80009040 comp rx=0 tx=0).stop 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 msgr2=0x7fee900af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fee88009e50 tx=0x7fee880197a0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 shutdown_connections 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee90144080 0x7fee900af290 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fee900d1040 0x7fee900c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fee7c06f400 0x7fee7c071910 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 --2- 172.21.15.83:0/1199021262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073370 0x7fee900ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:51.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 >> 172.21.15.83:0/1199021262 conn(0x7fee9006cd90 msgr2=0x7fee90141580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:51.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 shutdown_connections 2024-05-17T22:03:51.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:51.925+0000 7fee951cd640 1 -- 172.21.15.83:0/1199021262 wait complete. 2024-05-17T22:03:51.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-17T22:03:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 Processor -- start 2024-05-17T22:03:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 -- start start 2024-05-17T22:03:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84600b3d00 0x7f84600d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8460143f20 0x7f8460072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84600737d0 con 0x7f84600733c0 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f84600b4110 con 0x7f8460143f20 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8466b96640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8460070770 con 0x7f84600b3d00 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f845ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8460143f20 0x7f8460072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8465133640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f84600733c0 0x7f846006eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34628/0 (socket says 172.21.15.83:34628) 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8465133640 1 -- 172.21.15.83:0/1456521702 learned_addr learned my addr 172.21.15.83:0/1456521702 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f8464932640 1 --2- 172.21.15.83:0/1456521702 >> v2:172.21.15.155:3300/0 conn(0x7f84600b3d00 0x7f84600d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f845ffff640 1 -- 172.21.15.83:0/1456521702 >> v2:172.21.15.155:3300/0 conn(0x7f84600b3d00 msgr2=0x7f84600d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f845ffff640 1 --2- 172.21.15.83:0/1456521702 >> v2:172.21.15.155:3300/0 conn(0x7f84600b3d00 0x7f84600d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f845ffff640 1 -- 172.21.15.83:0/1456521702 >> v1:172.21.15.83:6789/0 conn(0x7f84600733c0 legacy=0x7f846006eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.085+0000 7f845ffff640 1 -- 172.21.15.83:0/1456521702 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84600d10d0 con 0x7f8460143f20 2024-05-17T22:03:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f845ffff640 1 --2- 172.21.15.83:0/1456521702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8460143f20 0x7f8460072e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f846006d6e0 tx=0x7f844801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93d876c681865a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f845f7fe640 1 -- 172.21.15.83:0/1456521702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480b0070 con 0x7f8460143f20 2024-05-17T22:03:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f845f7fe640 1 -- 172.21.15.83:0/1456521702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84480ab600 con 0x7f8460143f20 2024-05-17T22:03:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f845f7fe640 1 -- 172.21.15.83:0/1456521702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480ab9d0 con 0x7f8460143f20 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 -- 172.21.15.83:0/1456521702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8460143f20 msgr2=0x7f8460072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 --2- 172.21.15.83:0/1456521702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8460143f20 0x7f8460072e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f846006d6e0 tx=0x7f844801c3f0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 -- 172.21.15.83:0/1456521702 shutdown_connections 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 --2- 172.21.15.83:0/1456521702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8460143f20 0x7f8460072e80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 --2- 172.21.15.83:0/1456521702 >> v2:172.21.15.155:3300/0 conn(0x7f84600b3d00 0x7f84600d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 -- 172.21.15.83:0/1456521702 >> 172.21.15.83:0/1456521702 conn(0x7f846006cd90 msgr2=0x7f84600b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 -- 172.21.15.83:0/1456521702 shutdown_connections 2024-05-17T22:03:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.089+0000 7f8466b96640 1 -- 172.21.15.83:0/1456521702 wait complete. 2024-05-17T22:03:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 Processor -- start 2024-05-17T22:03:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- start start 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84600733c0 0x7f846015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8460143f20 0x7f8460144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8460070c00 con 0x7f84600b3d00 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8460144330 con 0x7f8460143f20 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f846013a060 con 0x7f84600733c0 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55824/0 (socket says 172.21.15.83:55824) 2024-05-17T22:03:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 -- 172.21.15.83:0/1266346071 learned_addr learned my addr 172.21.15.83:0/1266346071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8465133640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8460143f20 0x7f8460144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8464932640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84600733c0 0x7f846015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 -- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84600733c0 msgr2=0x7f846015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84600733c0 0x7f846015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 -- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8460143f20 msgr2=0x7f8460144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8460143f20 0x7f8460144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 -- 172.21.15.83:0/1266346071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8460137a70 con 0x7f84600b3d00 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845ffff640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f84480ae040 tx=0x7f8448019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845d7fa640 1 -- 172.21.15.83:0/1266346071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480ba070 con 0x7f84600b3d00 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845d7fa640 1 -- 172.21.15.83:0/1266346071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84480b43f0 con 0x7f84600b3d00 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8460137d50 con 0x7f84600b3d00 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845d7fa640 1 -- 172.21.15.83:0/1266346071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480b0040 con 0x7f84600b3d00 2024-05-17T22:03:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8460138310 con 0x7f84600b3d00 2024-05-17T22:03:52.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.093+0000 7f845d7fa640 1 -- 172.21.15.83:0/1266346071 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f84480bd070 con 0x7f84600b3d00 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f845d7fa640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 0x7f8440071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f845d7fa640 1 -- 172.21.15.83:0/1266346071 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f84481282e0 con 0x7f84600b3d00 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 0x7f8440071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 0x7f8440071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f84580096f0 tx=0x7f8458008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 Processor -- start 2024-05-17T22:03:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 -- start start 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e8620560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84ba030 0x5584e8620aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5584e84befa0 con 0x5584e84dae00 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5584e84332e0 con 0x5584e84ba030 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5584e8422e80 con 0x5584e84d0460 2024-05-17T22:03:52.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55838/0 (socket says 172.21.15.83:55838) 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 -- 172.21.15.83:0/1599357006 learned_addr learned my addr 172.21.15.83:0/1599357006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 -- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 msgr2=0x5584e8620560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f845ffff640 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e8620560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e8620560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 -- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84ba030 msgr2=0x5584e8620aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8465133640 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84ba030 0x5584e8620aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84ba030 0x5584e8620aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 -- 172.21.15.83:0/1599357006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e84cdb80 con 0x5584e84dae00 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f8464932640 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f845800e850 tx=0x7f8458011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4f9fbce3c2a3c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f84567fc640 1 -- 172.21.15.83:0/1599357006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458025070 con 0x5584e84dae00 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f84567fc640 1 -- 172.21.15.83:0/1599357006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8458015640 con 0x5584e84dae00 2024-05-17T22:03:52.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.097+0000 7f84567fc640 1 -- 172.21.15.83:0/1599357006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845800f040 con 0x5584e84dae00 2024-05-17T22:03:52.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1599357006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 msgr2=0x5584e8620020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f845800e850 tx=0x7f8458011000 comp rx=0 tx=0).stop 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1599357006 shutdown_connections 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84ba030 0x5584e8620aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e8620560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- 172.21.15.83:0/1599357006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84dae00 0x5584e8620020 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1599357006 >> 172.21.15.83:0/1599357006 conn(0x5584e80e4760 msgr2=0x5584e85dd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1599357006 shutdown_connections 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1599357006 wait complete. 2024-05-17T22:03:52.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- start start 2024-05-17T22:03:52.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 Processor -- start 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 0x5584e86277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84dae00 0x5584e8627d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5584e83deeb0 con 0x5584e84ba030 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5584e8426ec0 con 0x5584e84d0460 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5584e80ec820 con 0x5584e84dae00 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 10 client.0 _ll_register_callbacks cb 0x5584e85c2d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55848/0 (socket says 172.21.15.83:55848) 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 -- 172.21.15.83:0/1147349914 learned_addr learned my addr 172.21.15.83:0/1147349914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f845ffff640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 0x5584e86277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8465133640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84dae00 0x5584e8627d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 -- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84dae00 msgr2=0x5584e8627d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84dae00 0x5584e8627d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 -- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 msgr2=0x5584e86277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 0x5584e86277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 -- 172.21.15.83:0/1147349914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e846bef0 con 0x5584e84ba030 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f845ffff640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 0x5584e86277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f8464932640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8458018040 tx=0x7f8458004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7ccec683e8a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f843f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458030060 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8458015640 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f8458029430 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f8458036050 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 10 client.6948 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.101+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5584e8460910 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8458015640 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f8458039020 con 0x5584e84ba030 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 10 client.6948 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 10 client.6948 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 20 client.6948 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80c9ad0 0x5584e8631680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 4 client.6948 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:52.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584e83becf0 con 0x5584e80c9ad0 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f845ffff640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80c9ad0 0x5584e8631680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 10 client.6948 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f845ffff640 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80c9ad0 0x5584e8631680 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1922a90602486a server_cookie=4e6ea8124bff63e6 in_seq=0 out_seq=0 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 1 -- 172.21.15.83:0/1147349914 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584e83becf0 con 0x5584e80c9ad0 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 10 client.6948 handle_command_reply: tid=0 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 client.6948 shutdown 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 msgr2=0x5584e86272a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8458018040 tx=0x7f8458004740 comp rx=0 tx=0).stop 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 shutdown_connections 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80c9ad0 0x5584e8631680 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84dae00 0x5584e8627d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84d0460 0x5584e86277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 --2- 172.21.15.83:0/1147349914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e86272a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 >> 172.21.15.83:0/1147349914 conn(0x5584e80e4760 msgr2=0x5584e85d54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f843f7fe640 0 client.6948 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 shutdown_connections 2024-05-17T22:03:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 1 -- 172.21.15.83:0/1147349914 wait complete. 2024-05-17T22:03:52.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.105+0000 7f846de10000 20 client.6948 trim_cache size 0 max 16384 2024-05-17T22:03:52.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 Processor -- start 2024-05-17T22:03:52.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 -- start start 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e868ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e868f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8460203720 con 0x5584e84ba030 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8460202780 con 0x5584e84dae00 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8460201c40 con 0x5584e84d0460 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f8464932640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55860/0 (socket says 172.21.15.83:55860) 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.117+0000 7f8464932640 1 -- 172.21.15.83:0/2467837548 learned_addr learned my addr 172.21.15.83:0/2467837548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 -- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 msgr2=0x5584e868ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e868ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 -- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 msgr2=0x5584e868f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e868f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 -- 172.21.15.83:0/2467837548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e83e0670 con 0x5584e84ba030 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8464932640 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8458008010 tx=0x7f8458027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f7e471d58eb410 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8455ffb640 1 -- 172.21.15.83:0/2467837548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458025040 con 0x5584e84ba030 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8455ffb640 1 -- 172.21.15.83:0/2467837548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8458015440 con 0x5584e84ba030 2024-05-17T22:03:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f8455ffb640 1 -- 172.21.15.83:0/2467837548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458029a50 con 0x5584e84ba030 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- 172.21.15.83:0/2467837548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 msgr2=0x5584e868e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8458008010 tx=0x7f8458027000 comp rx=0 tx=0).stop 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- 172.21.15.83:0/2467837548 shutdown_connections 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e868f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84d0460 0x5584e868ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- 172.21.15.83:0/2467837548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84ba030 0x5584e868e7c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- 172.21.15.83:0/2467837548 >> 172.21.15.83:0/2467837548 conn(0x5584e80e4760 msgr2=0x5584e85d5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- 172.21.15.83:0/2467837548 shutdown_connections 2024-05-17T22:03:52.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- 172.21.15.83:0/2467837548 wait complete. 2024-05-17T22:03:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- start start 2024-05-17T22:03:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 Processor -- start 2024-05-17T22:03:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84ba030 0x5584e86926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e8693160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5584e84a78f0 con 0x5584e84d0460 2024-05-17T22:03:52.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8460202130 con 0x5584e84dae00 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5584e83be850 con 0x5584e84ba030 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f846de10000 10 client.0 _ll_register_callbacks cb 0x7f8458023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f845ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f845ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55868/0 (socket says 172.21.15.83:55868) 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.121+0000 7f845ffff640 1 -- 172.21.15.83:0/151032815 learned_addr learned my addr 172.21.15.83:0/151032815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 -- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84ba030 msgr2=0x5584e86926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84ba030 0x5584e86926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 -- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 msgr2=0x5584e8693160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e8693160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 -- 172.21.15.83:0/151032815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e83df100 con 0x5584e84d0460 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f845ffff640 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f84480f1cb0 tx=0x7f84480f4930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f085161ba0cc3f4c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480fd040 con 0x5584e84d0460 2024-05-17T22:03:52.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f84480fd410 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8458015440 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f84480f5140 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 10 client.6954 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5584e84a7d80 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8458015440 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f84480b3070 con 0x5584e84d0460 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 10 client.6954 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 10 client.6954 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 20 client.6954 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80b7b30 0x5584e864f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 4 client.6954 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5584e8435230 con 0x5584e80b7b30 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8464932640 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80b7b30 0x5584e864f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 10 client.6954 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8464932640 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80b7b30 0x5584e864f0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcc36633ce027ac server_cookie=f60dc1faaa415215 in_seq=0 out_seq=0 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 1 -- 172.21.15.83:0/151032815 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+417 (crc 0 0 0) 0x5584e8435230 con 0x5584e80b7b30 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 10 client.6954 handle_command_reply: tid=0 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 client.6954 shutdown 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 msgr2=0x5584e8692c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f84480f1cb0 tx=0x7f84480f4930 comp rx=0 tx=0).stop 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 shutdown_connections 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x5584e80b7b30 0x5584e864f0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5584e84dae00 0x5584e8693160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584e84d0460 0x5584e8692c20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f843f7fe640 0 client.6954 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 --2- 172.21.15.83:0/151032815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5584e84ba030 0x5584e86926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 >> 172.21.15.83:0/151032815 conn(0x5584e80e4760 msgr2=0x5584e85d62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 shutdown_connections 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 1 -- 172.21.15.83:0/151032815 wait complete. 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f846de10000 20 client.6954 trim_cache size 0 max 16384 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "dir_frag", 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 51510583, 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627776, 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "frag": "*", 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir" 2024-05-17T22:03:52.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 612725979, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/testdir/hardlink" 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 2463279525, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627776, 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir" 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:52.120 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 msgr2=0x7f8440071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 0x7f8440071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f84580096f0 tx=0x7f8458008040 comp rx=0 tx=0).stop 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 msgr2=0x7f8460144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f84480ae040 tx=0x7f8448019190 comp rx=0 tx=0).stop 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 shutdown_connections 2024-05-17T22:03:52.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f844006f400 0x7f8440071910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8460143f20 0x7f8460144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84600b3d00 0x7f8460144540 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 --2- 172.21.15.83:0/1266346071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84600733c0 0x7f846015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 >> 172.21.15.83:0/1266346071 conn(0x7f846006cd90 msgr2=0x7f8460140600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 shutdown_connections 2024-05-17T22:03:52.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.125+0000 7f8466b96640 1 -- 172.21.15.83:0/1266346071 wait complete. 2024-05-17T22:03:52.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:03:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.281+0000 7faed74ac640 1 Processor -- start 2024-05-17T22:03:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.281+0000 7faed74ac640 1 -- start start 2024-05-17T22:03:52.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed74ac640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faed00d1040 0x7faed0144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed74ac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed0073540 0x7faed006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed74ac640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faed015ea40 con 0x7faed015e080 2024-05-17T22:03:52.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed74ac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faed015ed20 con 0x7faed0073540 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed74ac640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7faed015f000 con 0x7faed00d1040 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed0073540 0x7faed006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faed0073540 0x7faed006f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55882/0 (socket says 172.21.15.83:55882) 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 -- 172.21.15.83:0/2951926999 learned_addr learned my addr 172.21.15.83:0/2951926999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5248640 1 --2- 172.21.15.83:0/2951926999 >> v2:172.21.15.155:3300/0 conn(0x7faed00d1040 0x7faed0144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 -- 172.21.15.83:0/2951926999 >> v2:172.21.15.155:3300/0 conn(0x7faed00d1040 msgr2=0x7faed0144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 --2- 172.21.15.83:0/2951926999 >> v2:172.21.15.155:3300/0 conn(0x7faed00d1040 0x7faed0144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 -- 172.21.15.83:0/2951926999 >> v1:172.21.15.155:6789/0 conn(0x7faed015e080 legacy=0x7faed0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.285+0000 7faed5a49640 1 -- 172.21.15.83:0/2951926999 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed006ef30 con 0x7faed0073540 2024-05-17T22:03:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed5a49640 1 --2- 172.21.15.83:0/2951926999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0073540 0x7faed006f730 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faed0071d60 tx=0x7faecc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4783891a7b2f811d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faec7fff640 1 -- 172.21.15.83:0/2951926999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab5f0 con 0x7faed0073540 2024-05-17T22:03:52.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faec7fff640 1 -- 172.21.15.83:0/2951926999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0b0070 con 0x7faed0073540 2024-05-17T22:03:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faec7fff640 1 -- 172.21.15.83:0/2951926999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab9b0 con 0x7faed0073540 2024-05-17T22:03:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- 172.21.15.83:0/2951926999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0073540 msgr2=0x7faed006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 --2- 172.21.15.83:0/2951926999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0073540 0x7faed006f730 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faed0071d60 tx=0x7faecc01c520 comp rx=0 tx=0).stop 2024-05-17T22:03:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- 172.21.15.83:0/2951926999 shutdown_connections 2024-05-17T22:03:52.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 --2- 172.21.15.83:0/2951926999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0073540 0x7faed006f730 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 --2- 172.21.15.83:0/2951926999 >> v2:172.21.15.155:3300/0 conn(0x7faed00d1040 0x7faed0144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- 172.21.15.83:0/2951926999 >> 172.21.15.83:0/2951926999 conn(0x7faed006cd90 msgr2=0x7faed00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- 172.21.15.83:0/2951926999 shutdown_connections 2024-05-17T22:03:52.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- 172.21.15.83:0/2951926999 wait complete. 2024-05-17T22:03:52.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 Processor -- start 2024-05-17T22:03:52.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.289+0000 7faed74ac640 1 -- start start 2024-05-17T22:03:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faed00d1040 0x7faed00aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faed015e080 0x7faed015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faed0070810 con 0x7faed0140860 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faed0070690 con 0x7faed00d1040 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faed0070990 con 0x7faed015e080 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55896/0 (socket says 172.21.15.83:55896) 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 -- 172.21.15.83:0/4050960051 learned_addr learned my addr 172.21.15.83:0/4050960051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5248640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faed00d1040 0x7faed00aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 -- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faed015e080 msgr2=0x7faed015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faed015e080 0x7faed015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 -- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faed00d1040 msgr2=0x7faed00aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faed00d1040 0x7faed00aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 -- 172.21.15.83:0/4050960051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed013f4d0 con 0x7faed0140860 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5a49640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ad040 tx=0x7faecc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b4800 con 0x7faed0140860 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0b49c0 con 0x7faed0140860 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b4db0 con 0x7faed0140860 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed013e710 con 0x7faed0140860 2024-05-17T22:03:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed013ecd0 con 0x7faed0140860 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed0140d60 con 0x7faed0140860 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faecc0be3f0 con 0x7faed0140860 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 0x7faeb4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7faecc1297f0 con 0x7faed0140860 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5248640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 0x7faeb4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.293+0000 7faed5248640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 0x7faeb4071940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e90 tx=0x7faec800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.297+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7faecc0c4060 con 0x7faed0140860 2024-05-17T22:03:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faed0142050 con 0x7faed0140860 2024-05-17T22:03:52.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faec5ffb640 1 -- 172.21.15.83:0/4050960051 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7faecc016110 con 0x7faed0140860 2024-05-17T22:03:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:03:52.470 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 msgr2=0x7faeb4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 0x7faeb4071940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e90 tx=0x7faec800b040 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 msgr2=0x7faed00aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ad040 tx=0x7faecc01ad60 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 shutdown_connections 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed0140860 0x7faed00aa310 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faed015e080 0x7faed015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faeb406f430 0x7faeb4071940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 --2- 172.21.15.83:0/4050960051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faed00d1040 0x7faed00aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 >> 172.21.15.83:0/4050960051 conn(0x7faed006cd90 msgr2=0x7faed01446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 shutdown_connections 2024-05-17T22:03:52.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.481+0000 7faed74ac640 1 -- 172.21.15.83:0/4050960051 wait complete. 2024-05-17T22:03:52.489 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-17T22:03:52.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:52.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-17T22:03:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 Processor -- start 2024-05-17T22:03:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 -- start start 2024-05-17T22:03:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f41570e0 0x7fa7f40b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7f40b4f20 0x7fa7f40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f40d0820 con 0x7fa7f4160420 2024-05-17T22:03:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f40b4a90 con 0x7fa7f40b4f20 2024-05-17T22:03:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7fac69640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f40d0b00 con 0x7fa7f41570e0 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f9206640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7f40b4f20 0x7fa7f40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f9206640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa7f40b4f20 0x7fa7f40b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55906/0 (socket says 172.21.15.83:55906) 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f9206640 1 -- 172.21.15.83:0/1424203495 learned_addr learned my addr 172.21.15.83:0/1424203495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f41570e0 0x7fa7f40b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f9206640 1 -- 172.21.15.83:0/1424203495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f41570e0 msgr2=0x7fa7f40b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.649+0000 7fa7f9206640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f41570e0 0x7fa7f40b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f9206640 1 -- 172.21.15.83:0/1424203495 >> v1:172.21.15.155:6789/0 conn(0x7fa7f4160420 legacy=0x7fa7f4160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f9206640 1 -- 172.21.15.83:0/1424203495 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f40d0de0 con 0x7fa7f40b4f20 2024-05-17T22:03:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f9206640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f40b4f20 0x7fa7f40b5330 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009e50 tx=0x7fa7e401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80542e952c2e053f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f37fe640 1 -- 172.21.15.83:0/1424203495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5760 con 0x7fa7f40b4f20 2024-05-17T22:03:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f37fe640 1 -- 172.21.15.83:0/1424203495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e40a5900 con 0x7fa7f40b4f20 2024-05-17T22:03:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7f37fe640 1 -- 172.21.15.83:0/1424203495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5cd0 con 0x7fa7f40b4f20 2024-05-17T22:03:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 -- 172.21.15.83:0/1424203495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f40b4f20 msgr2=0x7fa7f40b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f40b4f20 0x7fa7f40b5330 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009e50 tx=0x7fa7e401ca00 comp rx=0 tx=0).stop 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 -- 172.21.15.83:0/1424203495 shutdown_connections 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f40b4f20 0x7fa7f40b5330 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 --2- 172.21.15.83:0/1424203495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f41570e0 0x7fa7f40b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 -- 172.21.15.83:0/1424203495 >> 172.21.15.83:0/1424203495 conn(0x7fa7f406cd90 msgr2=0x7fa7f40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 -- 172.21.15.83:0/1424203495 shutdown_connections 2024-05-17T22:03:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.653+0000 7fa7fac69640 1 -- 172.21.15.83:0/1424203495 wait complete. 2024-05-17T22:03:52.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 Processor -- start 2024-05-17T22:03:52.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- start start 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7f40b4f20 0x7fa7f415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f4160420 0x7fa7f4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41695b0 con 0x7fa7f41570e0 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f40d05c0 con 0x7fa7f40b4f20 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f4169730 con 0x7fa7f4160420 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55920/0 (socket says 172.21.15.83:55920) 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 -- 172.21.15.83:0/2893310603 learned_addr learned my addr 172.21.15.83:0/2893310603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7f40b4f20 0x7fa7f415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f9206640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f4160420 0x7fa7f4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 -- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f4160420 msgr2=0x7fa7f4153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f4160420 0x7fa7f4153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 -- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7f40b4f20 msgr2=0x7fa7f415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7f40b4f20 0x7fa7f415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 -- 172.21.15.83:0/2893310603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f415a7d0 con 0x7fa7f41570e0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b440 tx=0x7fa7dc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f17fa640 1 -- 172.21.15.83:0/2893310603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc015020 con 0x7fa7f41570e0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f17fa640 1 -- 172.21.15.83:0/2893310603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc011890 con 0x7fa7f41570e0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f41fcb60 con 0x7fa7f41570e0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7f17fa640 1 -- 172.21.15.83:0/2893310603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc011c80 con 0x7fa7f41570e0 2024-05-17T22:03:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.657+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41fd350 con 0x7fa7f41570e0 2024-05-17T22:03:52.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f17fa640 1 -- 172.21.15.83:0/2893310603 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa7dc018020 con 0x7fa7f41570e0 2024-05-17T22:03:52.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f17fa640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 0x7fa7d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f17fa640 1 -- 172.21.15.83:0/2893310603 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fa7dc08f470 con 0x7fa7f41570e0 2024-05-17T22:03:52.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 0x7fa7d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 0x7fa7d4071940 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b930 tx=0x7fa7ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 Processor -- start 2024-05-17T22:03:52.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 -- start start 2024-05-17T22:03:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 0x5587216f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587214ea220 con 0x558721590fc0 2024-05-17T22:03:52.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587214d9dc0 con 0x5587215a7130 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5587214dde00 con 0x5587215b5410 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55932/0 (socket says 172.21.15.83:55932) 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 -- 172.21.15.83:0/664566479 learned_addr learned my addr 172.21.15.83:0/664566479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 0x5587216f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 -- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 msgr2=0x5587216f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 0x5587216f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 -- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 msgr2=0x5587216f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 -- 172.21.15.83:0/664566479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587214b5bf0 con 0x558721590fc0 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 0x5587216f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:03:52.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7f9206640 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40af040 tx=0x7fa7e4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131e8d9f294da239 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7ea7fc640 1 -- 172.21.15.83:0/664566479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5760 con 0x558721590fc0 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7ea7fc640 1 -- 172.21.15.83:0/664566479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e40a5900 con 0x558721590fc0 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.661+0000 7fa7ea7fc640 1 -- 172.21.15.83:0/664566479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5cd0 con 0x558721590fc0 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/664566479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 msgr2=0x5587216f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40af040 tx=0x7fa7e4004ac0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/664566479 shutdown_connections 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558721590fc0 0x5587216f9390 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- 172.21.15.83:0/664566479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215b5410 0x5587216f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/664566479 >> 172.21.15.83:0/664566479 conn(0x55872119b810 msgr2=0x5587216ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/664566479 shutdown_connections 2024-05-17T22:03:52.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/664566479 wait complete. 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- start start 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 Processor -- start 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x5587216feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558721462f30 con 0x5587215b5410 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587214d6740 con 0x5587215a7130 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x558721460cc0 con 0x558721590fc0 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 10 client.0 _ll_register_callbacks cb 0x5587214fb080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55942/0 (socket says 172.21.15.83:55942) 2024-05-17T22:03:52.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 -- 172.21.15.83:0/468146748 learned_addr learned my addr 172.21.15.83:0/468146748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x5587216feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 -- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 msgr2=0x5587216feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x5587216feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 -- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 msgr2=0x5587216ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 -- 172.21.15.83:0/468146748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558721493460 con 0x5587215b5410 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7f9206640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40072a0 tx=0x7fa7e401b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c652cafbb1ec2b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5d70 con 0x5587215b5410 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fa7e40039a0 con 0x5587215b5410 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e40a5900 con 0x5587215b5410 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa7e40cd050 con 0x5587215b5410 2024-05-17T22:03:52.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 10 client.6972 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.665+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5587214989e0 con 0x5587215b5410 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e40a5900 con 0x5587215b5410 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fa7e40d0020 con 0x5587215b5410 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 10 client.6972 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 10 client.6972 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 20 client.6972 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x558721180b80 0x558721708f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 4 client.6972 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558721475d40 con 0x558721180b80 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x558721180b80 0x558721708f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 10 client.6972 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x558721180b80 0x558721708f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ec65a8f65d4748 server_cookie=6848adda55fa823a in_seq=0 out_seq=0 2024-05-17T22:03:52.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 1 -- 172.21.15.83:0/468146748 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558721475d40 con 0x558721180b80 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 10 client.6972 handle_command_reply: tid=0 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 client.6972 shutdown 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 msgr2=0x5587216ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40072a0 tx=0x7fa7e401b240 comp rx=0 tx=0).stop 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 shutdown_connections 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x558721180b80 0x558721708f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x5587216ff600 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215a7130 0x5587216ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 --2- 172.21.15.83:0/468146748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x5587216feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 >> 172.21.15.83:0/468146748 conn(0x55872119b810 msgr2=0x5587216b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa7d37fe640 0 client.6972 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 shutdown_connections 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 1 -- 172.21.15.83:0/468146748 wait complete. 2024-05-17T22:03:52.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.669+0000 7fa801ee3000 20 client.6972 trim_cache size 0 max 16384 2024-05-17T22:03:52.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.685+0000 7fa801ee3000 1 Processor -- start 2024-05-17T22:03:52.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.685+0000 7fa801ee3000 1 -- start start 2024-05-17T22:03:52.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x558721766230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215b5410 0x558721766cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4204b00 con 0x5587215a7130 2024-05-17T22:03:52.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f4073c10 con 0x5587215b5410 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f415be20 con 0x558721590fc0 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55954/0 (socket says 172.21.15.83:55954) 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 -- 172.21.15.83:0/1223886338 learned_addr learned my addr 172.21.15.83:0/1223886338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 -- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 msgr2=0x558721766230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x558721766230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 -- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215b5410 msgr2=0x558721766cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215b5410 0x558721766cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 -- 172.21.15.83:0/1223886338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872157ebe0 con 0x5587215a7130 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7f3fff640 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc05b8d0 tx=0x7fa7dc064430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6120654f273661b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7e9ffb640 1 -- 172.21.15.83:0/1223886338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc08b070 con 0x5587215a7130 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7e9ffb640 1 -- 172.21.15.83:0/1223886338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc05c2d0 con 0x5587215a7130 2024-05-17T22:03:52.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa7e9ffb640 1 -- 172.21.15.83:0/1223886338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0647e0 con 0x5587215a7130 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- 172.21.15.83:0/1223886338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 msgr2=0x558721766770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc05b8d0 tx=0x7fa7dc064430 comp rx=0 tx=0).stop 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- 172.21.15.83:0/1223886338 shutdown_connections 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587215b5410 0x558721766cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215a7130 0x558721766770 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- 172.21.15.83:0/1223886338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x558721590fc0 0x558721766230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- 172.21.15.83:0/1223886338 >> 172.21.15.83:0/1223886338 conn(0x55872119b810 msgr2=0x558721705420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- 172.21.15.83:0/1223886338 shutdown_connections 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- 172.21.15.83:0/1223886338 wait complete. 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- start start 2024-05-17T22:03:52.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 Processor -- start 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558721590fc0 0x5587216afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215a7130 0x5587216b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.689+0000 7fa801ee3000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587214a8dd0 con 0x5587215b5410 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587214e8ff0 con 0x558721590fc0 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5587214dd110 con 0x5587215a7130 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 10 client.0 _ll_register_callbacks cb 0x7fa7e40071f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55958/0 (socket says 172.21.15.83:55958) 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 -- 172.21.15.83:0/378203718 learned_addr learned my addr 172.21.15.83:0/378203718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558721590fc0 0x5587216afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 -- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215a7130 msgr2=0x5587216b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215a7130 0x5587216b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 -- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558721590fc0 msgr2=0x5587216afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558721590fc0 0x5587216afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 -- 172.21.15.83:0/378203718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558721496dc0 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f9206640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa7e401c830 tx=0x7fa7e40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c213b0552cefcf22 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40a5760 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fa7e40b9b80 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7dc05c2d0 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fa7e40a5900 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 10 client.6978 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5587214813e0 con 0x5587215b5410 2024-05-17T22:03:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc05c2d0 con 0x5587215b5410 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fa7e40d0070 con 0x5587215b5410 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 10 client.6978 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 10 client.6978 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 20 client.6978 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55872116ebe0 0x558721726830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 4 client.6978 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x5587214ec170 con 0x55872116ebe0 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55872116ebe0 0x558721726830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 10 client.6978 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7f8a05640 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55872116ebe0 0x558721726830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604fc14ba6f92d5c server_cookie=41d72d07243ee328 in_seq=0 out_seq=0 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 1 -- 172.21.15.83:0/378203718 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587214ec170 con 0x55872116ebe0 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 10 client.6978 handle_command_reply: tid=0 2024-05-17T22:03:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 client.6978 shutdown 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 msgr2=0x558721725e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa7e401c830 tx=0x7fa7e40197a0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 shutdown_connections 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55872116ebe0 0x558721726830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587215b5410 0x558721725e70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7d37fe640 0 client.6978 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587215a7130 0x5587216b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 --2- 172.21.15.83:0/378203718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558721590fc0 0x5587216afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 >> 172.21.15.83:0/378203718 conn(0x55872119b810 msgr2=0x5587216ac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 shutdown_connections 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 1 -- 172.21.15.83:0/378203718 wait complete. 2024-05-17T22:03:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa801ee3000 20 client.6978 trim_cache size 0 max 16384 2024-05-17T22:03:52.688 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:52.688 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:03:52.688 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "493de50f-b9e5-4160-b077-d7a6c221fa36", 2024-05-17T22:03:52.688 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:03:52.688 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:52.688 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:03:52.693+0000 7f5df7ec4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv19 ap=3 DIRTYPARENT f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)->f(v1 m2024-05-17T22:03:04.213534+0000) n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)->n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d2e646100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-17T22:03:52.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 msgr2=0x7fa7d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 0x7fa7d4071940 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b930 tx=0x7fa7ec008040 comp rx=0 tx=0).stop 2024-05-17T22:03:52.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 msgr2=0x7fa7f41fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b440 tx=0x7fa7dc00b900 comp rx=0 tx=0).stop 2024-05-17T22:03:52.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 shutdown_connections 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7f4160420 0x7fa7f4153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7f41570e0 0x7fa7f41fc620 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa7d406f430 0x7fa7d4071940 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 --2- 172.21.15.83:0/2893310603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7f40b4f20 0x7fa7f415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 >> 172.21.15.83:0/2893310603 conn(0x7fa7f406cd90 msgr2=0x7fa7f40d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 shutdown_connections 2024-05-17T22:03:52.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.693+0000 7fa7fac69640 1 -- 172.21.15.83:0/2893310603 wait complete. 2024-05-17T22:03:52.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:03:52.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:03:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.853+0000 7f8ed2e14640 1 Processor -- start 2024-05-17T22:03:52.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.853+0000 7f8ed2e14640 1 -- start start 2024-05-17T22:03:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed2e14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed2e14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ecc0b44c0 0x7f8ecc0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed2e14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ecc06f2d0 con 0x7f8ecc06eec0 2024-05-17T22:03:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed2e14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8ecc15c3b0 con 0x7f8ecc0b44c0 2024-05-17T22:03:52.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed2e14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc06f5a0 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed13b1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ecc0b44c0 0x7f8ecc0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:60472/0 (socket says 172.21.15.83:60472) 2024-05-17T22:03:52.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ed0bb0640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8ecc06eec0 0x7f8ecc1601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:49362/0 (socket says 172.21.15.83:49362) 2024-05-17T22:03:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 -- 172.21.15.83:0/1392018681 learned_addr learned my addr 172.21.15.83:0/1392018681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 -- 172.21.15.83:0/1392018681 >> v2:172.21.15.155:3300/0 conn(0x7f8ecc0b44c0 msgr2=0x7f8ecc0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/1392018681 >> v2:172.21.15.155:3300/0 conn(0x7f8ecc0b44c0 0x7f8ecc0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 -- 172.21.15.83:0/1392018681 >> v1:172.21.15.155:6789/0 conn(0x7f8ecc06eec0 legacy=0x7f8ecc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.857+0000 7f8ec3fff640 1 -- 172.21.15.83:0/1392018681 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc06f880 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/1392018681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009870 tx=0x7f8ebc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b30bede2def708 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ec37fe640 1 -- 172.21.15.83:0/1392018681 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0ab650 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ec37fe640 1 -- 172.21.15.83:0/1392018681 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc0b0070 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ec37fe640 1 -- 172.21.15.83:0/1392018681 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0aba10 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- 172.21.15.83:0/1392018681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 msgr2=0x7f8ecc15be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/1392018681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009870 tx=0x7f8ebc01c560 comp rx=0 tx=0).stop 2024-05-17T22:03:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- 172.21.15.83:0/1392018681 shutdown_connections 2024-05-17T22:03:52.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/1392018681 >> v2:172.21.15.155:3300/0 conn(0x7f8ecc0b44c0 0x7f8ecc0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/1392018681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc15be70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- 172.21.15.83:0/1392018681 >> 172.21.15.83:0/1392018681 conn(0x7f8ecc06cd90 msgr2=0x7f8ecc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- 172.21.15.83:0/1392018681 shutdown_connections 2024-05-17T22:03:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- 172.21.15.83:0/1392018681 wait complete. 2024-05-17T22:03:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 Processor -- start 2024-05-17T22:03:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.861+0000 7f8ed2e14640 1 -- start start 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 0x7f8ecc160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc160a40 con 0x7f8ecc06eec0 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1608c0 con 0x7f8ecc0b44c0 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc0b38d0 con 0x7f8ecc0b08a0 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55968/0 (socket says 172.21.15.83:55968) 2024-05-17T22:03:52.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/801292701 learned_addr learned my addr 172.21.15.83:0/801292701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 0x7f8ecc160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 msgr2=0x7f8ecc151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 msgr2=0x7f8ecc160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 0x7f8ecc160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/801292701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc153c00 con 0x7f8ecc06eec0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 0x7f8ecc160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c430 tx=0x7f8ec800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ec17fa640 1 -- 172.21.15.83:0/801292701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80074e0 con 0x7f8ecc06eec0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ec17fa640 1 -- 172.21.15.83:0/801292701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec8007680 con 0x7f8ecc06eec0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc153ee0 con 0x7f8ecc06eec0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ec17fa640 1 -- 172.21.15.83:0/801292701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004a10 con 0x7f8ecc06eec0 2024-05-17T22:03:52.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.865+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc1544a0 con 0x7f8ecc06eec0 2024-05-17T22:03:52.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec17fa640 1 -- 172.21.15.83:0/801292701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ec801a020 con 0x7f8ecc06eec0 2024-05-17T22:03:52.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec17fa640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 0x7f8eb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec17fa640 1 -- 172.21.15.83:0/801292701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f8ec808eb30 con 0x7f8ecc06eec0 2024-05-17T22:03:52.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 0x7f8eb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 0x7f8eb8071940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0af000 tx=0x7f8ebc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 Processor -- start 2024-05-17T22:03:52.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 -- start start 2024-05-17T22:03:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cd7f80 0x55eb92e3de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc4880 0x55eb92e3e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb92cdca90 con 0x55eb92cc5090 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eb92c50120 con 0x55eb92cc4880 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55eb92c3fcc0 con 0x55eb92cd7f80 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55980/0 (socket says 172.21.15.83:55980) 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 -- 172.21.15.83:0/3692804678 learned_addr learned my addr 172.21.15.83:0/3692804678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cd7f80 0x55eb92e3de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc4880 0x55eb92e3e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 -- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cd7f80 msgr2=0x55eb92e3de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cd7f80 0x55eb92e3de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 -- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc4880 msgr2=0x55eb92e3e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc4880 0x55eb92e3e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 -- 172.21.15.83:0/3692804678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb92bfc110 con 0x55eb92cc5090 2024-05-17T22:03:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40074c0 tx=0x7f8ec4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db73c6352de81d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eb27fc640 1 -- 172.21.15.83:0/3692804678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4019020 con 0x55eb92cc5090 2024-05-17T22:03:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eb27fc640 1 -- 172.21.15.83:0/3692804678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4015600 con 0x55eb92cc5090 2024-05-17T22:03:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.869+0000 7f8eb27fc640 1 -- 172.21.15.83:0/3692804678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40159f0 con 0x55eb92cc5090 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/3692804678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 msgr2=0x55eb92e3e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40074c0 tx=0x7f8ec4009750 comp rx=0 tx=0).stop 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/3692804678 shutdown_connections 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc5090 0x55eb92e3e8a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc4880 0x55eb92e3e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- 172.21.15.83:0/3692804678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cd7f80 0x55eb92e3de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/3692804678 >> 172.21.15.83:0/3692804678 conn(0x55eb92d13010 msgr2=0x55eb92df35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/3692804678 shutdown_connections 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/3692804678 wait complete. 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- start start 2024-05-17T22:03:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 Processor -- start 2024-05-17T22:03:52.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92dfaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92dffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb92bfbec0 con 0x55eb92cc4880 2024-05-17T22:03:52.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55eb92c43d00 con 0x55eb92cd7f80 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55eb92909680 con 0x55eb92cc5090 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 10 client.0 _ll_register_callbacks cb 0x55eb92cf71c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55988/0 (socket says 172.21.15.83:55988) 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/1076092561 learned_addr learned my addr 172.21.15.83:0/1076092561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92dfaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 msgr2=0x55eb92dfaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92dfaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 msgr2=0x55eb92dffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92dffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92dffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/1076092561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb92bfe890 con 0x55eb92cc4880 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8000f90 tx=0x7f8ec805b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d4a8dbc1e1cc0d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec805b290 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f8ec8004bb0 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.873+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4015600 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8087070 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 10 client.6987 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55eb92c00650 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4015600 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f8ec805f180 con 0x55eb92cc4880 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 10 client.6987 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 10 client.6987 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 20 client.6987 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928e6930 0x55eb92e44e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 4 client.6987 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb92bdbc40 con 0x55eb928e6930 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928e6930 0x55eb92e44e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928e6930 0x55eb92e44e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554214df75c9c528 server_cookie=a643303d711e2203 in_seq=0 out_seq=0 2024-05-17T22:03:52.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 10 client.6987 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 1 -- 172.21.15.83:0/1076092561 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb92bdbc40 con 0x55eb928e6930 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 10 client.6987 handle_command_reply: tid=0 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 client.6987 shutdown 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 msgr2=0x55eb92dfaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8000f90 tx=0x7f8ec805b8b0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 shutdown_connections 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928e6930 0x55eb92e44e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92dffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92dfaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 --2- 172.21.15.83:0/1076092561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92dfaa20 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eab7fe640 0 client.6987 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 >> 172.21.15.83:0/1076092561 conn(0x55eb92d13010 msgr2=0x55eb92df35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 shutdown_connections 2024-05-17T22:03:52.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 1 -- 172.21.15.83:0/1076092561 wait complete. 2024-05-17T22:03:52.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.877+0000 7f8eda08e000 20 client.6987 trim_cache size 0 max 16384 2024-05-17T22:03:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.893+0000 7f8eda08e000 1 Processor -- start 2024-05-17T22:03:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.893+0000 7f8eda08e000 1 -- start start 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc4880 0x55eb92eb5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc5090 0x55eb92eb5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc203650 con 0x55eb92cd7f80 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc15aa90 con 0x55eb92cc5090 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc1fb280 con 0x55eb92cc4880 2024-05-17T22:03:52.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55992/0 (socket says 172.21.15.83:55992) 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 -- 172.21.15.83:0/998611824 learned_addr learned my addr 172.21.15.83:0/998611824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 -- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc4880 msgr2=0x55eb92eb5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc4880 0x55eb92eb5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 -- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc5090 msgr2=0x55eb92eb5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc5090 0x55eb92eb5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 -- 172.21.15.83:0/998611824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb92df7630 con 0x55eb92cd7f80 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8ed13b1640 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b010 tx=0x7f8ec4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84acc292418347d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/998611824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4019040 con 0x55eb92cd7f80 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/998611824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec40158d0 con 0x55eb92cd7f80 2024-05-17T22:03:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/998611824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4015cc0 con 0x55eb92cd7f80 2024-05-17T22:03:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- 172.21.15.83:0/998611824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 msgr2=0x55eb92eb6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b010 tx=0x7f8ec4014000 comp rx=0 tx=0).stop 2024-05-17T22:03:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- 172.21.15.83:0/998611824 shutdown_connections 2024-05-17T22:03:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cd7f80 0x55eb92eb6000 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cc5090 0x55eb92eb5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 --2- 172.21.15.83:0/998611824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc4880 0x55eb92eb5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- 172.21.15.83:0/998611824 >> 172.21.15.83:0/998611824 conn(0x55eb92d13010 msgr2=0x55eb92df99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- 172.21.15.83:0/998611824 shutdown_connections 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- 172.21.15.83:0/998611824 wait complete. 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 -- start start 2024-05-17T22:03:52.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.897+0000 7f8eda08e000 1 Processor -- start 2024-05-17T22:03:52.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92e718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92df1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb92be70f0 con 0x55eb92cc4880 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc15a440 con 0x55eb92cd7f80 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55eb92c48330 con 0x55eb92cc5090 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 10 client.0 _ll_register_callbacks cb 0x7f8ec80578e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55994/0 (socket says 172.21.15.83:55994) 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/2669887410 learned_addr learned my addr 172.21.15.83:0/2669887410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 msgr2=0x55eb92e718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92e718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 msgr2=0x55eb92df1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92df1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 -- 172.21.15.83:0/2669887410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb92bdf8c0 con 0x55eb92cc4880 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ed0bb0640 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8057880 tx=0x7f8ec808cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1633ccf2ed0d7b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:03:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004db0 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f8ec805dd00 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec40158d0 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f8ec805b290 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 10 client.6993 fetch_fsmap learned FSMap version 66 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55eb92bfafa0 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec40158d0 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f8ec805a4f0 con 0x55eb92cc4880 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 10 client.6993 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 10 client.6993 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 20 client.6993 populate_metadata read hostname 'smithi083' 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928d4990 0x55eb92df2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:03:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 4 client.6993 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb92c52070 con 0x55eb928d4990 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928d4990 0x55eb92df2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 10 client.6993 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8ec3fff640 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928d4990 0x55eb92df2690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a99c670c119e60 server_cookie=1fd36249de66a9b0 in_seq=0 out_seq=0 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 1 -- 172.21.15.83:0/2669887410 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55eb92c52070 con 0x55eb928d4990 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 10 client.6993 handle_command_reply: tid=0 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 client.6993 shutdown 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 msgr2=0x55eb92e71370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8057880 tx=0x7f8ec808cea0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 shutdown_connections 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55eb928d4990 0x55eb92df2690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb92cd7f80 0x55eb92df1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55eb92cc5090 0x55eb92e718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eab7fe640 0 client.6993 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:03:52.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 --2- 172.21.15.83:0/2669887410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb92cc4880 0x55eb92e71370 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 >> 172.21.15.83:0/2669887410 conn(0x55eb92d13010 msgr2=0x55eb92dfa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 shutdown_connections 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 1 -- 172.21.15.83:0/2669887410 wait complete. 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.901+0000 7f8eda08e000 20 client.6993 trim_cache size 0 max 16384 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "493de50f-b9e5-4160-b077-d7a6c221fa36": { 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir", 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "493de50f-b9e5-4160-b077-d7a6c221fa36", 2024-05-17T22:03:52.896 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T22:03:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:52.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:03:52.897 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:03:52.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 msgr2=0x7f8eb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 0x7f8eb8071940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0af000 tx=0x7f8ebc0a7000 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 msgr2=0x7f8ecc153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c430 tx=0x7f8ec800c8f0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 shutdown_connections 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8eb806f430 0x7f8eb8071940 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ecc0b44c0 0x7f8ecc160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ecc0b08a0 0x7f8ecc151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 --2- 172.21.15.83:0/801292701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ecc06eec0 0x7f8ecc153140 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 >> 172.21.15.83:0/801292701 conn(0x7f8ecc06cd90 msgr2=0x7f8ecc154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 shutdown_connections 2024-05-17T22:03:52.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:03:52.905+0000 7f8ed2e14640 1 -- 172.21.15.83:0/801292701 wait complete. 2024-05-17T22:03:52.910 INFO:tasks.cephfs.filesystem:scrub status for tag:493de50f-b9e5-4160-b077-d7a6c221fa36 - {'path': '/dir', 'tag': '493de50f-b9e5-4160-b077-d7a6c221fa36', 'options': 'recursive,repair'} 2024-05-17T22:04:00.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:04:00.909 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:04:22.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:22.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:04:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 Processor -- start 2024-05-17T22:04:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 -- start start 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68bc0cdf70 0x7f68bc0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68bc0d1f50 0x7f68bc0cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68bc06c930 con 0x7f68bc0b2f70 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f68bc06d540 con 0x7f68bc0cdf70 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68c1051640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f68bc06cc10 con 0x7f68bc0d1f50 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68ba59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68bc0cdf70 0x7f68bc0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68bad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68bc0b2f70 0x7f68bc0d1840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35384/0 (socket says 172.21.15.83:35384) 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68bad9d640 1 -- 172.21.15.83:0/3852804408 learned_addr learned my addr 172.21.15.83:0/3852804408 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68bb59e640 1 --2- 172.21.15.83:0/3852804408 >> v2:172.21.15.155:3300/0 conn(0x7f68bc0d1f50 0x7f68bc0cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68ba59c640 1 -- 172.21.15.83:0/3852804408 >> v2:172.21.15.155:3300/0 conn(0x7f68bc0d1f50 msgr2=0x7f68bc0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68ba59c640 1 --2- 172.21.15.83:0/3852804408 >> v2:172.21.15.155:3300/0 conn(0x7f68bc0d1f50 0x7f68bc0cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68ba59c640 1 -- 172.21.15.83:0/3852804408 >> v1:172.21.15.83:6789/0 conn(0x7f68bc0b2f70 legacy=0x7f68bc0d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.085+0000 7f68ba59c640 1 -- 172.21.15.83:0/3852804408 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc06cef0 con 0x7f68bc0cdf70 2024-05-17T22:04:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68ba59c640 1 --2- 172.21.15.83:0/3852804408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0cdf70 0x7f68bc0ce6f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b82e0 tx=0x7f68ac01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f034e25b7f22097a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68b9d9b640 1 -- 172.21.15.83:0/3852804408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0b0070 con 0x7f68bc0cdf70 2024-05-17T22:04:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68b9d9b640 1 -- 172.21.15.83:0/3852804408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ac0ab600 con 0x7f68bc0cdf70 2024-05-17T22:04:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68b9d9b640 1 -- 172.21.15.83:0/3852804408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ab9d0 con 0x7f68bc0cdf70 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 -- 172.21.15.83:0/3852804408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0cdf70 msgr2=0x7f68bc0ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 --2- 172.21.15.83:0/3852804408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0cdf70 0x7f68bc0ce6f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b82e0 tx=0x7f68ac01c3f0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 -- 172.21.15.83:0/3852804408 shutdown_connections 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 --2- 172.21.15.83:0/3852804408 >> v2:172.21.15.155:3300/0 conn(0x7f68bc0d1f50 0x7f68bc0cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 --2- 172.21.15.83:0/3852804408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0cdf70 0x7f68bc0ce6f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 -- 172.21.15.83:0/3852804408 >> 172.21.15.83:0/3852804408 conn(0x7f68bc0b4170 msgr2=0x7f68bc0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 -- 172.21.15.83:0/3852804408 shutdown_connections 2024-05-17T22:04:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.089+0000 7f68c1051640 1 -- 172.21.15.83:0/3852804408 wait complete. 2024-05-17T22:04:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 Processor -- start 2024-05-17T22:04:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 -- start start 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68bc0cdf70 0x7f68bc157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68bc0d1f50 0x7f68bc15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0cfd10 con 0x7f68bc0b2f70 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0cfb90 con 0x7f68bc0d1f50 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68c1051640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68bc0b8060 con 0x7f68bc0cdf70 2024-05-17T22:04:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38926/0 (socket says 172.21.15.83:38926) 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 -- 172.21.15.83:0/2182791888 learned_addr learned my addr 172.21.15.83:0/2182791888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68ba59c640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68bc0cdf70 0x7f68bc157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bb59e640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68bc0d1f50 0x7f68bc15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 -- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68bc0cdf70 msgr2=0x7f68bc157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68bc0cdf70 0x7f68bc157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 -- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68bc0d1f50 msgr2=0x7f68bc15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68bc0d1f50 0x7f68bc15ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.093+0000 7f68bad9d640 1 -- 172.21.15.83:0/2182791888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc156160 con 0x7f68bc0b2f70 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c3f0 tx=0x7f68b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 -- 172.21.15.83:0/2182791888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0017070 con 0x7f68bc0b2f70 2024-05-17T22:04:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 -- 172.21.15.83:0/2182791888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b0005cf0 con 0x7f68bc0b2f70 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc14f9b0 con 0x7f68bc0b2f70 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 -- 172.21.15.83:0/2182791888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0007310 con 0x7f68bc0b2f70 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc14ff70 con 0x7f68bc0b2f70 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 -- 172.21.15.83:0/2182791888 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68b001a070 con 0x7f68bc0b2f70 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 0x7f68a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68ba59c640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 0x7f68a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68a37fe640 1 -- 172.21.15.83:0/2182791888 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f68b008e6f0 con 0x7f68bc0b2f70 2024-05-17T22:04:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68ba59c640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 0x7f68a8071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ad000 tx=0x7f68ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68c82cb000 1 Processor -- start 2024-05-17T22:04:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.097+0000 7f68c82cb000 1 -- start start 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65a880 0x557dca7d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557dca672a90 con 0x557dca66df80 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557dca5e6120 con 0x557dca65a880 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557dca5d5cc0 con 0x557dca65b090 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38932/0 (socket says 172.21.15.83:38932) 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 -- 172.21.15.83:0/2427377572 learned_addr learned my addr 172.21.15.83:0/2427377572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68ba59c640 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65a880 0x557dca7d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 -- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 msgr2=0x557dca7d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 -- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65a880 msgr2=0x557dca7d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65a880 0x557dca7d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 -- 172.21.15.83:0/2427377572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dca592110 con 0x557dca66df80 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68bad9d640 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f68b0056ed0 tx=0x7f68b0056e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee359c684aca01d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68a0ff9640 1 -- 172.21.15.83:0/2427377572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b005f200 con 0x557dca66df80 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68a0ff9640 1 -- 172.21.15.83:0/2427377572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b005d070 con 0x557dca66df80 2024-05-17T22:04:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68a0ff9640 1 -- 172.21.15.83:0/2427377572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00076d0 con 0x557dca66df80 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- 172.21.15.83:0/2427377572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 msgr2=0x557dca7d3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f68b0056ed0 tx=0x7f68b0056e40 comp rx=0 tx=0).stop 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- 172.21.15.83:0/2427377572 shutdown_connections 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65a880 0x557dca7d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 --2- 172.21.15.83:0/2427377572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca7d3e20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- 172.21.15.83:0/2427377572 >> 172.21.15.83:0/2427377572 conn(0x557dca6a9010 msgr2=0x557dca7895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- 172.21.15.83:0/2427377572 shutdown_connections 2024-05-17T22:04:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- 172.21.15.83:0/2427377572 wait complete. 2024-05-17T22:04:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.101+0000 7f68c82cb000 1 -- start start 2024-05-17T22:04:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 Processor -- start 2024-05-17T22:04:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65a880 0x557dca790a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65b090 0x557dca790f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557dca591ec0 con 0x557dca66df80 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557dca5d9d00 con 0x557dca65b090 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557dca29f680 con 0x557dca65a880 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 10 client.0 _ll_register_callbacks cb 0x557dca68d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38948/0 (socket says 172.21.15.83:38948) 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 -- 172.21.15.83:0/243184281 learned_addr learned my addr 172.21.15.83:0/243184281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68ba59c640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65b090 0x557dca790f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bad9d640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65a880 0x557dca790a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 -- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65a880 msgr2=0x557dca790a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65a880 0x557dca790a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 -- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65b090 msgr2=0x557dca790f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65b090 0x557dca790f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 -- 172.21.15.83:0/243184281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dca594890 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bb59e640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e960 tx=0x7f68b400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5882a76871d798 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4017070 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f68b400ca60 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b005d070 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f68b4022050 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 10 client.7008 fetch_fsmap learned FSMap version 66 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x557dca596650 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b005d070 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f68b4025020 con 0x557dca66df80 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 10 client.7008 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:04:23.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 10 client.7008 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 20 client.7008 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca27c930 0x557dca7dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 4 client.7008 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dca571c40 con 0x557dca27c930 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bad9d640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca27c930 0x557dca7dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f6899ffb640 10 client.7008 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.105+0000 7f68bad9d640 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca27c930 0x557dca7dae90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d448e7beaf943b server_cookie=cb86140871058536 in_seq=0 out_seq=0 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f6899ffb640 1 -- 172.21.15.83:0/243184281 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557dca571c40 con 0x557dca27c930 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f6899ffb640 10 client.7008 handle_command_reply: tid=0 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 client.7008 shutdown 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 msgr2=0x557dca795be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e960 tx=0x7f68b400ee20 comp rx=0 tx=0).stop 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 shutdown_connections 2024-05-17T22:04:23.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca27c930 0x557dca7dae90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca66df80 0x557dca795be0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca65b090 0x557dca790f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f6899ffb640 0 client.7008 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 --2- 172.21.15.83:0/243184281 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65a880 0x557dca790a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 >> 172.21.15.83:0/243184281 conn(0x557dca6a9010 msgr2=0x557dca7895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 shutdown_connections 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 1 -- 172.21.15.83:0/243184281 wait complete. 2024-05-17T22:04:23.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.109+0000 7f68c82cb000 20 client.7008 trim_cache size 0 max 16384 2024-05-17T22:04:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.125+0000 7f68c82cb000 1 Processor -- start 2024-05-17T22:04:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.125+0000 7f68c82cb000 1 -- start start 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca7ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc2057d0 con 0x557dca65a880 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc204890 con 0x557dca66df80 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68bc154e00 con 0x557dca65b090 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38958/0 (socket says 172.21.15.83:38958) 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 -- 172.21.15.83:0/239859646 learned_addr learned my addr 172.21.15.83:0/239859646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 -- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 msgr2=0x557dca7ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 -- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 msgr2=0x557dca7ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca7ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 -- 172.21.15.83:0/239859646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dca5758c0 con 0x557dca65a880 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68bad9d640 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f68b008cbe0 tx=0x7f68b005d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bd994cbc43cbce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f689bfff640 1 -- 172.21.15.83:0/239859646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b005f200 con 0x557dca65a880 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f689bfff640 1 -- 172.21.15.83:0/239859646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b00570e0 con 0x557dca65a880 2024-05-17T22:04:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f689bfff640 1 -- 172.21.15.83:0/239859646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00076d0 con 0x557dca65a880 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- 172.21.15.83:0/239859646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 msgr2=0x557dca790c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f68b008cbe0 tx=0x7f68b005d000 comp rx=0 tx=0).stop 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- 172.21.15.83:0/239859646 shutdown_connections 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca7ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca7ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 --2- 172.21.15.83:0/239859646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca790c20 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- 172.21.15.83:0/239859646 >> 172.21.15.83:0/239859646 conn(0x557dca6a9010 msgr2=0x557dca7d34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- 172.21.15.83:0/239859646 shutdown_connections 2024-05-17T22:04:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- 172.21.15.83:0/239859646 wait complete. 2024-05-17T22:04:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 -- start start 2024-05-17T22:04:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.129+0000 7f68c82cb000 1 Processor -- start 2024-05-17T22:04:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca846ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca78abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557dca5de330 con 0x557dca65a880 2024-05-17T22:04:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557dca57d0f0 con 0x557dca66df80 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557dca29f680 con 0x557dca65b090 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 10 client.0 _ll_register_callbacks cb 0x7f68b4007630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38966/0 (socket says 172.21.15.83:38966) 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 -- 172.21.15.83:0/1210718395 learned_addr learned my addr 172.21.15.83:0/1210718395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bb59e640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca78abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68ba59c640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca846ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 -- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 msgr2=0x557dca846ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca846ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 -- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 msgr2=0x557dca78abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca78abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 -- 172.21.15.83:0/1210718395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dca590fa0 con 0x557dca65a880 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bb59e640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca78abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68ba59c640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca846ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68bad9d640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f68b0056ed0 tx=0x7f68b0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cbfa86242abcc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b005f200 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f68b00076c0 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b00570e0 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f68b005a0b0 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 10 client.7014 fetch_fsmap learned FSMap version 66 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x557dca290220 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b00570e0 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f68b005a250 con 0x557dca65a880 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 10 client.7014 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 10 client.7014 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 20 client.7014 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca26a990 0x557dca78b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 4 client.7014 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557dca5e8070 con 0x557dca26a990 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68ba59c640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca26a990 0x557dca78b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 10 client.7014 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68ba59c640 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca26a990 0x557dca78b100 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc469fc7d032843 server_cookie=d98f2cfba76b647f in_seq=0 out_seq=0 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 1 -- 172.21.15.83:0/1210718395 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557dca5e8070 con 0x557dca26a990 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 10 client.7014 handle_command_reply: tid=0 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 client.7014 shutdown 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 msgr2=0x557dca8467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f68b0056ed0 tx=0x7f68b0000bd0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 shutdown_connections 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x557dca26a990 0x557dca78b100 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dca66df80 0x557dca78abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557dca65b090 0x557dca846ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f6899ffb640 0 client.7014 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 --2- 172.21.15.83:0/1210718395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557dca65a880 0x557dca8467a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 >> 172.21.15.83:0/1210718395 conn(0x557dca6a9010 msgr2=0x557dca7d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 shutdown_connections 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 1 -- 172.21.15.83:0/1210718395 wait complete. 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c82cb000 20 client.7014 trim_cache size 0 max 16384 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:04:23.128 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:04:23.129 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:04:23.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 msgr2=0x7f68a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 0x7f68a8071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ad000 tx=0x7f68ac008040 comp rx=0 tx=0).stop 2024-05-17T22:04:23.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 msgr2=0x7f68bc155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c3f0 tx=0x7f68b000c8b0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 shutdown_connections 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68a806f400 0x7f68a8071910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68bc0d1f50 0x7f68bc15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68bc0cdf70 0x7f68bc157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 --2- 172.21.15.83:0/2182791888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68bc0b2f70 0x7f68bc155c20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 >> 172.21.15.83:0/2182791888 conn(0x7f68bc0b4170 msgr2=0x7f68bc0b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 shutdown_connections 2024-05-17T22:04:23.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.133+0000 7f68c1051640 1 -- 172.21.15.83:0/2182791888 wait complete. 2024-05-17T22:04:23.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:04:23.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:04:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 Processor -- start 2024-05-17T22:04:23.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 -- start start 2024-05-17T22:04:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61680d1040 0x7f6168144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f616815e080 0x7f6168072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f6168073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f616815ea40 con 0x7f61680d1040 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f616815ed20 con 0x7f616815e080 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616ebba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f616815f000 con 0x7f616815e490 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61680d1040 0x7f6168144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61680d1040 0x7f6168144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38968/0 (socket says 172.21.15.83:38968) 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 -- 172.21.15.83:0/2774606587 learned_addr learned my addr 172.21.15.83:0/2774606587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f615ffff640 1 --2- 172.21.15.83:0/2774606587 >> v2:172.21.15.155:3300/0 conn(0x7f616815e080 0x7f6168072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616d157640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f6168073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 -- 172.21.15.83:0/2774606587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 msgr2=0x7f6168073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f6168073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 -- 172.21.15.83:0/2774606587 >> v2:172.21.15.155:3300/0 conn(0x7f616815e080 msgr2=0x7f6168072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 --2- 172.21.15.83:0/2774606587 >> v2:172.21.15.155:3300/0 conn(0x7f616815e080 0x7f6168072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.297+0000 7f616c956640 1 -- 172.21.15.83:0/2774606587 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616806ef30 con 0x7f61680d1040 2024-05-17T22:04:23.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616c956640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f6168144080 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f616815ef30 tx=0x7f616401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d951ebe3bed3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f615f7fe640 1 -- 172.21.15.83:0/2774606587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640ab5f0 con 0x7f61680d1040 2024-05-17T22:04:23.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f615f7fe640 1 -- 172.21.15.83:0/2774606587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61640b0070 con 0x7f61680d1040 2024-05-17T22:04:23.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f615f7fe640 1 -- 172.21.15.83:0/2774606587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640ab9b0 con 0x7f61680d1040 2024-05-17T22:04:23.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 -- 172.21.15.83:0/2774606587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 msgr2=0x7f6168144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f6168144080 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f616815ef30 tx=0x7f616401c520 comp rx=0 tx=0).stop 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 -- 172.21.15.83:0/2774606587 shutdown_connections 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f6168073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 --2- 172.21.15.83:0/2774606587 >> v2:172.21.15.155:3300/0 conn(0x7f616815e080 0x7f6168072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 --2- 172.21.15.83:0/2774606587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f6168144080 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 -- 172.21.15.83:0/2774606587 >> 172.21.15.83:0/2774606587 conn(0x7f616806cd90 msgr2=0x7f61680b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 -- 172.21.15.83:0/2774606587 shutdown_connections 2024-05-17T22:04:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.301+0000 7f616ebba640 1 -- 172.21.15.83:0/2774606587 wait complete. 2024-05-17T22:04:23.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 Processor -- start 2024-05-17T22:04:23.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- start start 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 0x7f616814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f61680b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6168162260 con 0x7f61680d1040 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f61681620e0 con 0x7f616815e080 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f61681623e0 con 0x7f616815e490 2024-05-17T22:04:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38982/0 (socket says 172.21.15.83:38982) 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 -- 172.21.15.83:0/3866580095 learned_addr learned my addr 172.21.15.83:0/3866580095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f615ffff640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 0x7f616814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616d157640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f61680b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 -- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 msgr2=0x7f61680b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f61680b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 -- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 msgr2=0x7f616814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 0x7f616814df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 -- 172.21.15.83:0/3866580095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61680b2d40 con 0x7f61680d1040 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f615ffff640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 0x7f616814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616c956640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f616401cdc0 tx=0x7f616401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640abe60 con 0x7f61680d1040 2024-05-17T22:04:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61640b4800 con 0x7f61680d1040 2024-05-17T22:04:23.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640b4bb0 con 0x7f61680d1040 2024-05-17T22:04:23.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616815baf0 con 0x7f61680d1040 2024-05-17T22:04:23.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61680b46d0 con 0x7f61680d1040 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.305+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168144080 con 0x7f61680d1040 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.309+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61640b3380 con 0x7f61680d1040 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.309+0000 7f615d7fa640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 0x7f6154071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.309+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f6164129540 con 0x7f61680d1040 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.309+0000 7f615ffff640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 0x7f6154071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.309+0000 7f615ffff640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 0x7f6154071940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f616813fff0 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.313+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f61640c4060 con 0x7f61680d1040 2024-05-17T22:04:23.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f616806f630 con 0x7f61680d1040 2024-05-17T22:04:23.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f615d7fa640 1 -- 172.21.15.83:0/3866580095 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7f6164016110 con 0x7f61680d1040 2024-05-17T22:04:23.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:23.490 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:23.463499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":6}]} 2024-05-17T22:04:23.491 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 66 2024-05-17T22:04:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 msgr2=0x7f6154071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 0x7f6154071940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f616813fff0 tx=0x7f6158008040 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 msgr2=0x7f616814e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f616401cdc0 tx=0x7f616401ad60 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 shutdown_connections 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616815e490 0x7f61680b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f615406f430 0x7f6154071940 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616815e080 0x7f616814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 --2- 172.21.15.83:0/3866580095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61680d1040 0x7f616814e740 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 >> 172.21.15.83:0/3866580095 conn(0x7f616806cd90 msgr2=0x7f616813d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 shutdown_connections 2024-05-17T22:04:23.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.501+0000 7f616ebba640 1 -- 172.21.15.83:0/3866580095 wait complete. 2024-05-17T22:04:23.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-17T22:04:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.669+0000 7f4761b5e640 1 Processor -- start 2024-05-17T22:04:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.669+0000 7f4761b5e640 1 -- start start 2024-05-17T22:04:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4761b5e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 0x7f475c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4761b5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4761b5e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475c0b5640 con 0x7f475c0cf830 2024-05-17T22:04:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4761b5e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f475c070870 con 0x7f475c0cdf40 2024-05-17T22:04:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4761b5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f475c0b58e0 con 0x7f475c15d530 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4753fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 0x7f475c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4753fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 0x7f475c15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:47882/0 (socket says 172.21.15.83:47882) 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 --2- 172.21.15.83:0/2128741991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:51558/0 (socket says 172.21.15.83:51558) 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f4753fff640 1 -- 172.21.15.83:0/2128741991 learned_addr learned my addr 172.21.15.83:0/2128741991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 -- 172.21.15.83:0/2128741991 >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 msgr2=0x7f475c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 --2- 172.21.15.83:0/2128741991 >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 0x7f475c15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 -- 172.21.15.83:0/2128741991 >> v1:172.21.15.155:6789/0 conn(0x7f475c0cf830 legacy=0x7f475c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.673+0000 7f475bfff640 1 -- 172.21.15.83:0/2128741991 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c06f060 con 0x7f475c15d530 2024-05-17T22:04:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f475bfff640 1 --2- 172.21.15.83:0/2128741991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4754009d90 tx=0x7f475401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6701ece742ca61ad server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f475affd640 1 -- 172.21.15.83:0/2128741991 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540b2070 con 0x7f475c15d530 2024-05-17T22:04:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f475affd640 1 -- 172.21.15.83:0/2128741991 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47540a5940 con 0x7f475c15d530 2024-05-17T22:04:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f475affd640 1 -- 172.21.15.83:0/2128741991 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540a5d10 con 0x7f475c15d530 2024-05-17T22:04:23.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 -- 172.21.15.83:0/2128741991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 msgr2=0x7f475c1663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 --2- 172.21.15.83:0/2128741991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4754009d90 tx=0x7f475401c970 comp rx=0 tx=0).stop 2024-05-17T22:04:23.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 -- 172.21.15.83:0/2128741991 shutdown_connections 2024-05-17T22:04:23.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 --2- 172.21.15.83:0/2128741991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c15d530 0x7f475c1663d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 --2- 172.21.15.83:0/2128741991 >> v2:172.21.15.155:3300/0 conn(0x7f475c0cdf40 0x7f475c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 -- 172.21.15.83:0/2128741991 >> 172.21.15.83:0/2128741991 conn(0x7f475c06cd90 msgr2=0x7f475c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 -- 172.21.15.83:0/2128741991 shutdown_connections 2024-05-17T22:04:23.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.677+0000 7f4761b5e640 1 -- 172.21.15.83:0/2128741991 wait complete. 2024-05-17T22:04:23.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 Processor -- start 2024-05-17T22:04:23.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- start start 2024-05-17T22:04:23.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c0cdf40 0x7f475c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c15d530 0x7f475c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f475c1694c0 con 0x7f475c0cf830 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f475c169340 con 0x7f475c15d530 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f475c169640 con 0x7f475c0cdf40 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38996/0 (socket says 172.21.15.83:38996) 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 -- 172.21.15.83:0/3057154084 learned_addr learned my addr 172.21.15.83:0/3057154084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f475b7fe640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c0cdf40 0x7f475c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f475bfff640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c15d530 0x7f475c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 -- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c0cdf40 msgr2=0x7f475c1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c0cdf40 0x7f475c1554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 -- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c15d530 msgr2=0x7f475c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c15d530 0x7f475c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 -- 172.21.15.83:0/3057154084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c154900 con 0x7f475c0cf830 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4753fff640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f474400e820 tx=0x7f474400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4758ff9640 1 -- 172.21.15.83:0/3057154084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744017070 con 0x7f475c0cf830 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4758ff9640 1 -- 172.21.15.83:0/3057154084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474400c8e0 con 0x7f475c0cf830 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4758ff9640 1 -- 172.21.15.83:0/3057154084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400ccd0 con 0x7f475c0cf830 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c152de0 con 0x7f475c0cf830 2024-05-17T22:04:23.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.681+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c1533a0 con 0x7f475c0cf830 2024-05-17T22:04:23.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4758ff9640 1 -- 172.21.15.83:0/3057154084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f474401a070 con 0x7f475c0cf830 2024-05-17T22:04:23.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4758ff9640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 0x7f473c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4758ff9640 1 -- 172.21.15.83:0/3057154084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f474408f510 con 0x7f475c0cf830 2024-05-17T22:04:23.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f475b7fe640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 0x7f473c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f475b7fe640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 0x7f473c071910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f475c15a9a0 tx=0x7f474c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 Processor -- start 2024-05-17T22:04:23.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 -- start start 2024-05-17T22:04:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad38020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cddabd6fa0 con 0x55cddabe8460 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cddab4b2e0 con 0x55cddabd2030 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cddab3ae80 con 0x55cddabf2e00 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39008/0 (socket says 172.21.15.83:39008) 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 -- 172.21.15.83:0/1602958525 learned_addr learned my addr 172.21.15.83:0/1602958525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f475bfff640 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f475b7fe640 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad38020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 -- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 msgr2=0x55cddad38020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad38020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 -- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 msgr2=0x55cddad38aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad38aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.685+0000 7f4753fff640 1 -- 172.21.15.83:0/1602958525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddabe5b80 con 0x55cddabe8460 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4753fff640 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f474405bc90 tx=0x7f474405b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c1f63af512091c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f47517fa640 1 -- 172.21.15.83:0/1602958525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744021040 con 0x55cddabe8460 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f47517fa640 1 -- 172.21.15.83:0/1602958525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744064420 con 0x55cddabe8460 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f47517fa640 1 -- 172.21.15.83:0/1602958525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474408f2d0 con 0x55cddabe8460 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- 172.21.15.83:0/1602958525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 msgr2=0x55cddad38560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f474405bc90 tx=0x7f474405b570 comp rx=0 tx=0).stop 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- 172.21.15.83:0/1602958525 shutdown_connections 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad38560 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1602958525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad38020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- 172.21.15.83:0/1602958525 >> 172.21.15.83:0/1602958525 conn(0x55cdda7fc760 msgr2=0x55cddacf5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- 172.21.15.83:0/1602958525 shutdown_connections 2024-05-17T22:04:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- 172.21.15.83:0/1602958525 wait complete. 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- start start 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 Processor -- start 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddad3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabe8460 0x55cddad3f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cddaaf6eb0 con 0x55cddabf2e00 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cddab3eec0 con 0x55cddabe8460 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cdda804820 con 0x55cddabd2030 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4768dd8000 10 client.0 _ll_register_callbacks cb 0x55cddacdad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39022/0 (socket says 172.21.15.83:39022) 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 -- 172.21.15.83:0/1270868526 learned_addr learned my addr 172.21.15.83:0/1270868526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f4753fff640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabe8460 0x55cddad3f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475b7fe640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddad3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 -- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 msgr2=0x55cddad3f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddad3f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 -- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabe8460 msgr2=0x55cddad3f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabe8460 0x55cddad3f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.689+0000 7f475bfff640 1 -- 172.21.15.83:0/1270868526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddab83ef0 con 0x55cddabf2e00 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f475bfff640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f47540b0040 tx=0x7f475401b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46dd6605c77eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540b2040 con 0x55cddabf2e00 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f47540a5c60 con 0x55cddabf2e00 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4744064420 con 0x55cddabf2e00 2024-05-17T22:04:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f47540c8050 con 0x55cddabf2e00 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 10 client.7029 fetch_fsmap learned FSMap version 66 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55cddab78910 con 0x55cddabf2e00 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744064420 con 0x55cddabf2e00 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f47540cb020 con 0x55cddabf2e00 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 10 client.7029 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 10 client.7029 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 20 client.7029 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7e1ad0 0x55cddad49680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 4 client.7029 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cddaad6cf0 con 0x55cdda7e1ad0 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f475b7fe640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7e1ad0 0x55cddad49680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 10 client.7029 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f475b7fe640 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7e1ad0 0x55cddad49680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9077ccdf10adc3e server_cookie=cb46415fb5c7f0b in_seq=0 out_seq=0 2024-05-17T22:04:23.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 1 -- 172.21.15.83:0/1270868526 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cddaad6cf0 con 0x55cdda7e1ad0 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 10 client.7029 handle_command_reply: tid=0 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 client.7029 shutdown 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 msgr2=0x55cddad3fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f47540b0040 tx=0x7f475401b1b0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 shutdown_connections 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7e1ad0 0x55cddad49680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabf2e00 0x55cddad3fd20 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabe8460 0x55cddad3f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f473a7fc640 0 client.7029 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 --2- 172.21.15.83:0/1270868526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddad3f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 >> 172.21.15.83:0/1270868526 conn(0x55cdda7fc760 msgr2=0x55cddaced4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 shutdown_connections 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 1 -- 172.21.15.83:0/1270868526 wait complete. 2024-05-17T22:04:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.693+0000 7f4768dd8000 20 client.7029 trim_cache size 0 max 16384 2024-05-17T22:04:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 Processor -- start 2024-05-17T22:04:23.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- start start 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddacf0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabf2e00 0x55cddad3f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cddacf2fd0 con 0x55cddabe8460 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f475c2035b0 con 0x55cddabf2e00 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f475c202590 con 0x55cddabd2030 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39028/0 (socket says 172.21.15.83:39028) 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 -- 172.21.15.83:0/3697273482 learned_addr learned my addr 172.21.15.83:0/3697273482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 -- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 msgr2=0x55cddacf0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddacf0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 -- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabf2e00 msgr2=0x55cddad3f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f475bfff640 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabf2e00 0x55cddad3f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabf2e00 0x55cddad3f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 -- 172.21.15.83:0/3697273482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cddaceb1c0 con 0x55cddabe8460 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4753fff640 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f474400d010 tx=0x7f474401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1617014c287756 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4750ff9640 1 -- 172.21.15.83:0/3697273482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744021040 con 0x55cddabe8460 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4750ff9640 1 -- 172.21.15.83:0/3697273482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744064420 con 0x55cddabe8460 2024-05-17T22:04:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4750ff9640 1 -- 172.21.15.83:0/3697273482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744057bc0 con 0x55cddabe8460 2024-05-17T22:04:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- 172.21.15.83:0/3697273482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 msgr2=0x55cddacf09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f474400d010 tx=0x7f474401e000 comp rx=0 tx=0).stop 2024-05-17T22:04:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- 172.21.15.83:0/3697273482 shutdown_connections 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabf2e00 0x55cddad3f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddacf09c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 --2- 172.21.15.83:0/3697273482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabd2030 0x55cddacf0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- 172.21.15.83:0/3697273482 >> 172.21.15.83:0/3697273482 conn(0x55cdda7fc760 msgr2=0x55cddacefd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- 172.21.15.83:0/3697273482 shutdown_connections 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.709+0000 7f4768dd8000 1 -- 172.21.15.83:0/3697273482 wait complete. 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- start start 2024-05-17T22:04:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 Processor -- start 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad44db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad67550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f475c201f20 con 0x55cddabe8460 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f475c201920 con 0x55cddabd2030 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cddabfa260 con 0x55cddabf2e00 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 10 client.0 _ll_register_callbacks cb 0x7f47540b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39040/0 (socket says 172.21.15.83:39040) 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 -- 172.21.15.83:0/4073512544 learned_addr learned my addr 172.21.15.83:0/4073512544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 -- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 msgr2=0x55cddad67550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad67550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 -- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 msgr2=0x55cddad44db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f475b7fe640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad44db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad44db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 -- 172.21.15.83:0/4073512544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cddaaf8670 con 0x55cddabe8460 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4753fff640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f474405ba00 tx=0x7f474401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e245d8e1a62fcfeb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744021040 con 0x55cddabe8460 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f474405af60 con 0x55cddabe8460 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4744064420 con 0x55cddabe8460 2024-05-17T22:04:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f474405b470 con 0x55cddabe8460 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 10 client.7035 fetch_fsmap learned FSMap version 66 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55cddaaf7100 con 0x55cddabe8460 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744064420 con 0x55cddabe8460 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f47440622b0 con 0x55cddabe8460 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 10 client.7035 fetch_fsmap finished waiting for FSMap version 66 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 10 client.7035 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 20 client.7035 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7cfb30 0x55cddad67e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 4 client.7035 mds_command: new command op to 6801 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 --> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55cddab4d230 con 0x55cdda7cfb30 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f475b7fe640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7cfb30 0x55cddad67e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 10 client.7035 ms_handle_connect on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f475b7fe640 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7cfb30 0x55cddad67e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f1a8a27927932f server_cookie=fdd6fb2ae2af71f1 in_seq=0 out_seq=0 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 1 -- 172.21.15.83:0/4073512544 <== mds.0 v2:172.21.15.83:6834/165849096 1 ==== command_reply(tid 0: 0 ) ==== 8+0+143 (crc 0 0 0) 0x55cddab4d230 con 0x55cdda7cfb30 2024-05-17T22:04:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 10 client.7035 handle_command_reply: tid=0 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 client.7035 shutdown 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 msgr2=0x55cddad452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f474405ba00 tx=0x7f474401d000 comp rx=0 tx=0).stop 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 shutdown_connections 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:6834/165849096,v1:172.21.15.83:6836/165849096] conn(0x55cdda7cfb30 0x55cddad67e80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cddabf2e00 0x55cddad67550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cddabe8460 0x55cddad452f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f473a7fc640 0 client.7035 ms_handle_reset on v2:172.21.15.83:6834/165849096 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 --2- 172.21.15.83:0/4073512544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cddabd2030 0x55cddad44db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.713+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 >> 172.21.15.83:0/4073512544 conn(0x55cdda7fc760 msgr2=0x55cddacf0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 shutdown_connections 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4768dd8000 1 -- 172.21.15.83:0/4073512544 wait complete. 2024-05-17T22:04:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4768dd8000 20 client.7035 trim_cache size 0 max 16384 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: "damage_type": "backtrace", 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 612725979, 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1099511627777, 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/testdir/hardlink" 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:04:23.710 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:04:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 msgr2=0x7f473c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 0x7f473c071910 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f475c15a9a0 tx=0x7f474c009040 comp rx=0 tx=0).stop 2024-05-17T22:04:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 msgr2=0x7f475c153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f474400e820 tx=0x7f474400ece0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 shutdown_connections 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f473c06f400 0x7f473c071910 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c15d530 0x7f475c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f475c0cf830 0x7f475c153e80 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 --2- 172.21.15.83:0/3057154084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f475c0cdf40 0x7f475c1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 >> 172.21.15.83:0/3057154084 conn(0x7f475c06cd90 msgr2=0x7f475c075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 shutdown_connections 2024-05-17T22:04:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.717+0000 7f4761b5e640 1 -- 172.21.15.83:0/3057154084 wait complete. 2024-05-17T22:04:23.723 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:04:23.723 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:04:23.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:04:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 Processor -- start 2024-05-17T22:04:23.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 -- start start 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b08a0 0x7f5ddc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ddc06f2d0 con 0x7f5ddc06eec0 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5ddc15c3b0 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5de0c6e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ddc06f5a0 con 0x7f5ddc0b08a0 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39056/0 (socket says 172.21.15.83:39056) 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 -- 172.21.15.83:0/527172176 learned_addr learned my addr 172.21.15.83:0/527172176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5dd9d9b640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b08a0 0x7f5ddc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 -- 172.21.15.83:0/527172176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b08a0 msgr2=0x7f5ddc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b08a0 0x7f5ddc15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.913+0000 7f5ddad9d640 1 -- 172.21.15.83:0/527172176 >> v1:172.21.15.155:6789/0 conn(0x7f5ddc06eec0 legacy=0x7f5ddc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5ddad9d640 1 -- 172.21.15.83:0/527172176 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc06f880 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5ddad9d640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e50 tx=0x7f5dd401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc8daf1b81aeb46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5dd959a640 1 -- 172.21.15.83:0/527172176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40a5770 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5dd959a640 1 -- 172.21.15.83:0/527172176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd40a5910 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5dd959a640 1 -- 172.21.15.83:0/527172176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40a5ce0 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 -- 172.21.15.83:0/527172176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc0b44c0 msgr2=0x7f5ddc0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009e50 tx=0x7f5dd401ca30 comp rx=0 tx=0).stop 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 -- 172.21.15.83:0/527172176 shutdown_connections 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc0b44c0 0x7f5ddc0b48d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/527172176 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b08a0 0x7f5ddc15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 -- 172.21.15.83:0/527172176 >> 172.21.15.83:0/527172176 conn(0x7f5ddc06cd90 msgr2=0x7f5ddc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 -- 172.21.15.83:0/527172176 shutdown_connections 2024-05-17T22:04:23.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.917+0000 7f5de0c6e640 1 -- 172.21.15.83:0/527172176 wait complete. 2024-05-17T22:04:23.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 Processor -- start 2024-05-17T22:04:23.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- start start 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 0x7f5ddc1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b44c0 0x7f5ddc14e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc160a40 con 0x7f5ddc06eec0 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1608c0 con 0x7f5ddc0b08a0 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ddc072e80 con 0x7f5ddc0b44c0 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39070/0 (socket says 172.21.15.83:39070) 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 -- 172.21.15.83:0/832206012 learned_addr learned my addr 172.21.15.83:0/832206012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:23.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dd9d9b640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 0x7f5ddc1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5ddad9d640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b44c0 0x7f5ddc14e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 -- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b44c0 msgr2=0x7f5ddc14e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b44c0 0x7f5ddc14e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 -- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 msgr2=0x7f5ddc1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 0x7f5ddc1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 -- 172.21.15.83:0/832206012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc14d640 con 0x7f5ddc06eec0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dd9d9b640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 0x7f5ddc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dda59c640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00c3c0 tx=0x7f5dcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc017070 con 0x7f5ddc06eec0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dcc0052d0 con 0x7f5ddc06eec0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc14d920 con 0x7f5ddc06eec0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0054c0 con 0x7f5ddc06eec0 2024-05-17T22:04:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc14dee0 con 0x7f5ddc06eec0 2024-05-17T22:04:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.921+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc15be70 con 0x7f5ddc06eec0 2024-05-17T22:04:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.925+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5dcc01a070 con 0x7f5ddc06eec0 2024-05-17T22:04:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.925+0000 7f5dcaffd640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 0x7f5dbc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.925+0000 7f5dd9d9b640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 0x7f5dbc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.925+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f5dcc08eaf0 con 0x7f5ddc06eec0 2024-05-17T22:04:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.925+0000 7f5dd9d9b640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 0x7f5dbc071910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc152870 tx=0x7f5dd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:23.929+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5dcc021020 con 0x7f5ddc06eec0 2024-05-17T22:04:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ddc070d20 con 0x7f5ddc06eec0 2024-05-17T22:04:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5dcaffd640 1 -- 172.21.15.83:0/832206012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) ==== 80+0+122 (secure 0 0 0) 0x7f5dcc05b9e0 con 0x7f5ddc06eec0 2024-05-17T22:04:24.078 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:04:24.078 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6834/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:04:24.078 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 msgr2=0x7f5dbc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 0x7f5dbc071910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc152870 tx=0x7f5dd0008040 comp rx=0 tx=0).stop 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 msgr2=0x7f5ddc1439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00c3c0 tx=0x7f5dcc00c880 comp rx=0 tx=0).stop 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 shutdown_connections 2024-05-17T22:04:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ddc0b44c0 0x7f5ddc14e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dbc06f400 0x7f5dbc071910 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ddc0b08a0 0x7f5ddc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 --2- 172.21.15.83:0/832206012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ddc06eec0 0x7f5ddc1439d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 >> 172.21.15.83:0/832206012 conn(0x7f5ddc06cd90 msgr2=0x7f5ddc0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 shutdown_connections 2024-05-17T22:04:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.089+0000 7f5de0c6e640 1 -- 172.21.15.83:0/832206012 wait complete. 2024-05-17T22:04:24.098 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:04:24.098 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:04:24.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:04:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.289+0000 7f06546fe640 1 Processor -- start 2024-05-17T22:04:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.289+0000 7f06546fe640 1 -- start start 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f06546fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c0b88b0 0x7f064c0d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f06546fe640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c09c710 con 0x7f064c071400 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f06546fe640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c09c9f0 con 0x7f064c09ba00 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f06546fe640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f064c0d1a40 con 0x7f064c0b88b0 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c0b88b0 0x7f064c0d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064c0b88b0 0x7f064c0d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39084/0 (socket says 172.21.15.83:39084) 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f0652c9b640 1 --1- 172.21.15.83:0/1523936548 >> v1:172.21.15.83:6789/0 conn(0x7f064c071400 0x7f064c071810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35390/0 (socket says 172.21.15.83:35390) 2024-05-17T22:04:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 -- 172.21.15.83:0/1523936548 learned_addr learned my addr 172.21.15.83:0/1523936548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 -- 172.21.15.83:0/1523936548 >> v1:172.21.15.155:6789/0 conn(0x7f064c09ba00 legacy=0x7f064c09be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 -- 172.21.15.83:0/1523936548 >> v1:172.21.15.83:6789/0 conn(0x7f064c071400 legacy=0x7f064c071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.293+0000 7f065249a640 1 -- 172.21.15.83:0/1523936548 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c09ccd0 con 0x7f064c0b88b0 2024-05-17T22:04:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f065249a640 1 --2- 172.21.15.83:0/1523936548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c0d11f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f064c0d1bc0 tx=0x7f064801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca909dee8ef87f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f0651498640 1 -- 172.21.15.83:0/1523936548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c072730 con 0x7f064c0b88b0 2024-05-17T22:04:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f0651498640 1 -- 172.21.15.83:0/1523936548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06480a56c0 con 0x7f064c0b88b0 2024-05-17T22:04:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f0651498640 1 -- 172.21.15.83:0/1523936548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06480a5a90 con 0x7f064c0b88b0 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- 172.21.15.83:0/1523936548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 msgr2=0x7f064c0d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 --2- 172.21.15.83:0/1523936548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c0d11f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f064c0d1bc0 tx=0x7f064801c8f0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- 172.21.15.83:0/1523936548 shutdown_connections 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 --2- 172.21.15.83:0/1523936548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c0d11f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- 172.21.15.83:0/1523936548 >> 172.21.15.83:0/1523936548 conn(0x7f064c0d3620 msgr2=0x7f064c0d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- 172.21.15.83:0/1523936548 shutdown_connections 2024-05-17T22:04:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- 172.21.15.83:0/1523936548 wait complete. 2024-05-17T22:04:24.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 Processor -- start 2024-05-17T22:04:24.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.297+0000 7f06546fe640 1 -- start start 2024-05-17T22:04:24.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f064c071400 0x7f064c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 0x7f064c1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f064c16b910 con 0x7f064c0b88b0 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f064c09d040 con 0x7f064c071400 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f064c16ba90 con 0x7f064c09ba00 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39090/0 (socket says 172.21.15.83:39090) 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 -- 172.21.15.83:0/1718574258 learned_addr learned my addr 172.21.15.83:0/1718574258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f065249a640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f064c071400 0x7f064c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0651c99640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 0x7f064c1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 -- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 msgr2=0x7f064c1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 0x7f064c1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 -- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f064c071400 msgr2=0x7f064c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f064c071400 0x7f064c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 -- 172.21.15.83:0/1718574258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c155a80 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0651c99640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 0x7f064c1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0652c9b640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f064400c3c0 tx=0x7f064400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440074e0 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644007680 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c0c8510 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644004a10 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c0c8ad0 con 0x7f064c0b88b0 2024-05-17T22:04:24.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c0d11f0 con 0x7f064c0b88b0 2024-05-17T22:04:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f064401a020 con 0x7f064c0b88b0 2024-05-17T22:04:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 0x7f0630071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f064408ea30 con 0x7f064c0b88b0 2024-05-17T22:04:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.301+0000 7f065249a640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 0x7f0630071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.305+0000 7f065249a640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 0x7f0630071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f06480ae000 tx=0x7f06480a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.305+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f064405b990 con 0x7f064c0b88b0 2024-05-17T22:04:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.477+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f064c0cfc80 con 0x7f064c0b88b0 2024-05-17T22:04:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.477+0000 7f0642ffd640 1 -- 172.21.15.83:0/1718574258 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) ==== 80+0+122 (secure 0 0 0) 0x7f064405b290 con 0x7f064c0b88b0 2024-05-17T22:04:24.468 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:04:24.469 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6834/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:04:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 msgr2=0x7f0630071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 0x7f0630071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f06480ae000 tx=0x7f06480a7070 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 msgr2=0x7f064c158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f064400c3c0 tx=0x7f064400c880 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 shutdown_connections 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064c0b88b0 0x7f064c158af0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f064c09ba00 0x7f064c1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f063006f400 0x7f0630071910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 --2- 172.21.15.83:0/1718574258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f064c071400 0x7f064c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 >> 172.21.15.83:0/1718574258 conn(0x7f064c0d3620 msgr2=0x7f064c072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 shutdown_connections 2024-05-17T22:04:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:24.481+0000 7f06546fe640 1 -- 172.21.15.83:0/1718574258 wait complete. 2024-05-17T22:04:24.489 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:04:24.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:24.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:24.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[43967]: fuse finished with error 0 and tester_r 0 2024-05-17T22:04:24.500 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:04:30.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:04:30.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:30.602 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:04:30.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:04:30.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.825+0000 7f46a2f41640 1 Processor -- start 2024-05-17T22:04:30.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.825+0000 7f46a2f41640 1 -- start start 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a2f41640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f469c144080 0x7f469c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a2f41640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f469c073370 0x7f469c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a2f41640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f469c073780 con 0x7f469c0d1040 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a2f41640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f469c15e790 con 0x7f469c073370 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a2f41640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f469c15ea40 con 0x7f469c144080 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f469c073370 0x7f469c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f469c073370 0x7f469c06f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43688/0 (socket says 172.21.15.83:43688) 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 -- 172.21.15.83:0/3900313328 learned_addr learned my addr 172.21.15.83:0/3900313328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f469bfff640 1 --2- 172.21.15.83:0/3900313328 >> v2:172.21.15.155:3300/0 conn(0x7f469c144080 0x7f469c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 -- 172.21.15.83:0/3900313328 >> v2:172.21.15.155:3300/0 conn(0x7f469c144080 msgr2=0x7f469c072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:30.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 --2- 172.21.15.83:0/3900313328 >> v2:172.21.15.155:3300/0 conn(0x7f469c144080 0x7f469c072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 -- 172.21.15.83:0/3900313328 >> v1:172.21.15.155:6789/0 conn(0x7f469c0d1040 legacy=0x7f469c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:30.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.829+0000 7f46a14de640 1 -- 172.21.15.83:0/3900313328 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c15ed20 con 0x7f469c073370 2024-05-17T22:04:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a14de640 1 --2- 172.21.15.83:0/3900313328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c073370 0x7f469c06f730 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f469c0b6b10 tx=0x7f469401c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf5c09d2a9e1caa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f469b7fe640 1 -- 172.21.15.83:0/3900313328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a5660 con 0x7f469c073370 2024-05-17T22:04:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f469b7fe640 1 -- 172.21.15.83:0/3900313328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46940a5800 con 0x7f469c073370 2024-05-17T22:04:30.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f469b7fe640 1 -- 172.21.15.83:0/3900313328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a5bd0 con 0x7f469c073370 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- 172.21.15.83:0/3900313328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c073370 msgr2=0x7f469c06f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3900313328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c073370 0x7f469c06f730 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f469c0b6b10 tx=0x7f469401c7e0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- 172.21.15.83:0/3900313328 shutdown_connections 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3900313328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c073370 0x7f469c06f730 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3900313328 >> v2:172.21.15.155:3300/0 conn(0x7f469c144080 0x7f469c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- 172.21.15.83:0/3900313328 >> 172.21.15.83:0/3900313328 conn(0x7f469c06cd90 msgr2=0x7f469c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:30.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- 172.21.15.83:0/3900313328 shutdown_connections 2024-05-17T22:04:30.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- 172.21.15.83:0/3900313328 wait complete. 2024-05-17T22:04:30.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 Processor -- start 2024-05-17T22:04:30.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.833+0000 7f46a2f41640 1 -- start start 2024-05-17T22:04:30.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c073370 0x7f469c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 0x7f469c13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f469c070480 con 0x7f469c144080 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f469c144490 con 0x7f469c073370 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f469c070600 con 0x7f469c0d1040 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43702/0 (socket says 172.21.15.83:43702) 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 -- 172.21.15.83:0/3383972650 learned_addr learned my addr 172.21.15.83:0/3383972650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a0cdd640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c073370 0x7f469c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f469bfff640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 0x7f469c13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 -- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 msgr2=0x7f469c13a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 0x7f469c13a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 -- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c073370 msgr2=0x7f469c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c073370 0x7f469c0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 -- 172.21.15.83:0/3383972650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c13d320 con 0x7f469c144080 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f469bfff640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 0x7f469c13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a14de640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f469401ae90 tx=0x7f4694019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694009510 con 0x7f469c144080 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46940be070 con 0x7f469c144080 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c13b970 con 0x7f469c144080 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940b88d0 con 0x7f469c144080 2024-05-17T22:04:30.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c13bf30 con 0x7f469c144080 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c13c310 con 0x7f469c144080 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f46940b8a70 con 0x7f469c144080 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.837+0000 7f46997fa640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 0x7f4688071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.841+0000 7f46a0cdd640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 0x7f4688071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.841+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7f469412a620 con 0x7f469c144080 2024-05-17T22:04:30.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.841+0000 7f46a0cdd640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 0x7f4688071940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f468c0098e0 tx=0x7f468c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:30.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:30.841+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f469c13c310 con 0x7f469c144080 2024-05-17T22:04:30.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:04:30.966 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:04:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.037+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f469c070990 con 0x7f469c144080 2024-05-17T22:04:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.037+0000 7f46997fa640 1 -- 172.21.15.83:0/3383972650 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5187 (secure 0 0 0) 0x7f46940c7020 con 0x7f469c144080 2024-05-17T22:04:31.027 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:31.027 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:03:23.463499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6801},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6801":{"gid":6801,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.83:6836/165849096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":165849096},{"type":"v1","addr":"172.21.15.83:6836","nonce":165849096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6801,"qdb_cluster":[6801]},"id":6}]} 2024-05-17T22:04:31.027 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 66 2024-05-17T22:04:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 msgr2=0x7f4688071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 0x7f4688071940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f468c0098e0 tx=0x7f468c009060 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 msgr2=0x7f469c13ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f469401ae90 tx=0x7f4694019580 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 shutdown_connections 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f469c144080 0x7f469c13ae00 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c0d1040 0x7f469c13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f468806f430 0x7f4688071940 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 --2- 172.21.15.83:0/3383972650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c073370 0x7f469c0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 >> 172.21.15.83:0/3383972650 conn(0x7f469c06cd90 msgr2=0x7f469c1466f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 shutdown_connections 2024-05-17T22:04:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.041+0000 7f46a2f41640 1 -- 172.21.15.83:0/3383972650 wait complete. 2024-05-17T22:04:31.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T22:04:31.227 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:04:31.228 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:04:31.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:04:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.389+0000 7fc4f80c5640 1 Processor -- start 2024-05-17T22:04:31.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.389+0000 7fc4f80c5640 1 -- start start 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f80c5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc4f00b3d00 0x7fc4f00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f80c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f0073590 0x7fc4f01447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f80c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00d10d0 con 0x7fc4f0143f20 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f80c5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00b4110 con 0x7fc4f0073590 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f80c5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f0070770 con 0x7fc4f00b3d00 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f0073590 0x7fc4f01447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f5660640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc4f0143f20 0x7fc4f0072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51548/0 (socket says 172.21.15.83:51548) 2024-05-17T22:04:31.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4f0073590 0x7fc4f01447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43708/0 (socket says 172.21.15.83:43708) 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f5660640 1 -- 172.21.15.83:0/281784429 learned_addr learned my addr 172.21.15.83:0/281784429 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f5e61640 1 --2- 172.21.15.83:0/281784429 >> v2:172.21.15.155:3300/0 conn(0x7fc4f00b3d00 0x7fc4f00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 -- 172.21.15.83:0/281784429 >> v2:172.21.15.155:3300/0 conn(0x7fc4f00b3d00 msgr2=0x7fc4f00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 --2- 172.21.15.83:0/281784429 >> v2:172.21.15.155:3300/0 conn(0x7fc4f00b3d00 0x7fc4f00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 -- 172.21.15.83:0/281784429 >> v1:172.21.15.83:6789/0 conn(0x7fc4f0143f20 legacy=0x7fc4f0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f4e5f640 1 -- 172.21.15.83:0/281784429 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086573247 0 0) 0x7fc4f00d10d0 con 0x7fc4f0143f20 2024-05-17T22:04:31.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.393+0000 7fc4f6662640 1 -- 172.21.15.83:0/281784429 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f0070120 con 0x7fc4f0073590 2024-05-17T22:04:31.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f6662640 1 --2- 172.21.15.83:0/281784429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f01447f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80097c0 tx=0x7fc4e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34465075177fd9bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f4e5f640 1 -- 172.21.15.83:0/281784429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80b0070 con 0x7fc4f0073590 2024-05-17T22:04:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f4e5f640 1 -- 172.21.15.83:0/281784429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e80ab600 con 0x7fc4f0073590 2024-05-17T22:04:31.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f4e5f640 1 -- 172.21.15.83:0/281784429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80ab9d0 con 0x7fc4f0073590 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- 172.21.15.83:0/281784429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 msgr2=0x7fc4f01447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/281784429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f01447f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80097c0 tx=0x7fc4e801c460 comp rx=0 tx=0).stop 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- 172.21.15.83:0/281784429 shutdown_connections 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/281784429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f01447f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/281784429 >> v2:172.21.15.155:3300/0 conn(0x7fc4f00b3d00 0x7fc4f00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- 172.21.15.83:0/281784429 >> 172.21.15.83:0/281784429 conn(0x7fc4f006cd90 msgr2=0x7fc4f00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- 172.21.15.83:0/281784429 shutdown_connections 2024-05-17T22:04:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- 172.21.15.83:0/281784429 wait complete. 2024-05-17T22:04:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 Processor -- start 2024-05-17T22:04:31.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.397+0000 7fc4f80c5640 1 -- start start 2024-05-17T22:04:31.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4f00b3d00 0x7fc4f0139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 0x7fc4f0148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0070c00 con 0x7fc4f0073590 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0145540 con 0x7fc4f0143f20 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f015bb80 con 0x7fc4f00b3d00 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43712/0 (socket says 172.21.15.83:43712) 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 -- 172.21.15.83:0/3143144194 learned_addr learned my addr 172.21.15.83:0/3143144194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f6662640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 0x7fc4f0148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5660640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4f00b3d00 0x7fc4f0139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 -- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4f00b3d00 msgr2=0x7fc4f0139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4f00b3d00 0x7fc4f0139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 -- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 msgr2=0x7fc4f0148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 0x7fc4f0148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 -- 172.21.15.83:0/3143144194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f013e860 con 0x7fc4f0073590 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f6662640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 0x7fc4f0148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5e61640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e40 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014620 con 0x7fc4f0073590 2024-05-17T22:04:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec019070 con 0x7fc4f0073590 2024-05-17T22:04:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f013eb40 con 0x7fc4f0073590 2024-05-17T22:04:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014a00 con 0x7fc4f0073590 2024-05-17T22:04:31.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f013f100 con 0x7fc4f0073590 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f01447f0 con 0x7fc4f0073590 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4ec014ba0 con 0x7fc4f0073590 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 0x7fc4d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5853+0+0 (secure 0 0 0) 0x7fc4ec08fc30 con 0x7fc4f0073590 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.401+0000 7fc4f5660640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 0x7fc4d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.405+0000 7fc4f5660640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 0x7fc4d4071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01fa130 tx=0x7fc4e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.405+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc4ec05ca90 con 0x7fc4f0073590 2024-05-17T22:04:31.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.609+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc4f00b0d80 con 0x7fc4f0073590 2024-05-17T22:04:31.598 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:04:31.609+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:04:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.609+0000 7fc4e67fc640 1 -- 172.21.15.83:0/3143144194 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7fc4ec05c390 con 0x7fc4f0073590 2024-05-17T22:04:31.602 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.613+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 msgr2=0x7fc4d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.613+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 0x7fc4d4071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01fa130 tx=0x7fc4e0009040 comp rx=0 tx=0).stop 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.613+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 msgr2=0x7fc4f013ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.613+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e40 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 shutdown_connections 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc4d406f400 0x7fc4d4071910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4f0143f20 0x7fc4f0148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4f00b3d00 0x7fc4f0139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 --2- 172.21.15.83:0/3143144194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4f0073590 0x7fc4f013ddc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 >> 172.21.15.83:0/3143144194 conn(0x7fc4f006cd90 msgr2=0x7fc4f00cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 shutdown_connections 2024-05-17T22:04:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.617+0000 7fc4f80c5640 1 -- 172.21.15.83:0/3143144194 wait complete. 2024-05-17T22:04:31.610 INFO:tasks.ceph.mds.a.smithi083.stderr: -4601> 2024-05-17T22:03:19.450+0000 7f5dfbecc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:04:31.611 INFO:tasks.ceph.mds.a.smithi083.stderr: -4345> 2024-05-17T22:03:20.854+0000 7f5df7ec4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-17T22:04:31.611 INFO:tasks.ceph.mds.a.smithi083.stderr: -4183> 2024-05-17T22:03:21.110+0000 7f5df7ec4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0) n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x564d2e646100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-17T22:04:31.615 INFO:tasks.ceph.mds.a.smithi083.stderr: -2398> 2024-05-17T22:03:52.693+0000 7f5df7ec4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv19 ap=3 DIRTYPARENT f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)->f(v1 m2024-05-17T22:03:04.213534+0000) n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)->n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x564d2e646100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-17T22:03:04.213534+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-17T22:03:04.326122+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-17T22:04:31.627 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-17T22:04:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.697+0000 7f2a6459f8c0 1 -- start start 2024-05-17T22:04:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.697+0000 7f2a6459f8c0 1 Processor -- start 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f5a80 0x55aa85451080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f2d80 0x55aa85451b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aa85478480 con 0x55aa846f3200 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa85478600 con 0x55aa846f2d80 2024-05-17T22:04:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55aa85478780 con 0x55aa846f5a80 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43728/0 (socket says 172.21.15.83:43728) 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 -- 172.21.15.83:0/1925715582 learned_addr learned my addr 172.21.15.83:0/1925715582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f5a80 0x55aa85451080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6355f640 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f2d80 0x55aa85451b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 -- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f5a80 msgr2=0x55aa85451080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f5a80 0x55aa85451080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 -- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f2d80 msgr2=0x55aa85451b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f2d80 0x55aa85451b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 -- 172.21.15.83:0/1925715582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa806741e0 con 0x55aa846f3200 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6255d640 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x55aa854e59e0 tx=0x55aa84792960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65daad56d7eb0f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a60559640 1 -- 172.21.15.83:0/1925715582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b4b60 con 0x55aa846f3200 2024-05-17T22:04:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a60559640 1 -- 172.21.15.83:0/1925715582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55aa846c81c0 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a60559640 1 -- 172.21.15.83:0/1925715582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b41a0 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa80675c20 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55aa806754a0 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 4 waiting for MDS map... 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a60559640 1 -- 172.21.15.83:0/1925715582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x55aa855ae400 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a60559640 1 -- 172.21.15.83:0/1925715582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 67) ==== 2885+0+0 (secure 0 0 0) 0x55aa846d5180 con 0x55aa846f3200 2024-05-17T22:04:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 4 Got MDS map 67 2024-05-17T22:04:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 4 datascan.main: Using metadata pool output 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 Processor -- start 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.701+0000 7f2a6459f8c0 1 -- start start 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85451600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 0x55aa85452100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aa846df980 con 0x55aa846f4d00 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa85478f00 con 0x55aa846f5180 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55aa85479080 con 0x55aa846f4880 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43744/0 (socket says 172.21.15.83:43744) 2024-05-17T22:04:31.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 -- 172.21.15.83:0/3277682741 learned_addr learned my addr 172.21.15.83:0/3277682741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6355f640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85451600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6255d640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 0x55aa85452100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 -- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 msgr2=0x55aa85451600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85451600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 -- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 msgr2=0x55aa85452100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 0x55aa85452100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 -- 172.21.15.83:0/3277682741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa806741e0 con 0x55aa846f4d00 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6255d640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 0x55aa85452100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55aa855bafc0 tx=0x55aa855d4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d360d405aa351c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a5e555640 1 -- 172.21.15.83:0/3277682741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b4b60 con 0x55aa846f4d00 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a5e555640 1 -- 172.21.15.83:0/3277682741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55aa846c8c40 con 0x55aa846f4d00 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a5e555640 1 -- 172.21.15.83:0/3277682741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b41a0 con 0x55aa846f4d00 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/3277682741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 msgr2=0x55aa85450000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55aa855bafc0 tx=0x55aa855d4c00 comp rx=0 tx=0).stop 2024-05-17T22:04:31.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/3277682741 shutdown_connections 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f5180 0x55aa85452100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f4d00 0x55aa85450000 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/3277682741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85451600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/3277682741 >> 172.21.15.83:0/3277682741 conn(0x55aa846f2900 msgr2=0x55aa855a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/3277682741 shutdown_connections 2024-05-17T22:04:31.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/3277682741 wait complete. 2024-05-17T22:04:31.697 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:04:31.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 Processor -- start 2024-05-17T22:04:31.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.705+0000 7f2a6459f8c0 1 -- start start 2024-05-17T22:04:31.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85453180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f4d00 0x55aa85452c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aa85479380 con 0x55aa846f5180 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55aa85479200 con 0x55aa846f4d00 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55aa85479500 con 0x55aa846f4880 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43756/0 (socket says 172.21.15.83:43756) 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 learned_addr learned my addr 172.21.15.83:0/2742696093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85453180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f4d00 0x55aa85452c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 msgr2=0x55aa85453180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f4880 0x55aa85453180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f4d00 msgr2=0x55aa85452c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f4d00 0x55aa85452c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa806752c0 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55aa855ba2d0 tx=0x55aa847925a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a5cd52640 1 -- 172.21.15.83:0/2742696093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b44e0 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a5cd52640 1 -- 172.21.15.83:0/2742696093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55aa846c8e00 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a5cd52640 1 -- 172.21.15.83:0/2742696093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55aa846b4340 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa80675e00 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa80675c20 con 0x55aa846f5180 2024-05-17T22:04:31.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.709+0000 7f2a6459f8c0 4 datascan.init: resolving metadata pool 12 2024-05-17T22:04:31.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a5cd52640 1 -- 172.21.15.83:0/2742696093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55aa849c8c00 con 0x55aa846f5180 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a5cd52640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 0x55aa85453700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a5cd52640 1 -- 172.21.15.83:0/2742696093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x55aa849d0800 con 0x55aa846f5180 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6459f8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 0x55aa85453700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6459f8c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6459f8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 0x55aa85452100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca000 con 0x55aa846f5600 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 0x55aa85453700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55aa847947b0 tx=0x55aa855d4ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 0x55aa85452100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.713+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 0x55aa85452100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.717+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(1 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.717+0000 7f2a5dd54640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 0x55aa84ad8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.717+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9c00 con 0x55aa84ad0000 2024-05-17T22:04:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.717+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 0x55aa84ad8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.721+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 0x55aa84ad8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(2 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a5e555640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55aa84ad0480 0x55aa84ad8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9400 con 0x55aa84ad0480 2024-05-17T22:04:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55aa84ad0480 0x55aa84ad8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55aa84ad0480 0x55aa84ad8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(3 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a5dd54640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55aa84ad0900 0x55aa84ad8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca000 con 0x55aa84ad0900 2024-05-17T22:04:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55aa84ad0900 0x55aa84ad8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.725+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55aa84ad0900 0x55aa84ad8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.729+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(4 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.729+0000 7f2a5e555640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55aa84ad1200 0x55aa84ad9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.729+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb400 con 0x55aa84ad1200 2024-05-17T22:04:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55aa84ad1200 0x55aa84ad9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55aa84ad1200 0x55aa84ad9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(5 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9800 con 0x55aa84ad1200 2024-05-17T22:04:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(6 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb800 con 0x55aa846f5600 2024-05-17T22:04:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(7 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a5dd54640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55aa84ad0d80 0x55aa84ad9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb000 con 0x55aa84ad0d80 2024-05-17T22:04:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55aa84ad0d80 0x55aa84ad9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a62d5e640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55aa84ad0d80 0x55aa84ad9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(8 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.733+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb400 con 0x55aa84ad0000 2024-05-17T22:04:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(9 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9c00 con 0x55aa84ad0000 2024-05-17T22:04:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(10 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5e555640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55aa84ad1680 0x55aa84ad9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acbc00 con 0x55aa84ad1680 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55aa84ad1680 0x55aa84ad9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6255d640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55aa84ad1680 0x55aa84ad9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(11 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad1680 2024-05-17T22:04:31.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9800 con 0x55aa84ad0000 2024-05-17T22:04:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(12 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa000 con 0x55aa84ad0480 2024-05-17T22:04:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(13 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb000 con 0x55aa84ad1200 2024-05-17T22:04:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 3 ==== osd_op_reply(14 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa400 con 0x55aa84ad0000 2024-05-17T22:04:31.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(15 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb400 con 0x55aa84ad1200 2024-05-17T22:04:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.737+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 4 ==== osd_op_reply(16 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa800 con 0x55aa84ad0d80 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(17 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb800 con 0x55aa84ad0480 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(18 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afac00 con 0x55aa84ad1680 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(19 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad1680 2024-05-17T22:04:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afbc00 con 0x55aa84ad1200 2024-05-17T22:04:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 5 ==== osd_op_reply(20 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9400 con 0x55aa84ad1200 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 6 ==== osd_op_reply(21 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa000 con 0x55aa84ad0480 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(22 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9c00 con 0x55aa84ad0480 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(23 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa800 con 0x55aa84ad0d80 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(24 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb800 con 0x55aa84ad0900 2024-05-17T22:04:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(25 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.741+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb000 con 0x55aa846f5600 2024-05-17T22:04:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.745+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 3 ==== osd_op_reply(26 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.745+0000 7f2a5e555640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55aa84ad1b00 0x55aa84ada100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.745+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb000 con 0x55aa84ad1b00 2024-05-17T22:04:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.745+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55aa84ad1b00 0x55aa84ada100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.745+0000 7f2a6355f640 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55aa84ad1b00 0x55aa84ada100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(27 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1b00 2024-05-17T22:04:31.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa400 con 0x55aa84ad0000 2024-05-17T22:04:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 6 ==== osd_op_reply(28 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb400 con 0x55aa846f5600 2024-05-17T22:04:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 4 ==== osd_op_reply(29 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9800 con 0x55aa84ad1200 2024-05-17T22:04:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 7 ==== osd_op_reply(30 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb400 con 0x55aa84ad0900 2024-05-17T22:04:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(31 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb000 con 0x55aa84ad0480 2024-05-17T22:04:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(32 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.749+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afbc00 con 0x55aa84ad0000 2024-05-17T22:04:31.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 7 ==== osd_op_reply(33 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb400 con 0x55aa84ad0900 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(34 [pgnls start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa400 con 0x55aa84ad0480 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(35 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb800 con 0x55aa84ad1b00 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(36 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1b00 2024-05-17T22:04:31.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9400 con 0x55aa84ad0d80 2024-05-17T22:04:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(37 [pgnls start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa000 con 0x55aa84ad0000 2024-05-17T22:04:31.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 8 ==== osd_op_reply(38 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9800 con 0x55aa84ad1b00 2024-05-17T22:04:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(39 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1b00 2024-05-17T22:04:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afa800 con 0x55aa84ad1200 2024-05-17T22:04:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 8 ==== osd_op_reply(40 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.753+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa855b9c00 con 0x55aa84ad0900 2024-05-17T22:04:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(41 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb000 con 0x55aa84ad0900 2024-05-17T22:04:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 6 ==== osd_op_reply(42 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca000 con 0x55aa84ad0900 2024-05-17T22:04:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 7 ==== osd_op_reply(43 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06000 con 0x55aa84ad0480 2024-05-17T22:04:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(44 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acbc00 con 0x55aa84ad0900 2024-05-17T22:04:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 8 ==== osd_op_reply(45 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06400 con 0x55aa84ad1680 2024-05-17T22:04:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(46 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad1680 2024-05-17T22:04:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb800 con 0x55aa846f5600 2024-05-17T22:04:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 5 ==== osd_op_reply(47 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.757+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06800 con 0x55aa846f5600 2024-05-17T22:04:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 6 ==== osd_op_reply(48 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb400 con 0x55aa84ad1680 2024-05-17T22:04:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(49 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad1680 2024-05-17T22:04:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06c00 con 0x55aa84ad0480 2024-05-17T22:04:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 9 ==== osd_op_reply(50 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afac00 con 0x55aa846f5600 2024-05-17T22:04:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 7 ==== osd_op_reply(51 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b07000 con 0x55aa846f5600 2024-05-17T22:04:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 8 ==== osd_op_reply(52 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb400 con 0x55aa84ad0d80 2024-05-17T22:04:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(53 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.761+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b07400 con 0x55aa84ad1200 2024-05-17T22:04:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.2 v2:172.21.15.83:6808/3743214759 9 ==== osd_op_reply(54 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad1200 2024-05-17T22:04:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06000 con 0x55aa84ad0900 2024-05-17T22:04:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 9 ==== osd_op_reply(55 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b07800 con 0x55aa84ad0900 2024-05-17T22:04:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 10 ==== osd_op_reply(56 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06800 con 0x55aa846f5600 2024-05-17T22:04:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 9 ==== osd_op_reply(57 [pgnls start_epoch 85 out=81b] v81'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca400 con 0x55aa846f5600 2024-05-17T22:04:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6459f8c0 10 datascan.operator(): handling object 1099511627777.0 2024-05-17T22:04:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.765+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca000 con 0x55aa846f5600 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.769+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.7 v2:172.21.15.155:6816/3020759009 11 ==== osd_op_reply(59 10000000001.00000000 [call] v85'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55aa84782000 con 0x55aa846f5600 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.769+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84afb000 con 0x55aa84ad0480 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.769+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.1 v2:172.21.15.83:6800/3404427378 10 ==== osd_op_reply(60 [pgnls start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0480 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.769+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b06c00 con 0x55aa84ad0d80 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(61 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b07000 con 0x55aa84ad0d80 2024-05-17T22:04:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 <== osd.5 v2:172.21.15.155:6808/758721566 7 ==== osd_op_reply(62 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782000 con 0x55aa84ad0d80 2024-05-17T22:04:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84b07c00 con 0x55aa84ad0000 2024-05-17T22:04:31.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 9 ==== osd_op_reply(63 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca400 con 0x55aa84ad0900 2024-05-17T22:04:31.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a6255d640 1 -- 172.21.15.83:0/2742696093 <== osd.4 v2:172.21.15.155:6824/505763805 11 ==== osd_op_reply(64 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782780 con 0x55aa84ad0900 2024-05-17T22:04:31.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a5e555640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84acb000 con 0x55aa84ad0000 2024-05-17T22:04:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 10 ==== osd_op_reply(65 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.773+0000 7f2a5dd54640 1 -- 172.21.15.83:0/2742696093 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x55aa84aca000 con 0x55aa84ad0000 2024-05-17T22:04:31.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6355f640 1 -- 172.21.15.83:0/2742696093 <== osd.6 v2:172.21.15.155:6800/1025803281 11 ==== osd_op_reply(66 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55aa84782500 con 0x55aa84ad0000 2024-05-17T22:04:31.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55aa84ad1680 msgr2=0x55aa84ad9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55aa84ad1680 0x55aa84ad9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55aa84ad0480 msgr2=0x55aa84ad8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55aa84ad0480 0x55aa84ad8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55aa84ad1200 msgr2=0x55aa84ad9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55aa84ad1200 0x55aa84ad9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55aa84ad1b00 msgr2=0x55aa84ada100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55aa84ad1b00 0x55aa84ada100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55aa84ad0900 msgr2=0x55aa84ad8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55aa84ad0900 0x55aa84ad8b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55aa84ad0d80 msgr2=0x55aa84ad9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55aa84ad0d80 0x55aa84ad9080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 msgr2=0x55aa84ad8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 0x55aa84ad8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 reap_dead start 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 reap_dead start 2024-05-17T22:04:31.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a62d5e640 1 -- 172.21.15.83:0/2742696093 reap_dead start 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 msgr2=0x55aa85452100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 0x55aa85452100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 msgr2=0x55aa85453700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 0x55aa85453700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55aa847947b0 tx=0x55aa855d4ea0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 msgr2=0x55aa85452680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55aa855ba2d0 tx=0x55aa847925a0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 shutdown_connections 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55aa846f5600 0x55aa85452100 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f5180 0x55aa85452680 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55aa84ad0000 0x55aa84ad8000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/2742696093 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55aa846f3f80 0x55aa85453700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 >> 172.21.15.83:0/2742696093 conn(0x55aa846f2900 msgr2=0x55aa855a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 shutdown_connections 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/2742696093 wait complete. 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 msgr2=0x55aa85450b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x55aa854e59e0 tx=0x55aa84792960 comp rx=0 tx=0).stop 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 shutdown_connections 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa846f2d80 0x55aa85451b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa846f3200 0x55aa85450b00 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 --2- 172.21.15.83:0/1925715582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55aa846f5a80 0x55aa85451080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 >> 172.21.15.83:0/1925715582 conn(0x55aa846f4400 msgr2=0x55aa855a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 shutdown_connections 2024-05-17T22:04:31.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.777+0000 7f2a6459f8c0 1 -- 172.21.15.83:0/1925715582 wait complete. 2024-05-17T22:04:31.773 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.146225, result: 2024-05-17T22:04:31.774 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-17T22:04:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.837+0000 7fd404c9c8c0 1 -- start start 2024-05-17T22:04:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.837+0000 7fd404c9c8c0 1 Processor -- start 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e75680 0x559a41f76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e75f80 0x559a41f76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a42be4180 con 0x559a41e75b00 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a42be4300 con 0x559a41e75680 2024-05-17T22:04:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559a42be4480 con 0x559a41e75f80 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43774/0 (socket says 172.21.15.83:43774) 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 -- 172.21.15.83:0/4129773690 learned_addr learned my addr 172.21.15.83:0/4129773690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd40345b640 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e75680 0x559a41f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd403c5c640 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e75f80 0x559a41f76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 -- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e75f80 msgr2=0x559a41f76b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e75f80 0x559a41f76b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 -- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e75680 msgr2=0x559a41f76580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e75680 0x559a41f76580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 -- 172.21.15.83:0/4129773690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3ddf61e0 con 0x559a41e75b00 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd402c5a640 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x559a42c63c20 tx=0x559a41f15ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72043a7942cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd400c56640 1 -- 172.21.15.83:0/4129773690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e364e0 con 0x559a41e75b00 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd400c56640 1 -- 172.21.15.83:0/4129773690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559a41e4a1c0 con 0x559a41e75b00 2024-05-17T22:04:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd400c56640 1 -- 172.21.15.83:0/4129773690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e36b60 con 0x559a41e75b00 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3ddf74a0 con 0x559a41e75b00 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x559a3ddf7a40 con 0x559a41e75b00 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 4 waiting for MDS map... 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd400c56640 1 -- 172.21.15.83:0/4129773690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x559a42d26600 con 0x559a41e75b00 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd400c56640 1 -- 172.21.15.83:0/4129773690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 67) ==== 2885+0+0 (secure 0 0 0) 0x559a41e57180 con 0x559a41e75b00 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 4 Got MDS map 67 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 4 datascan.main: Using metadata pool output 2024-05-17T22:04:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:04:31.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 Processor -- start 2024-05-17T22:04:31.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.841+0000 7fd404c9c8c0 1 -- start start 2024-05-17T22:04:31.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a42be4600 con 0x559a41e76880 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a42be4780 con 0x559a41e74900 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559a42be4900 con 0x559a41e74d80 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43788/0 (socket says 172.21.15.83:43788) 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 -- 172.21.15.83:0/1197774911 learned_addr learned my addr 172.21.15.83:0/1197774911 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 -- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 msgr2=0x559a41f77080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 -- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 msgr2=0x559a41f77600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 -- 172.21.15.83:0/1197774911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3ddf7c20 con 0x559a41e76880 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x559a42d3b0b0 tx=0x559a41f15f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8c01787b3a6a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd3fec52640 1 -- 172.21.15.83:0/1197774911 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e36340 con 0x559a41e76880 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd3fec52640 1 -- 172.21.15.83:0/1197774911 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559a41e4ae00 con 0x559a41e76880 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd3fec52640 1 -- 172.21.15.83:0/1197774911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e364e0 con 0x559a41e76880 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/1197774911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 msgr2=0x559a41f77b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x559a42d3b0b0 tx=0x559a41f15f80 comp rx=0 tx=0).stop 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/1197774911 shutdown_connections 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/1197774911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/1197774911 >> 172.21.15.83:0/1197774911 conn(0x559a41e75200 msgr2=0x559a4212a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/1197774911 shutdown_connections 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/1197774911 wait complete. 2024-05-17T22:04:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 Processor -- start 2024-05-17T22:04:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- start start 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a42be5c80 con 0x559a41e76880 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559a42be5b00 con 0x559a41e74900 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd404c9c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559a42be5980 con 0x559a41e74d80 2024-05-17T22:04:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.845+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43800/0 (socket says 172.21.15.83:43800) 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 learned_addr learned my addr 172.21.15.83:0/545557482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 msgr2=0x559a41f77080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 msgr2=0x559a41f77600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e74900 0x559a41f77600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a3ddf61e0 con 0x559a41e76880 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e74d80 0x559a41f77080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x559a42d3bd70 tx=0x559a42d44c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 -- 172.21.15.83:0/545557482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e36b60 con 0x559a41e76880 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 -- 172.21.15.83:0/545557482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559a41e4ac40 con 0x559a41e76880 2024-05-17T22:04:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559a3ddf74a0 con 0x559a41e76880 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 -- 172.21.15.83:0/545557482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a41e36000 con 0x559a41e76880 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3ddf7e00 con 0x559a41e76880 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 4 datascan.init: resolving metadata pool 12 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 -- 172.21.15.83:0/545557482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x559a4213ac00 con 0x559a41e76880 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 0x559a41f78100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd3fd44f640 1 -- 172.21.15.83:0/545557482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x559a42140600 con 0x559a41e76880 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 0x559a41f78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x559a41e76d00 0x559a41f78680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a41e76d00 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 0x559a41f78100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x559a41f16660 tx=0x559a41f14900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x559a41e76d00 0x559a41f78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.849+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x559a41e76d00 0x559a41f78680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559a41e77180 0x559a41f78c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a41e77180 2024-05-17T22:04:31.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559a41e77180 0x559a41f78c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559a41e77180 0x559a41f78c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x559a41e77600 0x559a41f79180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c800 con 0x559a41e77600 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x559a41e77600 0x559a41f79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.853+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x559a41e77600 0x559a41f79180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a41e77180 2024-05-17T22:04:31.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fec52640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x559a42258000 0x559a42260000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c800 con 0x559a42258000 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x559a42258000 0x559a42260000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x559a42258000 0x559a42260000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fe451640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x559a42258480 0x559a42260580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a42258480 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x559a42258480 0x559a42260580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x559a42258480 0x559a42260580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fec52640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x559a42258900 0x559a42260b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.857+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a42258900 2024-05-17T22:04:31.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x559a42258900 0x559a42260b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x559a42258900 0x559a42260b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42258900 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e77600 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e76d00 2024-05-17T22:04:31.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a41e77180 2024-05-17T22:04:31.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a41e77180 2024-05-17T22:04:31.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fec52640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x559a42258d80 0x559a42261080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.861+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a42258d80 2024-05-17T22:04:31.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x559a42258d80 0x559a42261080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x559a42258d80 0x559a42261080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a41e77180 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a42258000 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a42258900 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a41e77180 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a42258900 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a41e76d00 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a42258000 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a3ddd1c00 con 0x559a42258d80 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.865+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258900 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a42258900 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226dc00 con 0x559a42258000 2024-05-17T22:04:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a42258000 2024-05-17T22:04:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a41e76d00 2024-05-17T22:04:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a42258480 2024-05-17T22:04:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a41e77600 2024-05-17T22:04:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fec52640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x559a42259200 0x559a42261600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d800 con 0x559a42259200 2024-05-17T22:04:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x559a42259200 0x559a42261600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.869+0000 7fd403c5c640 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x559a42259200 0x559a42261600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226dc00 con 0x559a41e77180 2024-05-17T22:04:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a41e77600 2024-05-17T22:04:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a3ddd1c00 con 0x559a42258900 2024-05-17T22:04:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258480 2024-05-17T22:04:31.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c800 con 0x559a42258000 2024-05-17T22:04:31.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e77180 2024-05-17T22:04:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.873+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a42258480 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a42258000 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a42259200 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a41e76d00 2024-05-17T22:04:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a41e77180 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42259200 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a42258900 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a42258480 2024-05-17T22:04:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.877+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d800 con 0x559a42258480 2024-05-17T22:04:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a42258480 2024-05-17T22:04:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226dc00 con 0x559a42258000 2024-05-17T22:04:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42258480 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c000 con 0x559a42258d80 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a41e77600 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c400 con 0x559a41e77600 2024-05-17T22:04:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258d80 2024-05-17T22:04:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.881+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c800 con 0x559a42258000 2024-05-17T22:04:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e77600 2024-05-17T22:04:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227cc00 con 0x559a41e77600 2024-05-17T22:04:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a41e76d00 2024-05-17T22:04:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d000 con 0x559a42258900 2024-05-17T22:04:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a42258480 2024-05-17T22:04:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.885+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d400 con 0x559a42258480 2024-05-17T22:04:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a41e77600 2024-05-17T22:04:31.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 85 out=81b] v85'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223cc00 con 0x559a41e77600 2024-05-17T22:04:31.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e77180 2024-05-17T22:04:31.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d800 con 0x559a42258000 2024-05-17T22:04:31.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a42258480 2024-05-17T22:04:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.889+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227dc00 con 0x559a42258900 2024-05-17T22:04:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a42258900 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227cc00 con 0x559a41e77600 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a41e76d00 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d000 con 0x559a41e77180 2024-05-17T22:04:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c800 con 0x559a41e77180 2024-05-17T22:04:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.893+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227dc00 con 0x559a42258d80 2024-05-17T22:04:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a41e77180 2024-05-17T22:04:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a42258000 2024-05-17T22:04:31.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a3ddd1c00 con 0x559a42258900 2024-05-17T22:04:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a41e77180 2024-05-17T22:04:31.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d400 con 0x559a42258900 2024-05-17T22:04:31.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226dc00 con 0x559a41e76d00 2024-05-17T22:04:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d000 con 0x559a42258000 2024-05-17T22:04:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258d80 2024-05-17T22:04:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.897+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c400 con 0x559a42258900 2024-05-17T22:04:31.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223cc00 con 0x559a42258900 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d800 con 0x559a42258000 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a42258000 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a41e76d00 2024-05-17T22:04:31.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227cc00 con 0x559a42258480 2024-05-17T22:04:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a41e77600 2024-05-17T22:04:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d400 con 0x559a42259200 2024-05-17T22:04:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.901+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a41e77180 2024-05-17T22:04:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c400 con 0x559a41e77600 2024-05-17T22:04:31.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a42258900 2024-05-17T22:04:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c000 con 0x559a42258480 2024-05-17T22:04:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42258000 2024-05-17T22:04:31.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.905+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c800 con 0x559a41e77180 2024-05-17T22:04:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a42258480 2024-05-17T22:04:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226dc00 con 0x559a42258000 2024-05-17T22:04:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42259200 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a41e76d00 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a41e77180 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c000 con 0x559a42259200 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.3 v2:172.21.15.83:6824/2257217219 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42259200 2024-05-17T22:04:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.909+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258900 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223cc00 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d800 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a3ddd1c00 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d800 con 0x559a42258000 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c800 con 0x559a42258480 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.913+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227dc00 con 0x559a42258d80 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d400 con 0x559a41e77600 2024-05-17T22:04:31.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d000 con 0x559a41e77600 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4223dc00 con 0x559a42258d80 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228e000 con 0x559a42258000 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227cc00 con 0x559a41e77600 2024-05-17T22:04:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.917+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228e400 con 0x559a41e77600 2024-05-17T22:04:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d400 con 0x559a41e76d00 2024-05-17T22:04:31.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228e800 con 0x559a42258900 2024-05-17T22:04:31.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.2 v2:172.21.15.83:6808/3743214759 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a42258900 2024-05-17T22:04:31.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c000 con 0x559a42258480 2024-05-17T22:04:31.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228ec00 con 0x559a42258480 2024-05-17T22:04:31.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.921+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227c800 con 0x559a41e77600 2024-05-17T22:04:31.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 85 out=81b] v85'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-17T22:04:31.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 10 datascan.operator(): handling object 10000000001.0 2024-05-17T22:04:31.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4223c400 con 0x559a41e77600 2024-05-17T22:04:31.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.7 v2:172.21.15.155:6816/3020759009 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x559a41f04000 con 0x559a41e77600 2024-05-17T22:04:31.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-17T22:04:31.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-17T22:04:31.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-17T22:04:31.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c000 con 0x559a42258d80 2024-05-17T22:04:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-17T22:04:31.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d000 con 0x559a41e77180 2024-05-17T22:04:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-17T22:04:31.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.get_frag_of: frag is * 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d400 con 0x559a42258d80 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x559a4223d800 con 0x559a42258d80 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 20 datascan.read_dentry: file_head not found in result 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-17T22:04:31.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.925+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c800 con 0x559a42258d80 2024-05-17T22:04:31.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.929+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.0 v2:172.21.15.83:6816/622135753 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v85'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559a41f04780 con 0x559a42258d80 2024-05-17T22:04:31.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.929+0000 7fd404c9c8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-17T22:04:31.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.929+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226d800 con 0x559a42258000 2024-05-17T22:04:31.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd402c5a640 1 -- 172.21.15.83:0/545557482 <== osd.1 v2:172.21.15.83:6800/3404427378 19 ==== osd_op_reply(123 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04780 con 0x559a42258000 2024-05-17T22:04:31.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226cc00 con 0x559a41e76d00 2024-05-17T22:04:31.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228f000 con 0x559a41e76d00 2024-05-17T22:04:31.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 <== osd.5 v2:172.21.15.155:6808/758721566 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04000 con 0x559a41e76d00 2024-05-17T22:04:31.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4226c400 con 0x559a41e77180 2024-05-17T22:04:31.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228f400 con 0x559a42258480 2024-05-17T22:04:31.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.4 v2:172.21.15.155:6824/505763805 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a42258480 2024-05-17T22:04:31.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.933+0000 7fd3fe451640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4227d800 con 0x559a41e77180 2024-05-17T22:04:31.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd3fec52640 1 -- 172.21.15.83:0/545557482 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x559a4228f800 con 0x559a41e77180 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd403c5c640 1 -- 172.21.15.83:0/545557482 <== osd.6 v2:172.21.15.155:6800/1025803281 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559a41f04500 con 0x559a41e77180 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x559a42258d80 msgr2=0x559a42261080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x559a42258d80 0x559a42261080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x559a42258000 msgr2=0x559a42260000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x559a42258000 0x559a42260000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x559a42258900 msgr2=0x559a42260b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x559a42258900 0x559a42260b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x559a42259200 msgr2=0x559a42261600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x559a42259200 0x559a42261600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 reap_dead start 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x559a42258480 msgr2=0x559a42260580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x559a42258480 0x559a42260580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x559a41e76d00 msgr2=0x559a41f78680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x559a41e76d00 0x559a41f78680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559a41e77180 msgr2=0x559a41f78c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559a41e77180 0x559a41f78c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x559a41e77600 msgr2=0x559a41f79180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x559a41e77600 0x559a41f79180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd40345b640 1 -- 172.21.15.83:0/545557482 reap_dead start 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 msgr2=0x559a41f78100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 0x559a41f78100 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x559a41f16660 tx=0x559a41f14900 comp rx=0 tx=0).stop 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 msgr2=0x559a41f77b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x559a42d3bd70 tx=0x559a42d44c00 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 shutdown_connections 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e76880 0x559a41f77b80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/545557482 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x559a41e77a80 0x559a41f78100 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 >> 172.21.15.83:0/545557482 conn(0x559a41e75200 msgr2=0x559a4212a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 shutdown_connections 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/545557482 wait complete. 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 msgr2=0x559a41f76000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x559a42c63c20 tx=0x559a41f15ec0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 shutdown_connections 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559a41e75f80 0x559a41f76b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a41e75b00 0x559a41f76000 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 --2- 172.21.15.83:0/4129773690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559a41e75680 0x559a41f76580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 >> 172.21.15.83:0/4129773690 conn(0x559a41e76400 msgr2=0x559a3ddff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 shutdown_connections 2024-05-17T22:04:31.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:31.937+0000 7fd404c9c8c0 1 -- 172.21.15.83:0/4129773690 wait complete. 2024-05-17T22:04:31.940 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.166073, result: 2024-05-17T22:04:31.940 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-17T22:04:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- start start 2024-05-17T22:04:32.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 Processor -- start 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d09680 0x562790e06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d09f80 0x562790e06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562791a6e180 con 0x562790d09b00 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562791a6e300 con 0x562790d09f80 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562791a6e480 con 0x562790d09680 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43808/0 (socket says 172.21.15.83:43808) 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 -- 172.21.15.83:0/967134182 learned_addr learned my addr 172.21.15.83:0/967134182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d09f80 0x562790e06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d09680 0x562790e06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 -- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d09680 msgr2=0x562790e06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:32.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d09680 0x562790e06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 -- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d09f80 msgr2=0x562790e06b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d09f80 0x562790e06b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 -- 172.21.15.83:0/967134182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278cc8a1e0 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5972a640 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x562791af3320 tx=0x562790da8840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96123b92b5eb968 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc57726640 1 -- 172.21.15.83:0/967134182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790cca9c0 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc57726640 1 -- 172.21.15.83:0/967134182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562790cdee00 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc57726640 1 -- 172.21.15.83:0/967134182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790cca4e0 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278cc8b4a0 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56278cc8a3c0 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 4 waiting for MDS map... 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc57726640 1 -- 172.21.15.83:0/967134182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x562791bba600 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc57726640 1 -- 172.21.15.83:0/967134182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 67) ==== 2885+0+0 (secure 0 0 0) 0x562790ceb180 con 0x562790d09b00 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 4 Got MDS map 67 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 4 datascan.main: Using metadata pool output 2024-05-17T22:04:32.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 Processor -- start 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.013+0000 7fcc5b76c8c0 1 -- start start 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562791a6e600 con 0x562790d0a880 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562791a6e780 con 0x562790d08d80 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562791a6e900 con 0x562790d08900 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43818/0 (socket says 172.21.15.83:43818) 2024-05-17T22:04:32.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 -- 172.21.15.83:0/1053509244 learned_addr learned my addr 172.21.15.83:0/1053509244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 -- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d08900 msgr2=0x562790e07600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 -- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08d80 msgr2=0x562790e07080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 -- 172.21.15.83:0/1053509244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278cc8a1e0 con 0x562790d0a880 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5972a640 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x562791bc64e0 tx=0x562790da86c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eeb51e3faee0b55 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc55722640 1 -- 172.21.15.83:0/1053509244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790ccab60 con 0x562790d0a880 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc55722640 1 -- 172.21.15.83:0/1053509244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562790cdec40 con 0x562790d0a880 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc55722640 1 -- 172.21.15.83:0/1053509244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790cca340 con 0x562790d0a880 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/1053509244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 msgr2=0x562790e07b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x562791bc64e0 tx=0x562790da86c0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/1053509244 shutdown_connections 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/1053509244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/1053509244 >> 172.21.15.83:0/1053509244 conn(0x562790d09200 msgr2=0x562790fb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/1053509244 shutdown_connections 2024-05-17T22:04:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/1053509244 wait complete. 2024-05-17T22:04:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 Processor -- start 2024-05-17T22:04:32.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.017+0000 7fcc5b76c8c0 1 -- start start 2024-05-17T22:04:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562791a6f500 con 0x562790d08d80 2024-05-17T22:04:32.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562791a6ea80 con 0x562790d08900 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562791a6f680 con 0x562790d0a880 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43830/0 (socket says 172.21.15.83:43830) 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 learned_addr learned my addr 172.21.15.83:0/118802220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d0a880 msgr2=0x562790e07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d0a880 0x562790e07b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08900 msgr2=0x562790e07600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278cc8b2c0 con 0x562790d08d80 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d08900 0x562790e07600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x562791bc7b90 tx=0x562791bccc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 -- 172.21.15.83:0/118802220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790ccab60 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 -- 172.21.15.83:0/118802220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562790cdea80 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56278cc8ba40 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 -- 172.21.15.83:0/118802220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562790cca340 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278cc8bc20 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 4 datascan.init: resolving metadata pool 12 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 -- 172.21.15.83:0/118802220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x562790fc8c00 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 0x562790e08100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc53f1f640 1 -- 172.21.15.83:0/118802220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x562791bf3000 con 0x562790d08d80 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 0x562790e08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 4 datascan.main: resolving metadata pool 12 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x562790d0ad00 0x562790e08680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x562790d0ad00 2024-05-17T22:04:32.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 0x562790e08100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x562790dab140 tx=0x562791bccf00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.021+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x562790d0ad00 0x562790e08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x562790d0ad00 0x562790e08680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(1 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627910d8000 0x5627910dc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627910d8000 0x5627910dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627910d8000 0x5627910dc000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(2 [pgnls start_epoch 85 out=177b] v85'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627910d8480 0x5627910dc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8480 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627910d8480 0x5627910dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627910d8480 0x5627910dc580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(3 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627910d8900 0x5627910dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8900 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627910d8900 0x5627910dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627910d8900 0x5627910dcb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(4 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 0x5627910dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.025+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 0x5627910dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 0x5627910dd080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(5 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627910d9680 0x5627910ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9680 2024-05-17T22:04:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627910d9680 0x5627910ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627910d9680 0x5627910ddb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(6 [pgnls start_epoch 85 out=149b] v84'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(7 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(8 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.029+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x562790d0ad00 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(9 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8480 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(10 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc59f2b640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627910d9f80 0x5627910de680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9f80 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627910d9f80 0x5627910de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627910d9f80 0x5627910de680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(11 [pgnls start_epoch 85 out=81b] v84'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x562790d0ad00 2024-05-17T22:04:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(12 [pgnls start_epoch 85 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8900 2024-05-17T22:04:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(13 [pgnls start_epoch 85 out=77b] v84'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(14 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.033+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(15 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x562790d0ad00 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(16 [pgnls start_epoch 85 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(17 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9680 2024-05-17T22:04:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(18 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9f80 2024-05-17T22:04:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(19 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 5 ==== osd_op_reply(20 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.037+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(21 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(22 [pgnls start_epoch 85 out=97b] v84'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 0x5627910de100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9b00 2024-05-17T22:04:32.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 0x5627910de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 0x5627910de100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(23 [pgnls start_epoch 85 out=74b] v80'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562790d98500 con 0x5627910d9b00 2024-05-17T22:04:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 6 ==== osd_op_reply(24 [pgnls start_epoch 85 out=73b] v84'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8000 2024-05-17T22:04:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 7 ==== osd_op_reply(25 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8900 2024-05-17T22:04:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(26 [pgnls start_epoch 85 out=98b] v84'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8480 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(27 [pgnls start_epoch 85 out=76b] v84'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.041+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x562790d0ad00 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(28 [pgnls start_epoch 85 out=73b] v84'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8900 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(29 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 6 ==== osd_op_reply(30 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 7 ==== osd_op_reply(31 [pgnls start_epoch 85 out=73b] v80'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d8d80 2024-05-17T22:04:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 8 ==== osd_op_reply(32 [pgnls start_epoch 85 out=101b] v84'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-17T22:04:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2400 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3400 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.045+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 20 datascan.scan_links: injected ino detected 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3c00 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-17T22:04:32.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3000 con 0x5627910d8000 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-17T22:04:32.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2000 con 0x5627910d9680 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2400 con 0x5627910d8000 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3400 con 0x5627910d9f80 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-17T22:04:32.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a000 con 0x5627910d8000 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.049+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a400 con 0x5627910d8d80 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-17T22:04:32.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a800 con 0x5627910d8000 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110ac00 con 0x5627910d8900 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-17T22:04:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110b000 con 0x5627910d8900 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-17T22:04:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3400 con 0x5627910d8d80 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x562790d0ad00 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 6 ==== osd_op_reply(47 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 16 ==== osd_op_reply(48 [pgnls start_epoch 85 out=177b] v85'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8480 2024-05-17T22:04:32.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 4 ==== osd_op_reply(49 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8900 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(50 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.053+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 11 ==== osd_op_reply(51 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9680 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(52 [pgnls start_epoch 85 out=149b] v84'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 17 ==== osd_op_reply(53 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 18 ==== osd_op_reply(54 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x562790d0ad00 2024-05-17T22:04:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 7 ==== osd_op_reply(55 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8480 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 5 ==== osd_op_reply(56 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9f80 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 4 ==== osd_op_reply(57 [pgnls start_epoch 85 out=81b] v84'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.057+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x562790d0ad00 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 8 ==== osd_op_reply(58 [pgnls start_epoch 85 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8900 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(59 [pgnls start_epoch 85 out=77b] v84'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 12 ==== osd_op_reply(60 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 19 ==== osd_op_reply(61 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x562790d0ad00 2024-05-17T22:04:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 9 ==== osd_op_reply(62 [pgnls start_epoch 85 out=49b] v84'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 13 ==== osd_op_reply(63 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.061+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9680 2024-05-17T22:04:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(64 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9f80 2024-05-17T22:04:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 5 ==== osd_op_reply(65 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 20 ==== osd_op_reply(66 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 14 ==== osd_op_reply(67 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 15 ==== osd_op_reply(68 [pgnls start_epoch 85 out=97b] v84'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.065+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9b00 2024-05-17T22:04:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(69 [pgnls start_epoch 85 out=74b] v80'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562790d98500 con 0x5627910d9b00 2024-05-17T22:04:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 21 ==== osd_op_reply(70 [pgnls start_epoch 85 out=73b] v84'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8000 2024-05-17T22:04:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 22 ==== osd_op_reply(71 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8900 2024-05-17T22:04:32.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 9 ==== osd_op_reply(72 [pgnls start_epoch 85 out=98b] v84'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8480 2024-05-17T22:04:32.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.3 v2:172.21.15.83:6824/2257217219 6 ==== osd_op_reply(73 [pgnls start_epoch 85 out=76b] v84'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562790d98780 con 0x5627910d8480 2024-05-17T22:04:32.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x562790d0ad00 2024-05-17T22:04:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.4 v2:172.21.15.155:6824/505763805 10 ==== osd_op_reply(74 [pgnls start_epoch 85 out=73b] v84'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98000 con 0x562790d0ad00 2024-05-17T22:04:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8900 2024-05-17T22:04:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 10 ==== osd_op_reply(75 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 16 ==== osd_op_reply(76 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.069+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 17 ==== osd_op_reply(77 [pgnls start_epoch 85 out=73b] v80'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d8d80 2024-05-17T22:04:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 18 ==== osd_op_reply(78 [pgnls start_epoch 85 out=101b] v84'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-17T22:04:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3c00 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8c00 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8400 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 20 datascan.scan_links: injected ino detected 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-17T22:04:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8800 con 0x5627910d8000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3400 con 0x5627910d8000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-17T22:04:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.073+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3800 con 0x5627910d9680 2024-05-17T22:04:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98780 con 0x5627910d9680 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3c00 con 0x5627910d8000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d3000 con 0x5627910d9f80 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.2 v2:172.21.15.83:6808/3743214759 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x562790d98000 con 0x5627910d9f80 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x5627910d2400 con 0x5627910d8000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-17T22:04:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8c00 con 0x5627910d8d80 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a000 con 0x5627910d8000 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-17T22:04:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a400 con 0x5627910d8900 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a800 con 0x5627910d8900 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.077+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110ac00 con 0x5627910d8d80 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.6 v2:172.21.15.155:6800/1025803281 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x562790d98500 con 0x5627910d8d80 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110b000 con 0x5627910d8000 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-17T22:04:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.081+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x56279110b400 con 0x5627910d8000 2024-05-17T22:04:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v85'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110b800 con 0x5627910d8000 2024-05-17T22:04:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-17T22:04:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 20 datascan.scan_links: first is now 2 2024-05-17T22:04:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.085+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x56279110bc00 con 0x5627910d8000 2024-05-17T22:04:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.0 v2:172.21.15.83:6816/622135753 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v85'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562790d98500 con 0x5627910d8000 2024-05-17T22:04:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-17T22:04:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 10 datascan.scan_links: updating inotable 2024-05-17T22:04:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x56279110a000 con 0x5627910d8900 2024-05-17T22:04:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-17T22:04:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-17T22:04:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8400 con 0x5627910d8900 2024-05-17T22:04:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5972a640 1 -- 172.21.15.83:0/118802220 <== osd.1 v2:172.21.15.83:6800/3404427378 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v85'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x562790d98000 con 0x5627910d8900 2024-05-17T22:04:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 10 datascan.scan_links: updating snaptable 2024-05-17T22:04:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.089+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) -- 0x562791bc8800 con 0x5627910d9b00 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5a72c640 1 -- 172.21.15.83:0/118802220 <== osd.7 v2:172.21.15.155:6816/3020759009 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x562790d98280 con 0x5627910d9b00 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627910d8000 msgr2=0x5627910dc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627910d8000 0x5627910dc000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627910d8900 msgr2=0x5627910dcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627910d8900 0x5627910dcb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627910d9f80 msgr2=0x5627910de680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627910d9f80 0x5627910de680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627910d8480 msgr2=0x5627910dc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627910d8480 0x5627910dc580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x562790d0ad00 msgr2=0x562790e08680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x562790d0ad00 0x562790e08680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627910d9680 msgr2=0x5627910ddb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 reap_dead start 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627910d9680 0x5627910ddb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 reap_dead start 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 msgr2=0x5627910dd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 0x5627910dd080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc59f2b640 1 -- 172.21.15.83:0/118802220 reap_dead start 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 msgr2=0x5627910de100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 0x5627910de100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 msgr2=0x562790e08100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 0x562790e08100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x562790dab140 tx=0x562791bccf00 comp rx=0 tx=0).stop 2024-05-17T22:04:32.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 msgr2=0x562790e07080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x562791bc7b90 tx=0x562791bccc00 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 shutdown_connections 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627910d9b00 0x5627910de100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d08d80 0x562790e07080 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627910d8d80 0x5627910dd080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/118802220 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x562790d0ba80 0x562790e08100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 >> 172.21.15.83:0/118802220 conn(0x562790d09200 msgr2=0x562790fb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 shutdown_connections 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/118802220 wait complete. 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 msgr2=0x562790e06000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x562791af3320 tx=0x562790da8840 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 shutdown_connections 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562790d09f80 0x562790e06b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562790d09b00 0x562790e06000 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 --2- 172.21.15.83:0/967134182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562790d09680 0x562790e06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 >> 172.21.15.83:0/967134182 conn(0x562790d0a400 msgr2=0x56278cc93800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 shutdown_connections 2024-05-17T22:04:32.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.093+0000 7fcc5b76c8c0 1 -- 172.21.15.83:0/967134182 wait complete. 2024-05-17T22:04:32.092 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.151494, result: 2024-05-17T22:04:32.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:04:32.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.253+0000 7f9a94fff640 1 Processor -- start 2024-05-17T22:04:32.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.253+0000 7f9a94fff640 1 -- start start 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a94fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a900b3be0 0x7f9a900af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a94fff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a900ce450 con 0x7f9a900b4bb0 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a94fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a900ce730 con 0x7f9a900af8b0 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a94fff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9a900cea10 con 0x7f9a900b3be0 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8e59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a900b3be0 0x7f9a900af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8dd9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9a900b4bb0 0x7f9a900cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51598/0 (socket says 172.21.15.83:51598) 2024-05-17T22:04:32.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8dd9b640 1 -- 172.21.15.83:0/3637746091 learned_addr learned my addr 172.21.15.83:0/3637746091 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3637746091 >> v1:172.21.15.155:6789/0 conn(0x7f9a900af8b0 legacy=0x7f9a9006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3637746091 >> v1:172.21.15.83:6789/0 conn(0x7f9a900b4bb0 legacy=0x7f9a900cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3637746091 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900cecf0 con 0x7f9a900b3be0 2024-05-17T22:04:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.257+0000 7f9a8d59a640 1 -- 172.21.15.83:0/3637746091 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832052747 0 0) 0x7f9a900ce450 con 0x7f9a900b4bb0 2024-05-17T22:04:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a8e59c640 1 --2- 172.21.15.83:0/3637746091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900b3be0 0x7f9a900af370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9a800097c0 tx=0x7f9a8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60f596fdb36a041 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a8d59a640 1 -- 172.21.15.83:0/3637746091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800b1070 con 0x7f9a900b3be0 2024-05-17T22:04:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a8d59a640 1 -- 172.21.15.83:0/3637746091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a800ac410 con 0x7f9a900b3be0 2024-05-17T22:04:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a8d59a640 1 -- 172.21.15.83:0/3637746091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800ac7e0 con 0x7f9a900b3be0 2024-05-17T22:04:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- 172.21.15.83:0/3637746091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900b3be0 msgr2=0x7f9a900af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3637746091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900b3be0 0x7f9a900af370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9a800097c0 tx=0x7f9a8001c4a0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- 172.21.15.83:0/3637746091 shutdown_connections 2024-05-17T22:04:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3637746091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900b3be0 0x7f9a900af370 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- 172.21.15.83:0/3637746091 >> 172.21.15.83:0/3637746091 conn(0x7f9a900b4090 msgr2=0x7f9a900b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- 172.21.15.83:0/3637746091 shutdown_connections 2024-05-17T22:04:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- 172.21.15.83:0/3637746091 wait complete. 2024-05-17T22:04:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 Processor -- start 2024-05-17T22:04:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.261+0000 7f9a94fff640 1 -- start start 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 0x7f9a9015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a900b4bb0 0x7f9a900c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9006cea0 con 0x7f9a900af8b0 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a900cfe10 con 0x7f9a900b3be0 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9006d020 con 0x7f9a900b4bb0 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43854/0 (socket says 172.21.15.83:43854) 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3382937771 learned_addr learned my addr 172.21.15.83:0/3382937771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8dd9b640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 0x7f9a9015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a900b4bb0 msgr2=0x7f9a900c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a900b4bb0 0x7f9a900c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 msgr2=0x7f9a9015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 0x7f9a9015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 -- 172.21.15.83:0/3382937771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900ca8c0 con 0x7f9a900af8b0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8dd9b640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 0x7f9a9015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a8e59c640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9a800af040 tx=0x7f9a8001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800bd070 con 0x7f9a900af8b0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a800b2040 con 0x7f9a900af8b0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a90168ac0 con 0x7f9a900af8b0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800b1050 con 0x7f9a900af8b0 2024-05-17T22:04:32.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a90169240 con 0x7f9a900af8b0 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a900b7d90 con 0x7f9a900af8b0 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9a800c0070 con 0x7f9a900af8b0 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.265+0000 7f9a7effd640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 0x7f9a70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.269+0000 7f9a8dd9b640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 0x7f9a70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.269+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f9a80128c00 con 0x7f9a900af8b0 2024-05-17T22:04:32.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.269+0000 7f9a8dd9b640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 0x7f9a70071910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a900cccb0 tx=0x7f9a84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.273+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9a8012f080 con 0x7f9a900af8b0 2024-05-17T22:04:32.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.465+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9a90169410 con 0x7f9a900af8b0 2024-05-17T22:04:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.613+0000 7f9a7effd640 1 -- 172.21.15.83:0/3382937771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v68) ==== 162+0+0 (secure 0 0 0) 0x7f9a80016090 con 0x7f9a900af8b0 2024-05-17T22:04:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 msgr2=0x7f9a70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 0x7f9a70071910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a900cccb0 tx=0x7f9a84009040 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 msgr2=0x7f9a90166420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9a800af040 tx=0x7f9a8001ace0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 shutdown_connections 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a900b4bb0 0x7f9a900c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a7006f400 0x7f9a70071910 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a900b3be0 0x7f9a9015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 --2- 172.21.15.83:0/3382937771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a900af8b0 0x7f9a90166420 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 >> 172.21.15.83:0/3382937771 conn(0x7f9a900b4090 msgr2=0x7f9a90070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 shutdown_connections 2024-05-17T22:04:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.617+0000 7f9a94fff640 1 -- 172.21.15.83:0/3382937771 wait complete. 2024-05-17T22:04:32.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:04:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 Processor -- start 2024-05-17T22:04:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 -- start start 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9740d1040 0x7ff974144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97415ea40 con 0x7ff974073540 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97415ed20 con 0x7ff97415e080 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff979cce640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff97415f000 con 0x7ff9740d1040 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff973fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff974073540 0x7ff97406f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51610/0 (socket says 172.21.15.83:51610) 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff973fff640 1 -- 172.21.15.83:0/3503668769 learned_addr learned my addr 172.21.15.83:0/3503668769 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9737fe640 1 --2- 172.21.15.83:0/3503668769 >> v2:172.21.15.83:3300/0 conn(0x7ff9740d1040 0x7ff974144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9727fc640 1 -- 172.21.15.83:0/3503668769 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280950211 0 0) 0x7ff97415ea40 con 0x7ff974073540 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9727fc640 1 -- 172.21.15.83:0/3503668769 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95c003530 con 0x7ff974073540 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9737fe640 1 -- 172.21.15.83:0/3503668769 >> v1:172.21.15.155:6789/0 conn(0x7ff97415e080 legacy=0x7ff974072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:32.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9737fe640 1 -- 172.21.15.83:0/3503668769 >> v1:172.21.15.83:6789/0 conn(0x7ff974073540 legacy=0x7ff97406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.773+0000 7ff9737fe640 1 -- 172.21.15.83:0/3503668769 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97406ef30 con 0x7ff9740d1040 2024-05-17T22:04:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff9737fe640 1 --2- 172.21.15.83:0/3503668769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9740d1040 0x7ff974144080 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d90 tx=0x7ff96401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af2e70a7673829b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff9727fc640 1 -- 172.21.15.83:0/3503668769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640b2070 con 0x7ff9740d1040 2024-05-17T22:04:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff9727fc640 1 -- 172.21.15.83:0/3503668769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9640a5940 con 0x7ff9740d1040 2024-05-17T22:04:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff9727fc640 1 -- 172.21.15.83:0/3503668769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640a5d10 con 0x7ff9740d1040 2024-05-17T22:04:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 -- 172.21.15.83:0/3503668769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9740d1040 msgr2=0x7ff974144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 --2- 172.21.15.83:0/3503668769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9740d1040 0x7ff974144080 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff964009d90 tx=0x7ff96401c970 comp rx=0 tx=0).stop 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 -- 172.21.15.83:0/3503668769 shutdown_connections 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 --2- 172.21.15.83:0/3503668769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9740d1040 0x7ff974144080 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 -- 172.21.15.83:0/3503668769 >> 172.21.15.83:0/3503668769 conn(0x7ff97406cd90 msgr2=0x7ff9740b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 -- 172.21.15.83:0/3503668769 shutdown_connections 2024-05-17T22:04:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.777+0000 7ff979cce640 1 -- 172.21.15.83:0/3503668769 wait complete. 2024-05-17T22:04:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 Processor -- start 2024-05-17T22:04:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- start start 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 0x7ff97413c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff97413afd0 0x7ff9740b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740709e0 con 0x7ff97415e080 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff974070860 con 0x7ff9740d1040 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff97416b860 con 0x7ff97413afd0 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43874/0 (socket says 172.21.15.83:43874) 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 -- 172.21.15.83:0/1713916318 learned_addr learned my addr 172.21.15.83:0/1713916318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff9737fe640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 0x7ff97413c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff973fff640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff97413afd0 0x7ff9740b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 -- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff97413afd0 msgr2=0x7ff9740b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff97413afd0 0x7ff9740b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 -- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 msgr2=0x7ff97413c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 0x7ff97413c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 -- 172.21.15.83:0/1713916318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97413d200 con 0x7ff97415e080 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff9737fe640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 0x7ff97413c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff972ffd640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c3c0 tx=0x7ff96800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968017070 con 0x7ff97415e080 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9680052d0 con 0x7ff97415e080 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97413d4e0 con 0x7ff97415e080 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968003800 con 0x7ff97415e080 2024-05-17T22:04:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97413daa0 con 0x7ff97415e080 2024-05-17T22:04:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.781+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97413cf70 con 0x7ff97415e080 2024-05-17T22:04:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.785+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff96801a070 con 0x7ff97415e080 2024-05-17T22:04:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.785+0000 7ff95bfff640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 0x7ff960071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.785+0000 7ff9737fe640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 0x7ff960071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:32.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.785+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7ff96808e880 con 0x7ff97415e080 2024-05-17T22:04:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.785+0000 7ff9737fe640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 0x7ff960071910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff9640af000 tx=0x7ff9640a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:32.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.789+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff968005480 con 0x7ff97415e080 2024-05-17T22:04:32.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff974144080 con 0x7ff97415e080 2024-05-17T22:04:32.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff95bfff640 1 -- 172.21.15.83:0/1713916318 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5183 (secure 0 0 0) 0x7ff96805b230 con 0x7ff97415e080 2024-05-17T22:04:32.970 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:32.970 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:32.622044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-17T22:04:32.971 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-05-17T22:04:32.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 msgr2=0x7ff960071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 0x7ff960071910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff9640af000 tx=0x7ff9640a7070 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 msgr2=0x7ff97415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c3c0 tx=0x7ff96800c880 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 shutdown_connections 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff97413afd0 0x7ff9740b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff97415e080 0x7ff97415d810 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff96006f400 0x7ff960071910 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 --2- 172.21.15.83:0/1713916318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740d1040 0x7ff97413c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 >> 172.21.15.83:0/1713916318 conn(0x7ff97406cd90 msgr2=0x7ff974144b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 shutdown_connections 2024-05-17T22:04:32.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:32.981+0000 7ff979cce640 1 -- 172.21.15.83:0/1713916318 wait complete. 2024-05-17T22:04:33.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:04:32.622044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'a', 'rank': 0, 'incarnation': 69, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6836/2140054867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2140054867}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 2140054867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:04:33.394 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7083:up:replay 2024-05-17T22:04:34.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:04:34.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 Processor -- start 2024-05-17T22:04:34.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 -- start start 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4144080 0x7f5df4072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5df4073370 0x7f5df406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df4073780 con 0x7f5df40d1040 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5df415e790 con 0x7f5df4073370 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df8893640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5df415ea40 con 0x7f5df4144080 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5df4073370 0x7f5df406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5df4073370 0x7f5df406f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43882/0 (socket says 172.21.15.83:43882) 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 -- 172.21.15.83:0/967142899 learned_addr learned my addr 172.21.15.83:0/967142899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df1d9b640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4144080 0x7f5df4072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 -- 172.21.15.83:0/967142899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4144080 msgr2=0x7f5df4072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:34.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4144080 0x7f5df4072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 -- 172.21.15.83:0/967142899 >> v1:172.21.15.155:6789/0 conn(0x7f5df40d1040 legacy=0x7f5df415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:34.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.569+0000 7f5df2d9d640 1 -- 172.21.15.83:0/967142899 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df415ed20 con 0x7f5df4073370 2024-05-17T22:04:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df2d9d640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df4073370 0x7f5df406f730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e50 tx=0x7f5dec01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca258471a1be342 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df159a640 1 -- 172.21.15.83:0/967142899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5660 con 0x7f5df4073370 2024-05-17T22:04:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df159a640 1 -- 172.21.15.83:0/967142899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec0a5800 con 0x7f5df4073370 2024-05-17T22:04:34.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df159a640 1 -- 172.21.15.83:0/967142899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5bd0 con 0x7f5df4073370 2024-05-17T22:04:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 -- 172.21.15.83:0/967142899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df4073370 msgr2=0x7f5df406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df4073370 0x7f5df406f730 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e50 tx=0x7f5dec01c820 comp rx=0 tx=0).stop 2024-05-17T22:04:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 -- 172.21.15.83:0/967142899 shutdown_connections 2024-05-17T22:04:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df4073370 0x7f5df406f730 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 --2- 172.21.15.83:0/967142899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4144080 0x7f5df4072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 -- 172.21.15.83:0/967142899 >> 172.21.15.83:0/967142899 conn(0x7f5df406cd90 msgr2=0x7f5df40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 -- 172.21.15.83:0/967142899 shutdown_connections 2024-05-17T22:04:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.573+0000 7f5df8893640 1 -- 172.21.15.83:0/967142899 wait complete. 2024-05-17T22:04:34.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 Processor -- start 2024-05-17T22:04:34.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- start start 2024-05-17T22:04:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 0x7f5df413fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5df4144080 0x7f5df4166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4070480 con 0x7f5df40d1040 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4144490 con 0x7f5df4144080 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5df4070600 con 0x7f5df4073370 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43884/0 (socket says 172.21.15.83:43884) 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 -- 172.21.15.83:0/333620665 learned_addr learned my addr 172.21.15.83:0/333620665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df259c640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 0x7f5df413fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df2d9d640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5df4144080 0x7f5df4166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 -- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 msgr2=0x7f5df413fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 0x7f5df413fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 -- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5df4144080 msgr2=0x7f5df4166c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5df4144080 0x7f5df4166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 -- 172.21.15.83:0/333620665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41462d0 con 0x7f5df40d1040 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df259c640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 0x7f5df413fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:34.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df1d9b640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d800 tx=0x7f5de400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400b6c0 con 0x7f5df40d1040 2024-05-17T22:04:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de4019070 con 0x7f5df40d1040 2024-05-17T22:04:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df40b4f10 con 0x7f5df40d1040 2024-05-17T22:04:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400ba80 con 0x7f5df40d1040 2024-05-17T22:04:34.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df40b5380 con 0x7f5df40d1040 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.577+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40b50e0 con 0x7f5df40d1040 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.581+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5de400bc20 con 0x7f5df40d1040 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.581+0000 7f5de2ffd640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 0x7f5dd4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.581+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f5de408f780 con 0x7f5df40d1040 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.581+0000 7f5df259c640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 0x7f5dd4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:34.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.581+0000 7f5df259c640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 0x7f5dd4071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5de800a850 tx=0x7f5de8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:34.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.585+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5df40b50e0 con 0x7f5df40d1040 2024-05-17T22:04:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5df4069c10 con 0x7f5df40d1040 2024-05-17T22:04:34.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5de2ffd640 1 -- 172.21.15.83:0/333620665 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5183 (secure 0 0 0) 0x7f5de405c610 con 0x7f5df40d1040 2024-05-17T22:04:34.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:34.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:33.643979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-17T22:04:34.774 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 71 2024-05-17T22:04:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 msgr2=0x7f5dd4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 0x7f5dd4071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5de800a850 tx=0x7f5de8008040 comp rx=0 tx=0).stop 2024-05-17T22:04:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 msgr2=0x7f5df40b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d800 tx=0x7f5de400dcc0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 shutdown_connections 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5dd406f400 0x7f5dd4071910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5df4144080 0x7f5df4166c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5df40d1040 0x7f5df40b6040 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 --2- 172.21.15.83:0/333620665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5df4073370 0x7f5df413fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 >> 172.21.15.83:0/333620665 conn(0x7f5df406cd90 msgr2=0x7f5df4146830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 shutdown_connections 2024-05-17T22:04:34.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:34.785+0000 7f5df8893640 1 -- 172.21.15.83:0/333620665 wait complete. 2024-05-17T22:04:35.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:04:33.643979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'a', 'rank': 0, 'incarnation': 69, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6836/2140054867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2140054867}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 2140054867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:04:35.198 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7083:up:rejoin 2024-05-17T22:04:36.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:04:36.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 Processor -- start 2024-05-17T22:04:36.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 -- start start 2024-05-17T22:04:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15e006d7a0 0x7f15e006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e006e0f0 con 0x7f15e015d8d0 2024-05-17T22:04:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e00700e0 con 0x7f15e00af7c0 2024-05-17T22:04:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e6ef3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f15e0071590 con 0x7f15e006d7a0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e5490640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f15e015d8d0 0x7f15e015dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51624/0 (socket says 172.21.15.83:51624) 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.369+0000 7f15e5490640 1 -- 172.21.15.83:0/3043278712 learned_addr learned my addr 172.21.15.83:0/3043278712 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15dffff640 1 --2- 172.21.15.83:0/3043278712 >> v2:172.21.15.155:3300/0 conn(0x7f15e006d7a0 0x7f15e006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095224469 0 0) 0x7f15d80014a0 con 0x7f15e015d8d0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15c8003530 con 0x7f15e015d8d0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3280287286 0 0) 0x7f15c8003530 con 0x7f15e015d8d0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 >> v2:172.21.15.155:3300/0 conn(0x7f15e006d7a0 msgr2=0x7f15e006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 --2- 172.21.15.83:0/3043278712 >> v2:172.21.15.155:3300/0 conn(0x7f15e006d7a0 0x7f15e006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 >> v1:172.21.15.155:6789/0 conn(0x7f15e00af7c0 legacy=0x7f15e0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00aff10 con 0x7f15e015d8d0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f15d8001c50 con 0x7f15e015d8d0 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 >> v1:172.21.15.83:6789/0 conn(0x7f15e015d8d0 legacy=0x7f15e015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15c8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15c8004810 0x7f15c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15c8005260 0x7f15c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0071590 con 0x7f15c8003a60 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15c800c1f0 con 0x7f15c8005260 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15c800c3f0 con 0x7f15c8004810 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15c8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e5490640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15c8005260 0x7f15c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15dffff640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15c8004810 0x7f15c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 -- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15c8004810 msgr2=0x7f15c8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15c8004810 0x7f15c8004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 -- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15c8005260 msgr2=0x7f15c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15c8005260 0x7f15c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e5490640 1 -- 172.21.15.83:0/3043278712 reap_dead start 2024-05-17T22:04:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 -- 172.21.15.83:0/3043278712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8003530 con 0x7f15c8003a60 2024-05-17T22:04:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15c8003ed0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f15d0001790 tx=0x7f15d001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f094736b84758a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00013e0 con 0x7f15c8003a60 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d00052d0 con 0x7f15c8003a60 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.373+0000 7f15df7fe640 1 -- 172.21.15.83:0/3043278712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0003840 con 0x7f15c8003a60 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 msgr2=0x7f15c8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15c8003ed0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f15d0001790 tx=0x7f15d001c3d0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- 172.21.15.83:0/3043278712 shutdown_connections 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/3043278712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15c8003ed0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- 172.21.15.83:0/3043278712 >> 172.21.15.83:0/3043278712 conn(0x7f15e0073670 msgr2=0x7f15e0073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- 172.21.15.83:0/3043278712 shutdown_connections 2024-05-17T22:04:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- 172.21.15.83:0/3043278712 wait complete. 2024-05-17T22:04:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 Processor -- start 2024-05-17T22:04:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- start start 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 0x7f15e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e01518e0 0x7f15e015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15e0071290 con 0x7f15c8003a60 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f15e00afbd0 con 0x7f15e01518e0 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e6ef3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15e0071410 con 0x7f15e00b2f60 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43888/0 (socket says 172.21.15.83:43888) 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 -- 172.21.15.83:0/2763901219 learned_addr learned my addr 172.21.15.83:0/2763901219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e5490640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e01518e0 0x7f15e015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15dffff640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 0x7f15e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 -- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 msgr2=0x7f15e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 0x7f15e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 -- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e01518e0 msgr2=0x7f15e015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e01518e0 0x7f15e015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15e4c8f640 1 -- 172.21.15.83:0/2763901219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e0156690 con 0x7f15c8003a60 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.377+0000 7f15dffff640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 0x7f15e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15e4c8f640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f15d001aad0 tx=0x7f15c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00013c0 con 0x7f15c8003a60 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d00bc070 con 0x7f15c8003a60 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e0156970 con 0x7f15c8003a60 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0005480 con 0x7f15c8003a60 2024-05-17T22:04:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e0156f30 con 0x7f15c8003a60 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e0068b30 con 0x7f15c8003a60 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15d00c1460 con 0x7f15c8003a60 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 0x7f15cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dffff640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 0x7f15cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f15d01294a0 con 0x7f15c8003a60 2024-05-17T22:04:36.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.381+0000 7f15dffff640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 0x7f15cc071940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15d40090e0 tx=0x7f15d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.385+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f15d0016130 con 0x7f15c8003a60 2024-05-17T22:04:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.581+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15e00b35c0 con 0x7f15c8003a60 2024-05-17T22:04:36.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.581+0000 7f15dd7fa640 1 -- 172.21.15.83:0/2763901219 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7f15d00f6330 con 0x7f15c8003a60 2024-05-17T22:04:36.572 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:04:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:04:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 msgr2=0x7f15cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 0x7f15cc071940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f15d40090e0 tx=0x7f15d4008040 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 msgr2=0x7f15e0146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f15d001aad0 tx=0x7f15c8005840 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 shutdown_connections 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f15cc06f430 0x7f15cc071940 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e01518e0 0x7f15e015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15e00b2f60 0x7f15e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 --2- 172.21.15.83:0/2763901219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15c8003a60 0x7f15e0146760 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 >> 172.21.15.83:0/2763901219 conn(0x7f15e0073670 msgr2=0x7f15e0153960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 shutdown_connections 2024-05-17T22:04:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:36.585+0000 7f15e6ef3640 1 -- 172.21.15.83:0/2763901219 wait complete. 2024-05-17T22:04:36.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:02:54.779677+0000', 'modified': '2024-05-17T22:04:35.641883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7083}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7083': {'gid': 7083, 'name': 'a', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6836/2140054867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2140054867}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 2140054867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7083, 'qdb_cluster': [7083]} 2024-05-17T22:04:36.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:04:36.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:04:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 Processor -- start 2024-05-17T22:04:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 -- start start 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 0x7fc1ac06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ac0ce450 con 0x7fc1ac0b4bb0 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ac0ce730 con 0x7fc1ac0b3be0 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1b2035640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac0cea10 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1ab7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 0x7fc1ac06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1ab7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:52694/0 (socket says 172.21.15.83:52694) 2024-05-17T22:04:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1aaffd640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc1ac0b4bb0 0x7fc1ac0cf4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:37356/0 (socket says 172.21.15.83:37356) 2024-05-17T22:04:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.173+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/385451189 learned_addr learned my addr 172.21.15.83:0/385451189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1abfff640 1 -- 172.21.15.83:0/385451189 >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 msgr2=0x7fc1ac0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1abfff640 1 --2- 172.21.15.83:0/385451189 >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1abfff640 1 -- 172.21.15.83:0/385451189 >> v1:172.21.15.155:6789/0 conn(0x7fc1ac0b4bb0 legacy=0x7fc1ac0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1abfff640 1 -- 172.21.15.83:0/385451189 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac0cecf0 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/385451189 >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:04:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1abfff640 1 --2- 172.21.15.83:0/385451189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 0x7fc1ac06c790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009d90 tx=0x7fc1a401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b70236f200b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1aa7fc640 1 -- 172.21.15.83:0/385451189 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40b2070 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1aa7fc640 1 -- 172.21.15.83:0/385451189 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a40a5940 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1aa7fc640 1 -- 172.21.15.83:0/385451189 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a40a5d10 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 -- 172.21.15.83:0/385451189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 msgr2=0x7fc1ac06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 --2- 172.21.15.83:0/385451189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 0x7fc1ac06c790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009d90 tx=0x7fc1a401c970 comp rx=0 tx=0).stop 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 -- 172.21.15.83:0/385451189 shutdown_connections 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 --2- 172.21.15.83:0/385451189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0af8b0 0x7fc1ac06c790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 --2- 172.21.15.83:0/385451189 >> v2:172.21.15.155:3300/0 conn(0x7fc1ac0b3be0 0x7fc1ac0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 -- 172.21.15.83:0/385451189 >> 172.21.15.83:0/385451189 conn(0x7fc1ac0b4090 msgr2=0x7fc1ac0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 -- 172.21.15.83:0/385451189 shutdown_connections 2024-05-17T22:04:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.177+0000 7fc1b2035640 1 -- 172.21.15.83:0/385451189 wait complete. 2024-05-17T22:04:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 Processor -- start 2024-05-17T22:04:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 -- start start 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 0x7fc1ac15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1ac0b4bb0 0x7fc1ac1687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac06d0e0 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac0cfe10 con 0x7fc1ac0b4bb0 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac0b2670 con 0x7fc1ac0b3be0 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36218/0 (socket says 172.21.15.83:36218) 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/1421686973 learned_addr learned my addr 172.21.15.83:0/1421686973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1abfff640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1ac0b4bb0 0x7fc1ac1687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 0x7fc1ac15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 msgr2=0x7fc1ac15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 0x7fc1ac15dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1ac0b4bb0 msgr2=0x7fc1ac1687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1ac0b4bb0 0x7fc1ac1687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/1421686973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac0cc290 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 0x7fc1ac15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e930 tx=0x7fc19c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc193fff640 1 -- 172.21.15.83:0/1421686973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc193fff640 1 -- 172.21.15.83:0/1421686973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c00ca00 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac167ce0 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.181+0000 7fc193fff640 1 -- 172.21.15.83:0/1421686973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00cdd0 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1682a0 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc193fff640 1 -- 172.21.15.83:0/1421686973 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc19c01a070 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc193fff640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 0x7fc198071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc193fff640 1 -- 172.21.15.83:0/1421686973 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fc19c08f650 con 0x7fc1ac0af8b0 2024-05-17T22:04:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 0x7fc198071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 0x7fc198071910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac167a60 tx=0x7fc1a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc1b92af000 1 Processor -- start 2024-05-17T22:04:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.185+0000 7fc1b92af000 1 -- start start 2024-05-17T22:04:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb987600 0x562bbbad7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb970d30 0x562bbbad8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bbb975ca0 con 0x562bbb991fa0 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562bbb8e8110 con 0x562bbb987600 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562bbb8d7cb0 con 0x562bbb970d30 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36228/0 (socket says 172.21.15.83:36228) 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/821625589 learned_addr learned my addr 172.21.15.83:0/821625589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb987600 0x562bbbad7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1abfff640 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb970d30 0x562bbbad8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb970d30 msgr2=0x562bbbad8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb970d30 0x562bbbad8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb987600 msgr2=0x562bbbad7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb987600 0x562bbbad7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/821625589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbb89c370 con 0x562bbb991fa0 2024-05-17T22:04:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006cb0 tx=0x7fc19c05b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d2fb6f9daa2472 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1917fa640 1 -- 172.21.15.83:0/821625589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c021040 con 0x562bbb991fa0 2024-05-17T22:04:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1917fa640 1 -- 172.21.15.83:0/821625589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c05bd50 con 0x562bbb991fa0 2024-05-17T22:04:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1917fa640 1 -- 172.21.15.83:0/821625589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c05b1d0 con 0x562bbb991fa0 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- 172.21.15.83:0/821625589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 msgr2=0x562bbbad78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006cb0 tx=0x7fc19c05b4c0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- 172.21.15.83:0/821625589 shutdown_connections 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb970d30 0x562bbbad8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb987600 0x562bbbad7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 --2- 172.21.15.83:0/821625589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb991fa0 0x562bbbad78b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- 172.21.15.83:0/821625589 >> 172.21.15.83:0/821625589 conn(0x562bbb599460 msgr2=0x562bbba8ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- 172.21.15.83:0/821625589 shutdown_connections 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- 172.21.15.83:0/821625589 wait complete. 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 -- start start 2024-05-17T22:04:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.189+0000 7fc1b92af000 1 Processor -- start 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbba94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbba99460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bbb893d40 con 0x562bbb970d30 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562bbb8dbcf0 con 0x562bbb991fa0 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562bbb880e90 con 0x562bbb987600 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 10 client.0 _ll_register_callbacks cb 0x562bbb8b5fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36236/0 (socket says 172.21.15.83:36236) 2024-05-17T22:04:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/2982464602 learned_addr learned my addr 172.21.15.83:0/2982464602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1abfff640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbba99460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbba94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 msgr2=0x562bbba94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbba94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 msgr2=0x562bbba99460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbba99460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/2982464602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbb8a66e0 con 0x562bbb970d30 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc19c08dae0 tx=0x7fc19c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a20b87e0c064ce server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c021040 con 0x562bbb970d30 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fc19c08b040 con 0x562bbb970d30 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c05bd50 con 0x562bbb970d30 2024-05-17T22:04:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc19c060100 con 0x562bbb970d30 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 10 client.7149 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x562bbb896690 con 0x562bbb970d30 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c05bd50 con 0x562bbb970d30 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7fc19c075030 con 0x562bbb970d30 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 10 client.7149 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 10 client.7149 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 20 client.7149 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb57e7d0 0x562bbbade920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 4 client.7149 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:04:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bbb873a00 con 0x562bbb57e7d0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb57e7d0 0x562bbbade920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb57e7d0 0x562bbbade920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591f011ba089081c server_cookie=19de65b0485312b5 in_seq=0 out_seq=0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 10 client.7149 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 1 -- 172.21.15.83:0/2982464602 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562bbb873a00 con 0x562bbb57e7d0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 10 client.7149 handle_command_reply: tid=0 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 client.7149 shutdown 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 msgr2=0x562bbba942d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc19c08dae0 tx=0x7fc19c000bd0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 shutdown_connections 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb57e7d0 0x562bbbade920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbba99460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbba94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc18a7fc640 0 client.7149 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:37.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 --2- 172.21.15.83:0/2982464602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbba942d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 >> 172.21.15.83:0/2982464602 conn(0x562bbb599460 msgr2=0x562bbba8ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 shutdown_connections 2024-05-17T22:04:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.193+0000 7fc1b92af000 1 -- 172.21.15.83:0/2982464602 wait complete. 2024-05-17T22:04:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.197+0000 7fc1b92af000 20 client.7149 trim_cache size 0 max 16384 2024-05-17T22:04:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 Processor -- start 2024-05-17T22:04:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 -- start start 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb970d30 0x562bbba8c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb991fa0 0x562bbba8cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bbb882900 con 0x562bbb987600 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562bbb893d40 con 0x562bbb970d30 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562bbb8dbcf0 con 0x562bbb991fa0 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36250/0 (socket says 172.21.15.83:36250) 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 -- 172.21.15.83:0/3200870254 learned_addr learned my addr 172.21.15.83:0/3200870254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1abfff640 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb991fa0 0x562bbba8cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 -- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb991fa0 msgr2=0x562bbba8cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb970d30 0x562bbba8c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb991fa0 0x562bbba8cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 -- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb970d30 msgr2=0x562bbba8c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb970d30 0x562bbba8c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 -- 172.21.15.83:0/3200870254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbba997d0 con 0x562bbb987600 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000f70 tx=0x7fc1a0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2963f9759f8dea38 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc190ff9640 1 -- 172.21.15.83:0/3200870254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a001d040 con 0x562bbb987600 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc190ff9640 1 -- 172.21.15.83:0/3200870254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a0029020 con 0x562bbb987600 2024-05-17T22:04:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.213+0000 7fc190ff9640 1 -- 172.21.15.83:0/3200870254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0005cb0 con 0x562bbb987600 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3200870254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 msgr2=0x562bbba8c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0000f70 tx=0x7fc1a0016040 comp rx=0 tx=0).stop 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3200870254 shutdown_connections 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb991fa0 0x562bbba8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb987600 0x562bbba8c9a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3200870254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb970d30 0x562bbba8c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3200870254 >> 172.21.15.83:0/3200870254 conn(0x562bbb599460 msgr2=0x562bbbaedea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3200870254 shutdown_connections 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3200870254 wait complete. 2024-05-17T22:04:37.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- start start 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 Processor -- start 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbbb4a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbbb4a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bbb8e6ee0 con 0x562bbb970d30 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562bbb880e90 con 0x562bbb991fa0 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562bbba97c20 con 0x562bbb987600 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 10 client.0 _ll_register_callbacks cb 0x7fc19c05e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36254/0 (socket says 172.21.15.83:36254) 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/3292790949 learned_addr learned my addr 172.21.15.83:0/3292790949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1abfff640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbbb4a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbbb4a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 msgr2=0x562bbbb4a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbbb4a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 msgr2=0x562bbbb4a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbbb4a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 -- 172.21.15.83:0/3292790949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbb95ecb0 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1abfff640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbbb4a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1ab7fe640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc19c08f970 tx=0x7fc19c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c5b0a5288c37fa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc18a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c021040 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fc19c076ce0 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a0029020 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc19c05bd50 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 10 client.7155 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.217+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x562bbb8974f0 con 0x562bbb970d30 2024-05-17T22:04:37.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a0029020 con 0x562bbb970d30 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7fc19c083020 con 0x562bbb970d30 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 10 client.7155 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 10 client.7155 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 20 client.7155 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb56c830 0x562bbbb05c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 4 client.7155 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "status"}) -- 0x562bbb8ea060 con 0x562bbb56c830 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb56c830 0x562bbbb05c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 10 client.7155 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1aaffd640 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb56c830 0x562bbbb05c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6455f35b7c61da1d server_cookie=7eea0a2e72de875e in_seq=0 out_seq=0 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 1 -- 172.21.15.83:0/3292790949 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562bbb8ea060 con 0x562bbb56c830 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 10 client.7155 handle_command_reply: tid=0 2024-05-17T22:04:37.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 client.7155 shutdown 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 msgr2=0x562bbbb49b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc19c08f970 tx=0x7fc19c009550 comp rx=0 tx=0).stop 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 shutdown_connections 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562bbb56c830 0x562bbbb05c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562bbb991fa0 0x562bbbb4a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562bbb987600 0x562bbbb4a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 --2- 172.21.15.83:0/3292790949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bbb970d30 0x562bbbb49b80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc18a7fc640 0 client.7155 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 >> 172.21.15.83:0/3292790949 conn(0x562bbb599460 msgr2=0x562bbbad7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 shutdown_connections 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 1 -- 172.21.15.83:0/3292790949 wait complete. 2024-05-17T22:04:37.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b92af000 20 client.7155 trim_cache size 0 max 16384 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7083, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.595758515, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 72, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 85, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.5125280539999997 2024-05-17T22:04:37.216 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:04:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 msgr2=0x7fc198071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 0x7fc198071910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac167a60 tx=0x7fc1a0009040 comp rx=0 tx=0).stop 2024-05-17T22:04:37.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 msgr2=0x7fc1ac0c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e930 tx=0x7fc19c00edf0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 shutdown_connections 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc19806f400 0x7fc198071910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1ac0b4bb0 0x7fc1ac1687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1ac0b3be0 0x7fc1ac15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 --2- 172.21.15.83:0/1421686973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac0af8b0 0x7fc1ac0c6200 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 >> 172.21.15.83:0/1421686973 conn(0x7fc1ac0b4090 msgr2=0x7fc1ac0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 shutdown_connections 2024-05-17T22:04:37.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.221+0000 7fc1b2035640 1 -- 172.21.15.83:0/1421686973 wait complete. 2024-05-17T22:04:37.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:04:37.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 Processor -- start 2024-05-17T22:04:37.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 -- start start 2024-05-17T22:04:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd6841605a0 0x7fd6841609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6841465d0 con 0x7fd68406ffd0 2024-05-17T22:04:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd684160ef0 con 0x7fd6841461c0 2024-05-17T22:04:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68c7eb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd68406ef30 con 0x7fd6841605a0 2024-05-17T22:04:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd689d86640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd68406ffd0 0x7fd6840703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52076/0 (socket says 172.21.15.83:52076) 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd689d86640 1 -- 172.21.15.83:0/2436862799 learned_addr learned my addr 172.21.15.83:0/2436862799 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68ad88640 1 --2- 172.21.15.83:0/2436862799 >> v2:172.21.15.83:3300/0 conn(0x7fd6841605a0 0x7fd6841609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd689585640 1 -- 172.21.15.83:0/2436862799 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869487173 0 0) 0x7fd6841465d0 con 0x7fd68406ffd0 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd689585640 1 -- 172.21.15.83:0/2436862799 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd670003530 con 0x7fd68406ffd0 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68ad88640 1 -- 172.21.15.83:0/2436862799 >> v1:172.21.15.155:6789/0 conn(0x7fd6841461c0 legacy=0x7fd68415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68ad88640 1 -- 172.21.15.83:0/2436862799 >> v1:172.21.15.83:6789/0 conn(0x7fd68406ffd0 legacy=0x7fd6840703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.385+0000 7fd68ad88640 1 -- 172.21.15.83:0/2436862799 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68406f210 con 0x7fd6841605a0 2024-05-17T22:04:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68ad88640 1 --2- 172.21.15.83:0/2436862799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841609b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009d90 tx=0x7fd67c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3d121ec9ef33b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd689585640 1 -- 172.21.15.83:0/2436862799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0b2070 con 0x7fd6841605a0 2024-05-17T22:04:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd689585640 1 -- 172.21.15.83:0/2436862799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c0a5a00 con 0x7fd6841605a0 2024-05-17T22:04:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd689585640 1 -- 172.21.15.83:0/2436862799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0a5dd0 con 0x7fd6841605a0 2024-05-17T22:04:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2436862799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 msgr2=0x7fd6841609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2436862799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841609b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009d90 tx=0x7fd67c01c940 comp rx=0 tx=0).stop 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2436862799 shutdown_connections 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2436862799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841609b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2436862799 >> 172.21.15.83:0/2436862799 conn(0x7fd68406cd90 msgr2=0x7fd6840b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2436862799 shutdown_connections 2024-05-17T22:04:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2436862799 wait complete. 2024-05-17T22:04:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 Processor -- start 2024-05-17T22:04:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.389+0000 7fd68c7eb640 1 -- start start 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68406ffd0 0x7fd6841f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 0x7fd68415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6840d10a0 con 0x7fd6841605a0 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd68415e880 con 0x7fd68406ffd0 2024-05-17T22:04:37.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd6840d1220 con 0x7fd6841461c0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36274/0 (socket says 172.21.15.83:36274) 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 -- 172.21.15.83:0/2253581507 learned_addr learned my addr 172.21.15.83:0/2253581507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68a587640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68406ffd0 0x7fd6841f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd689d86640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 0x7fd68415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 -- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 msgr2=0x7fd68415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 0x7fd68415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 -- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68406ffd0 msgr2=0x7fd6841f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68406ffd0 0x7fd6841f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 -- 172.21.15.83:0/2253581507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd684161c30 con 0x7fd6841605a0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd689d86640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 0x7fd68415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68ad88640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009d90 tx=0x7fd67c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0b2040 con 0x7fd6841605a0 2024-05-17T22:04:37.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c0bf070 con 0x7fd6841605a0 2024-05-17T22:04:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6840aef80 con 0x7fd6841605a0 2024-05-17T22:04:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0b8890 con 0x7fd6841605a0 2024-05-17T22:04:37.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6840af540 con 0x7fd6841605a0 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6841609b0 con 0x7fd6841605a0 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.393+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd67c0b8a30 con 0x7fd6841605a0 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.397+0000 7fd67affd640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 0x7fd668071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.397+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fd67c12a590 con 0x7fd6841605a0 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.397+0000 7fd68a587640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 0x7fd668071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:37.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.397+0000 7fd68a587640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 0x7fd668071910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd6840b11f0 tx=0x7fd680009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:37.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.401+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd67c016560 con 0x7fd6841605a0 2024-05-17T22:04:37.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.589+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd6840b41a0 con 0x7fd6841605a0 2024-05-17T22:04:37.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.589+0000 7fd67affd640 1 -- 172.21.15.83:0/2253581507 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) ==== 53+0+73 (secure 0 0 0) 0x7fd67c0f7390 con 0x7fd6841605a0 2024-05-17T22:04:37.578 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:04:37.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.589+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 msgr2=0x7fd668071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.589+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 0x7fd668071910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd6840b11f0 tx=0x7fd680009040 comp rx=0 tx=0).stop 2024-05-17T22:04:37.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 msgr2=0x7fd6841616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:37.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009d90 tx=0x7fd67c0196e0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 shutdown_connections 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6841605a0 0x7fd6841616f0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6841461c0 0x7fd68415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd66806f400 0x7fd668071910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 --2- 172.21.15.83:0/2253581507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68406ffd0 0x7fd6841f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 >> 172.21.15.83:0/2253581507 conn(0x7fd68406cd90 msgr2=0x7fd6840730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 shutdown_connections 2024-05-17T22:04:37.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37.593+0000 7fd68c7eb640 1 -- 172.21.15.83:0/2253581507 wait complete. 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:04:37.599 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:04:37.600 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:04:37.600 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:04:37.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:37.600 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.606 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42067sec preferred_lft 42067sec 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:04:37.607 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:04:37.608 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:04:37.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:37.609 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:04:37.658 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:04:37.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:37.659 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:04:37.709 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:04:37.709 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:04:37.709 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:04:37.710 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:04:37.710 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:04:37.710 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:04:37.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:04:37.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:04:37.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:04:37.711 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:04:37.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:04:37.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:04:37.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:37.826 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:04:37.874 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:04:37.874 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:04:37.874 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:04:37.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:37.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:04:37.927 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:04:37.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:04:37.927 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:37.973 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:04:37.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:04:37.974 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:38.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:04:38.079 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:04:38.126 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:04:38.126 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:04:38.126 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:04:38.127 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:38.128 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:38.129 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:38.129 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:38.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:38.130 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:04:38.178 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:04:38.178 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:04:38.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:04:38.237 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:04:38.286 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:04:38.287 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:38.288 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:38.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:38.289 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:04:38.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:04:38.341+0000 7ff6d0edd580 -1 init, newargv = 0x7ff6c001b550 newargc=15 2024-05-17T22:04:38.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[44949]: starting ceph client 2024-05-17T22:04:38.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[44949]: starting fuse 2024-05-17T22:04:38.615 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:04:38.625+0000 7f3030588640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:04:39.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:04:39.451 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:04:39.504 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:04:39.504 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:04:39.505 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:04:39.506 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:04:39.507 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:39.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:39.508 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:04:39.554 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:04:39.554 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:04:39.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:04:39.554 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:04:39.554 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:04:39.554 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:04:39.554 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:04:39.555 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:04:39.556 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:04:39.680 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.44949.asok 2024-05-17T22:04:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:04:39.692 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44949.asok 2024-05-17T22:04:39.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:39.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44949.asok status 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:04:39.877 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "44949", 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7173, 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7173 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:04:39.878 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2033872555 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2033872555 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.7173 192.168.0.1:0/2033872555", 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2033872555", 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 72, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 85, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:04:39.879 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:04:39.880 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:04:39.880 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:04:39.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:39.893 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:04:39.893 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:39.900 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:04:39.900 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:39.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:39.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:04:39.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:39.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-17T22:04:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 Processor -- start 2024-05-17T22:04:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 -- start start 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32240d2280 0x7f32240d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f32240d49a0 0x7f32240d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32240d2bd0 con 0x7f322415fd80 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3224161a40 con 0x7f32240d49a0 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f322a050640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f32240d3c10 con 0x7f32240d2280 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f32237fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f322415fd80 0x7f3224161330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:52078/0 (socket says 172.21.15.83:52078) 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f32237fe640 1 -- 172.21.15.83:0/3597659707 learned_addr learned my addr 172.21.15.83:0/3597659707 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3223fff640 1 --2- 172.21.15.83:0/3597659707 >> v2:172.21.15.83:3300/0 conn(0x7f32240d49a0 0x7f32240d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f321bfff640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32240d2280 0x7f32240d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3222ffd640 1 -- 172.21.15.83:0/3597659707 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682470883 0 0) 0x7f3214001120 con 0x7f322415fd80 2024-05-17T22:04:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3222ffd640 1 -- 172.21.15.83:0/3597659707 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3210003530 con 0x7f322415fd80 2024-05-17T22:04:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3223fff640 1 -- 172.21.15.83:0/3597659707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32240d2280 msgr2=0x7f32240d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3223fff640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32240d2280 0x7f32240d2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3223fff640 1 -- 172.21.15.83:0/3597659707 >> v1:172.21.15.83:6789/0 conn(0x7f322415fd80 legacy=0x7f3224161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.161+0000 7f3223fff640 1 -- 172.21.15.83:0/3597659707 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32240d11e0 con 0x7f32240d49a0 2024-05-17T22:04:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f3223fff640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d49a0 0x7f32240d4db0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f321c009dc0 tx=0x7f321c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24f7fca7962f840 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f3222ffd640 1 -- 172.21.15.83:0/3597659707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b2070 con 0x7f32240d49a0 2024-05-17T22:04:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f3222ffd640 1 -- 172.21.15.83:0/3597659707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c0a5900 con 0x7f32240d49a0 2024-05-17T22:04:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f3222ffd640 1 -- 172.21.15.83:0/3597659707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a5cd0 con 0x7f32240d49a0 2024-05-17T22:04:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 -- 172.21.15.83:0/3597659707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d49a0 msgr2=0x7f32240d4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d49a0 0x7f32240d4db0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f321c009dc0 tx=0x7f321c01c950 comp rx=0 tx=0).stop 2024-05-17T22:04:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 -- 172.21.15.83:0/3597659707 shutdown_connections 2024-05-17T22:04:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d49a0 0x7f32240d4db0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 --2- 172.21.15.83:0/3597659707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f32240d2280 0x7f32240d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 -- 172.21.15.83:0/3597659707 >> 172.21.15.83:0/3597659707 conn(0x7f3224074ac0 msgr2=0x7f322406de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 -- 172.21.15.83:0/3597659707 shutdown_connections 2024-05-17T22:04:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.165+0000 7f322a050640 1 -- 172.21.15.83:0/3597659707 wait complete. 2024-05-17T22:04:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 Processor -- start 2024-05-17T22:04:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- start start 2024-05-17T22:04:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32240d49a0 0x7f32240b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322415fd80 0x7f32241575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32240d0cc0 con 0x7f32240d2280 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f32240d52f0 con 0x7f32240d49a0 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f32240d0e40 con 0x7f322415fd80 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36292/0 (socket says 172.21.15.83:36292) 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 -- 172.21.15.83:0/4270400030 learned_addr learned my addr 172.21.15.83:0/4270400030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f3223fff640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322415fd80 0x7f32241575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f321bfff640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32240d49a0 0x7f32240b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 -- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322415fd80 msgr2=0x7f32241575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322415fd80 0x7f32241575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 -- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32240d49a0 msgr2=0x7f32240b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32240d49a0 0x7f32240b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 -- 172.21.15.83:0/4270400030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322414cb00 con 0x7f32240d2280 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f32237fe640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f321400c700 tx=0x7f321400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f3220ff9640 1 -- 172.21.15.83:0/4270400030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214017070 con 0x7f32240d2280 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f3220ff9640 1 -- 172.21.15.83:0/4270400030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214001d40 con 0x7f32240d2280 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32240cdf60 con 0x7f32240d2280 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f3220ff9640 1 -- 172.21.15.83:0/4270400030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214005cc0 con 0x7f32240d2280 2024-05-17T22:04:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.169+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32240cd1a0 con 0x7f32240d2280 2024-05-17T22:04:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f3220ff9640 1 -- 172.21.15.83:0/4270400030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f321401a070 con 0x7f32240d2280 2024-05-17T22:04:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f3220ff9640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 0x7f3204071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f3220ff9640 1 -- 172.21.15.83:0/4270400030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f321408e520 con 0x7f32240d2280 2024-05-17T22:04:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f321bfff640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 0x7f3204071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f321bfff640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 0x7f3204071910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f32240b4850 tx=0x7f320c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 Processor -- start 2024-05-17T22:04:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 -- start start 2024-05-17T22:04:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca79367e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca791ffdf0 0x55ca793683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca79179220 con 0x55ca79224240 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca79168dc0 con 0x55ca79215f60 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ca7916ce00 con 0x55ca791ffdf0 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36306/0 (socket says 172.21.15.83:36306) 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.173+0000 7f32237fe640 1 -- 172.21.15.83:0/3387657189 learned_addr learned my addr 172.21.15.83:0/3387657189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 -- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca791ffdf0 msgr2=0x55ca793683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca791ffdf0 0x55ca793683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 -- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 msgr2=0x55ca79367e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca79367e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 -- 172.21.15.83:0/3387657189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca79144bf0 con 0x55ca79224240 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3214005840 tx=0x7f321405a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6782119b7aca7e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32197fa640 1 -- 172.21.15.83:0/3387657189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140633d0 con 0x55ca79224240 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32197fa640 1 -- 172.21.15.83:0/3387657189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214063a80 con 0x55ca79224240 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32197fa640 1 -- 172.21.15.83:0/3387657189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321408e2d0 con 0x55ca79224240 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- 172.21.15.83:0/3387657189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 msgr2=0x55ca79367950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3214005840 tx=0x7f321405a930 comp rx=0 tx=0).stop 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- 172.21.15.83:0/3387657189 shutdown_connections 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca791ffdf0 0x55ca793683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca79367e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- 172.21.15.83:0/3387657189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79224240 0x55ca79367950 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- 172.21.15.83:0/3387657189 >> 172.21.15.83:0/3387657189 conn(0x55ca78e2a810 msgr2=0x55ca7931c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- 172.21.15.83:0/3387657189 shutdown_connections 2024-05-17T22:04:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- 172.21.15.83:0/3387657189 wait complete. 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- start start 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 Processor -- start 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca7936e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca7936e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca790f1f30 con 0x55ca791ffdf0 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca79165740 con 0x55ca79215f60 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ca790efcc0 con 0x55ca79224240 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32312ca000 10 client.0 _ll_register_callbacks cb 0x55ca793096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36320/0 (socket says 172.21.15.83:36320) 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.177+0000 7f32237fe640 1 -- 172.21.15.83:0/2477410487 learned_addr learned my addr 172.21.15.83:0/2477410487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f321bfff640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca7936e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f3223fff640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca7936e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 -- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 msgr2=0x55ca7936e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca7936e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 -- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 msgr2=0x55ca7936e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca7936e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 -- 172.21.15.83:0/2477410487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca79122460 con 0x55ca791ffdf0 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32237fe640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3214057fd0 tx=0x7f3214000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566becda3567a5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214074040 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f321408c380 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3214063a80 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f32140633d0 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 10 client.7188 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ca791279e0 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214063a80 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f3214085b00 con 0x55ca791ffdf0 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 10 client.7188 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 10 client.7188 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 20 client.7188 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78e0fb80 0x55ca793781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 4 client.7188 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca79104d40 con 0x55ca78e0fb80 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.181+0000 7f321bfff640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78e0fb80 0x55ca793781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32027fc640 10 client.7188 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f321bfff640 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78e0fb80 0x55ca793781b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792c2b5ea0f7eb00 server_cookie=6d18d0213baec4b0 in_seq=0 out_seq=0 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32027fc640 1 -- 172.21.15.83:0/2477410487 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca79104d40 con 0x55ca78e0fb80 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32027fc640 10 client.7188 handle_command_reply: tid=0 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 client.7188 shutdown 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 msgr2=0x55ca7936ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3214057fd0 tx=0x7f3214000bd0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 shutdown_connections 2024-05-17T22:04:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78e0fb80 0x55ca793781b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca7936e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79215f60 0x55ca7936e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 --2- 172.21.15.83:0/2477410487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca7936ddd0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32027fc640 0 client.7188 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 >> 172.21.15.83:0/2477410487 conn(0x55ca78e2a810 msgr2=0x55ca7931c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 shutdown_connections 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 1 -- 172.21.15.83:0/2477410487 wait complete. 2024-05-17T22:04:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.185+0000 7f32312ca000 20 client.7188 trim_cache size 0 max 16384 2024-05-17T22:04:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.201+0000 7f32312ca000 1 Processor -- start 2024-05-17T22:04:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.201+0000 7f32312ca000 1 -- start start 2024-05-17T22:04:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79215f60 0x55ca79367300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79224240 0x55ca79371e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3224204860 con 0x55ca791ffdf0 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f322414d6c0 con 0x55ca79224240 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3224204330 con 0x55ca79215f60 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36328/0 (socket says 172.21.15.83:36328) 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 -- 172.21.15.83:0/1929134697 learned_addr learned my addr 172.21.15.83:0/1929134697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 -- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79215f60 msgr2=0x55ca79367300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79215f60 0x55ca79367300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 -- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79224240 msgr2=0x55ca79371e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f3223fff640 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79224240 0x55ca79371e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79224240 0x55ca79371e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 -- 172.21.15.83:0/1929134697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca791eda10 con 0x55ca791ffdf0 2024-05-17T22:04:40.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32237fe640 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f321400a430 tx=0x7f3214087070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1c0dd50732ba40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f3218ff9640 1 -- 172.21.15.83:0/1929134697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321407f070 con 0x55ca791ffdf0 2024-05-17T22:04:40.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f3218ff9640 1 -- 172.21.15.83:0/1929134697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214089030 con 0x55ca791ffdf0 2024-05-17T22:04:40.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f3218ff9640 1 -- 172.21.15.83:0/1929134697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140633d0 con 0x55ca791ffdf0 2024-05-17T22:04:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- 172.21.15.83:0/1929134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 msgr2=0x55ca79366dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f321400a430 tx=0x7f3214087070 comp rx=0 tx=0).stop 2024-05-17T22:04:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- 172.21.15.83:0/1929134697 shutdown_connections 2024-05-17T22:04:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca79224240 0x55ca79371e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79215f60 0x55ca79367300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 --2- 172.21.15.83:0/1929134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca791ffdf0 0x55ca79366dc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- 172.21.15.83:0/1929134697 >> 172.21.15.83:0/1929134697 conn(0x55ca78e2a810 msgr2=0x55ca793666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- 172.21.15.83:0/1929134697 shutdown_connections 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- 172.21.15.83:0/1929134697 wait complete. 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 -- start start 2024-05-17T22:04:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.205+0000 7f32312ca000 1 Processor -- start 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 0x55ca793d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca793da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca79137dd0 con 0x55ca79215f60 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca79177ff0 con 0x55ca791ffdf0 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ca7916c110 con 0x55ca79224240 2024-05-17T22:04:40.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 10 client.0 _ll_register_callbacks cb 0x7f3214005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36330/0 (socket says 172.21.15.83:36330) 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 -- 172.21.15.83:0/4123273208 learned_addr learned my addr 172.21.15.83:0/4123273208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32237fe640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 0x55ca793d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f3223fff640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca793da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 -- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 msgr2=0x55ca793da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca793da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 -- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 msgr2=0x55ca793d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 0x55ca793d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 -- 172.21.15.83:0/4123273208 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca79125dc0 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32237fe640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 0x55ca793d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f321bfff640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e730 tx=0x7f320c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d377d903776a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c00f040 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f320c0262f0 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3214089030 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f320c036050 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 10 client.7194 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55ca791103e0 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214089030 con 0x55ca79215f60 2024-05-17T22:04:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f320c039020 con 0x55ca79215f60 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 10 client.7194 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 10 client.7194 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 20 client.7194 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78dfdbe0 0x55ca793957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 4 client.7194 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55ca7917b170 con 0x55ca78dfdbe0 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32237fe640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78dfdbe0 0x55ca793957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 10 client.7194 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32237fe640 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78dfdbe0 0x55ca793957f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6b802a45a938f3 server_cookie=51aff0fc252df73b in_seq=0 out_seq=0 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 1 -- 172.21.15.83:0/4123273208 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ca7917b170 con 0x55ca78dfdbe0 2024-05-17T22:04:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 10 client.7194 handle_command_reply: tid=0 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 client.7194 shutdown 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 msgr2=0x55ca793da3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e730 tx=0x7f320c00e7e0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 shutdown_connections 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x55ca78dfdbe0 0x55ca793957f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ca79224240 0x55ca793da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca79215f60 0x55ca793da3d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 --2- 172.21.15.83:0/4123273208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca791ffdf0 0x55ca793d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32027fc640 0 client.7194 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 >> 172.21.15.83:0/4123273208 conn(0x55ca78e2a810 msgr2=0x55ca79367070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.209+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 shutdown_connections 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f32312ca000 1 -- 172.21.15.83:0/4123273208 wait complete. 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f32312ca000 20 client.7194 trim_cache size 0 max 16384 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "27db2365-1d29-4e50-9133-c3e39b19f1f5", 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:04:40.207 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 msgr2=0x7f3204071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 0x7f3204071910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f32240b4850 tx=0x7f320c008040 comp rx=0 tx=0).stop 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 msgr2=0x7f322414c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f321400c700 tx=0x7f321400cbc0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 shutdown_connections 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322415fd80 0x7f32241575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f320406f400 0x7f3204071910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f32240d49a0 0x7f32240b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 --2- 172.21.15.83:0/4270400030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32240d2280 0x7f322414c5c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 >> 172.21.15.83:0/4270400030 conn(0x7f3224074ac0 msgr2=0x7f322415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 shutdown_connections 2024-05-17T22:04:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f322a050640 1 -- 172.21.15.83:0/4270400030 wait complete. 2024-05-17T22:04:40.210 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:04:40.213+0000 7f302c580640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5558616c0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:04:40.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:04:40.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.377+0000 7f860192e640 1 Processor -- start 2024-05-17T22:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.377+0000 7f860192e640 1 -- start start 2024-05-17T22:04:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85fc06eec0 0x7f85fc0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc15be70 0x7f85fc1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 0x7f85fc0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f85fc06f2d0 con 0x7f85fc06eec0 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f85fc06f5a0 con 0x7f85fc1606f0 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f860192e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc06f880 con 0x7f85fc15be70 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85fc06eec0 0x7f85fc0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85fc06eec0 0x7f85fc0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36344/0 (socket says 172.21.15.83:36344) 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 -- 172.21.15.83:0/156170971 learned_addr learned my addr 172.21.15.83:0/156170971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85f27fc640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc15be70 0x7f85fc1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85fb7fe640 1 --2- 172.21.15.83:0/156170971 >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 0x7f85fc0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 -- 172.21.15.83:0/156170971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc15be70 msgr2=0x7f85fc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc15be70 0x7f85fc1601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 -- 172.21.15.83:0/156170971 >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 msgr2=0x7f85fc0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 --2- 172.21.15.83:0/156170971 >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 0x7f85fc0b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 -- 172.21.15.83:0/156170971 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc06fb60 con 0x7f85fc06eec0 2024-05-17T22:04:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85fb7fe640 1 --2- 172.21.15.83:0/156170971 >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 0x7f85fc0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.381+0000 7f85faffd640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc0b08a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f85fc073ae0 tx=0x7f85ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b39cb3171af6e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f85fa7fc640 1 -- 172.21.15.83:0/156170971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b0070 con 0x7f85fc06eec0 2024-05-17T22:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f85fa7fc640 1 -- 172.21.15.83:0/156170971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec0ab600 con 0x7f85fc06eec0 2024-05-17T22:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f85fa7fc640 1 -- 172.21.15.83:0/156170971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0ab9d0 con 0x7f85fc06eec0 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- 172.21.15.83:0/156170971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 msgr2=0x7f85fc0b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc0b08a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f85fc073ae0 tx=0x7f85ec01c520 comp rx=0 tx=0).stop 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- 172.21.15.83:0/156170971 shutdown_connections 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 --2- 172.21.15.83:0/156170971 >> v2:172.21.15.155:3300/0 conn(0x7f85fc1606f0 0x7f85fc0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc15be70 0x7f85fc1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 --2- 172.21.15.83:0/156170971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc0b08a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- 172.21.15.83:0/156170971 >> 172.21.15.83:0/156170971 conn(0x7f85fc06cd90 msgr2=0x7f85fc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- 172.21.15.83:0/156170971 shutdown_connections 2024-05-17T22:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- 172.21.15.83:0/156170971 wait complete. 2024-05-17T22:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 Processor -- start 2024-05-17T22:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.385+0000 7f860192e640 1 -- start start 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85fc15be70 0x7f85fc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc1606f0 0x7f85fc0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc073210 con 0x7f85fc06eec0 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc073090 con 0x7f85fc15be70 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc073390 con 0x7f85fc1606f0 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36348/0 (socket says 172.21.15.83:36348) 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 -- 172.21.15.83:0/652648140 learned_addr learned my addr 172.21.15.83:0/652648140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 -- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc1606f0 msgr2=0x7f85fc0afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc1606f0 0x7f85fc0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 -- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85fc15be70 msgr2=0x7f85fc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85fc15be70 0x7f85fc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 -- 172.21.15.83:0/652648140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc157a30 con 0x7f85fc06eec0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85faffd640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0ae040 tx=0x7f85ec01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 -- 172.21.15.83:0/652648140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b0040 con 0x7f85fc06eec0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 -- 172.21.15.83:0/652648140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec0bc070 con 0x7f85fc06eec0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc157d10 con 0x7f85fc06eec0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 -- 172.21.15.83:0/652648140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0b6880 con 0x7f85fc06eec0 2024-05-17T22:04:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1582d0 con 0x7f85fc06eec0 2024-05-17T22:04:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 -- 172.21.15.83:0/652648140 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f85ec0ab4c0 con 0x7f85fc06eec0 2024-05-17T22:04:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 0x7f85dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f27fc640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 0x7f85dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f8ff9640 1 -- 172.21.15.83:0/652648140 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f85ec129690 con 0x7f85fc06eec0 2024-05-17T22:04:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.389+0000 7f85f27fc640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 0x7f85dc071910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0b0950 tx=0x7f85e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 Processor -- start 2024-05-17T22:04:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- start start 2024-05-17T22:04:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x56265819be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658023090 0x56265819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56265803aa90 con 0x562658022880 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562657fae120 con 0x562658035f80 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562657f9dcc0 con 0x562658023090 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36360/0 (socket says 172.21.15.83:36360) 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 -- 172.21.15.83:0/24792027 learned_addr learned my addr 172.21.15.83:0/24792027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85faffd640 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x56265819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 -- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658023090 msgr2=0x56265819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658023090 0x56265819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 -- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 msgr2=0x56265819be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x56265819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 -- 172.21.15.83:0/24792027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562657f5a110 con 0x562658022880 2024-05-17T22:04:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f27fc640 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d8e0 tx=0x7f85e4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9abba1a4be2caed server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f17fa640 1 -- 172.21.15.83:0/24792027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4024070 con 0x562658022880 2024-05-17T22:04:40.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f17fa640 1 -- 172.21.15.83:0/24792027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e400de60 con 0x562658022880 2024-05-17T22:04:40.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f85f17fa640 1 -- 172.21.15.83:0/24792027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4014e00 con 0x562658022880 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- 172.21.15.83:0/24792027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 msgr2=0x56265819c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d8e0 tx=0x7f85e4010000 comp rx=0 tx=0).stop 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- 172.21.15.83:0/24792027 shutdown_connections 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658023090 0x56265819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x56265819c360 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 --2- 172.21.15.83:0/24792027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x56265819be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- 172.21.15.83:0/24792027 >> 172.21.15.83:0/24792027 conn(0x562658071010 msgr2=0x5626581515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- 172.21.15.83:0/24792027 shutdown_connections 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.393+0000 7f8608ba8000 1 -- 172.21.15.83:0/24792027 wait complete. 2024-05-17T22:04:40.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- start start 2024-05-17T22:04:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 Processor -- start 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 0x562658158f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x56265815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562657f59ec0 con 0x562658022880 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562657fa1d00 con 0x562658023090 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562657c67680 con 0x562658035f80 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 10 client.0 _ll_register_callbacks cb 0x5626580551c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36376/0 (socket says 172.21.15.83:36376) 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 -- 172.21.15.83:0/2084623480 learned_addr learned my addr 172.21.15.83:0/2084623480 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 0x562658158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85fb7fe640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x56265815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 -- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 msgr2=0x56265815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x56265815dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 -- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 msgr2=0x562658158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 0x562658158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 -- 172.21.15.83:0/2084623480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562657f5c890 con 0x562658022880 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 0x562658158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85faffd640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0f2410 tx=0x7f85ec0f5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6205294ef9a30aa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec0fa1d0 con 0x562658022880 2024-05-17T22:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e400de60 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f85ec0f5080 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f85ec10f370 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 10 client.7206 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x562657f5e650 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e400de60 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f85ec0b3030 con 0x562658022880 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 10 client.7206 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 10 client.7206 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 20 client.7206 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c44930 0x5626581a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 4 client.7206 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562657f39c40 con 0x562657c44930 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c44930 0x5626581a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85da7fc640 10 client.7206 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.397+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c44930 0x5626581a2e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd404f289d1bac2b server_cookie=b7591be11908fb4c in_seq=0 out_seq=0 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f85da7fc640 1 -- 172.21.15.83:0/2084623480 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562657f39c40 con 0x562657c44930 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f85da7fc640 10 client.7206 handle_command_reply: tid=0 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 client.7206 shutdown 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 msgr2=0x562658158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0f2410 tx=0x7f85ec0f5650 comp rx=0 tx=0).stop 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 shutdown_connections 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c44930 0x5626581a2e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x56265815dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658023090 0x562658158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f85da7fc640 0 client.7206 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2084623480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658022880 0x562658158a10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 >> 172.21.15.83:0/2084623480 conn(0x562658071010 msgr2=0x5626581515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 shutdown_connections 2024-05-17T22:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 1 -- 172.21.15.83:0/2084623480 wait complete. 2024-05-17T22:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.401+0000 7f8608ba8000 20 client.7206 trim_cache size 0 max 16384 2024-05-17T22:04:40.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.417+0000 7f8608ba8000 1 Processor -- start 2024-05-17T22:04:40.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.417+0000 7f8608ba8000 1 -- start start 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658022880 0x5626582133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x562658213e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc203810 con 0x562658023090 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1fb590 con 0x562658035f80 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc1fa8f0 con 0x562658022880 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36382/0 (socket says 172.21.15.83:36382) 2024-05-17T22:04:40.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 -- 172.21.15.83:0/2134417165 learned_addr learned my addr 172.21.15.83:0/2134417165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85faffd640 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658022880 0x5626582133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 -- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658022880 msgr2=0x5626582133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658022880 0x5626582133c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 -- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 msgr2=0x562658213e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85fb7fe640 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x562658213e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x562658213e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 -- 172.21.15.83:0/2134417165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562657f3d8c0 con 0x562658023090 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f27fc640 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d880 tx=0x7f85e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2860f4c14a7af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f0ff9640 1 -- 172.21.15.83:0/2134417165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4024040 con 0x562658023090 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f0ff9640 1 -- 172.21.15.83:0/2134417165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e4030070 con 0x562658023090 2024-05-17T22:04:40.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f85f0ff9640 1 -- 172.21.15.83:0/2134417165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e401f6c0 con 0x562658023090 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- 172.21.15.83:0/2134417165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 msgr2=0x562658213900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d880 tx=0x7f85e4012000 comp rx=0 tx=0).stop 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- 172.21.15.83:0/2134417165 shutdown_connections 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658035f80 0x562658213e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658213900 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 --2- 172.21.15.83:0/2134417165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658022880 0x5626582133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- 172.21.15.83:0/2134417165 >> 172.21.15.83:0/2134417165 conn(0x562658071010 msgr2=0x5626581579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- 172.21.15.83:0/2134417165 shutdown_connections 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- 172.21.15.83:0/2134417165 wait complete. 2024-05-17T22:04:40.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 -- start start 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.421+0000 7f8608ba8000 1 Processor -- start 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658022880 0x562658217610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x562658218090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1faf40 con 0x562658023090 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562657fa6330 con 0x562658022880 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562657f64f10 con 0x562658035f80 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 10 client.0 _ll_register_callbacks cb 0x7f85ec0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36394/0 (socket says 172.21.15.83:36394) 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 -- 172.21.15.83:0/696546125 learned_addr learned my addr 172.21.15.83:0/696546125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85faffd640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658022880 0x562658217610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 -- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 msgr2=0x562658218090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x562658218090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 -- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658022880 msgr2=0x562658217610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658022880 0x562658217610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 -- 172.21.15.83:0/696546125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562657f58fa0 con 0x562658023090 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85f27fc640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f85e4017040 tx=0x7f85e4006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38085a8ef8c2171c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4039060 con 0x562658023090 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e4030070 con 0x562658023090 2024-05-17T22:04:40.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f85e402d7d0 con 0x562658023090 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f85e4024070 con 0x562658023090 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 10 client.7212 fetch_fsmap learned FSMap version 72 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x562657c58220 con 0x562658023090 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e4030070 con 0x562658023090 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f85e4013070 con 0x562658023090 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 10 client.7212 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 10 client.7212 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 20 client.7212 populate_metadata read hostname 'smithi083' 2024-05-17T22:04:40.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c32990 0x5626581536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 4 client.7212 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562657fb0070 con 0x562657c32990 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85faffd640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c32990 0x5626581536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 10 client.7212 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85faffd640 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c32990 0x5626581536e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b175cecc14e2ee30 server_cookie=24836f054fce4f32 in_seq=0 out_seq=0 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 1 -- 172.21.15.83:0/696546125 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x562657fb0070 con 0x562657c32990 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 10 client.7212 handle_command_reply: tid=0 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 client.7212 shutdown 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 msgr2=0x562658217b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f85e4017040 tx=0x7f85e4006e20 comp rx=0 tx=0).stop 2024-05-17T22:04:40.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 shutdown_connections 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x562657c32990 0x5626581536e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562658035f80 0x562658218090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562658023090 0x562658217b50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f85da7fc640 0 client.7212 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 --2- 172.21.15.83:0/696546125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562658022880 0x562658217610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 >> 172.21.15.83:0/696546125 conn(0x562658071010 msgr2=0x562658158380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 shutdown_connections 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 1 -- 172.21.15.83:0/696546125 wait complete. 2024-05-17T22:04:40.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f8608ba8000 20 client.7212 trim_cache size 0 max 16384 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "27db2365-1d29-4e50-9133-c3e39b19f1f5": { 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/dir", 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "27db2365-1d29-4e50-9133-c3e39b19f1f5", 2024-05-17T22:04:40.422 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T22:04:40.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:04:40.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:04:40.423 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:04:40.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 msgr2=0x7f85dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 0x7f85dc071910 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0b0950 tx=0x7f85e4008040 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 msgr2=0x7f85fc14e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0ae040 tx=0x7f85ec01ad60 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 shutdown_connections 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85fc1606f0 0x7f85fc0afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85dc06f400 0x7f85dc071910 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85fc15be70 0x7f85fc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 --2- 172.21.15.83:0/652648140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc06eec0 0x7f85fc14e170 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 >> 172.21.15.83:0/652648140 conn(0x7f85fc06cd90 msgr2=0x7f85fc0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 shutdown_connections 2024-05-17T22:04:40.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:04:40.425+0000 7f860192e640 1 -- 172.21.15.83:0/652648140 wait complete. 2024-05-17T22:04:40.434 INFO:tasks.cephfs.filesystem:scrub status for tag:27db2365-1d29-4e50-9133-c3e39b19f1f5 - {'path': '/dir', 'tag': '27db2365-1d29-4e50-9133-c3e39b19f1f5', 'options': 'recursive,repair'} 2024-05-17T22:05:01.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:05:01.082 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:05:10.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:10.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-17T22:05:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 Processor -- start 2024-05-17T22:05:10.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 -- start start 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f926809ba00 0x7f926809be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9268071400 0x7f9268071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f926809c350 con 0x7f92680b88b0 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f92680d1a40 con 0x7f9268071400 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926ed08640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f926809d040 con 0x7f926809ba00 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9268071400 0x7f9268071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9268071400 0x7f9268071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60402/0 (socket says 172.21.15.83:60402) 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 -- 172.21.15.83:0/681146750 learned_addr learned my addr 172.21.15.83:0/681146750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f925ffff640 1 --2- 172.21.15.83:0/681146750 >> v2:172.21.15.155:3300/0 conn(0x7f926809ba00 0x7f926809be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 -- 172.21.15.83:0/681146750 >> v2:172.21.15.155:3300/0 conn(0x7f926809ba00 msgr2=0x7f926809be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 --2- 172.21.15.83:0/681146750 >> v2:172.21.15.155:3300/0 conn(0x7f926809ba00 0x7f926809be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 -- 172.21.15.83:0/681146750 >> v1:172.21.15.155:6789/0 conn(0x7f92680b88b0 legacy=0x7f92680d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.612+0000 7f926d2a5640 1 -- 172.21.15.83:0/681146750 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926809c710 con 0x7f9268071400 2024-05-17T22:05:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926d2a5640 1 --2- 172.21.15.83:0/681146750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9268071400 0x7f9268071810 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f926806d560 tx=0x7f926001c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071bd6994d06ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f925f7fe640 1 -- 172.21.15.83:0/681146750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600ac020 con 0x7f9268071400 2024-05-17T22:05:10.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f925f7fe640 1 -- 172.21.15.83:0/681146750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92600a6490 con 0x7f9268071400 2024-05-17T22:05:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f925f7fe640 1 -- 172.21.15.83:0/681146750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600a6860 con 0x7f9268071400 2024-05-17T22:05:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 -- 172.21.15.83:0/681146750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9268071400 msgr2=0x7f9268071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 --2- 172.21.15.83:0/681146750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9268071400 0x7f9268071810 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f926806d560 tx=0x7f926001c6f0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 -- 172.21.15.83:0/681146750 shutdown_connections 2024-05-17T22:05:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 --2- 172.21.15.83:0/681146750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9268071400 0x7f9268071810 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 --2- 172.21.15.83:0/681146750 >> v2:172.21.15.155:3300/0 conn(0x7f926809ba00 0x7f926809be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 -- 172.21.15.83:0/681146750 >> 172.21.15.83:0/681146750 conn(0x7f92680d3620 msgr2=0x7f92680d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 -- 172.21.15.83:0/681146750 shutdown_connections 2024-05-17T22:05:10.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.616+0000 7f926ed08640 1 -- 172.21.15.83:0/681146750 wait complete. 2024-05-17T22:05:10.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 Processor -- start 2024-05-17T22:05:10.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- start start 2024-05-17T22:05:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 0x7f92680ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 0x7f926815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f926816a870 con 0x7f92680b88b0 2024-05-17T22:05:10.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f926816a6f0 con 0x7f926809ba00 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f926816a9f0 con 0x7f9268071400 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60414/0 (socket says 172.21.15.83:60414) 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 -- 172.21.15.83:0/4117531985 learned_addr learned my addr 172.21.15.83:0/4117531985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926caa4640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 0x7f92680ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f925ffff640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 0x7f926815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 -- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 msgr2=0x7f92680ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 0x7f92680ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 -- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 msgr2=0x7f926815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 0x7f926815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 -- 172.21.15.83:0/4117531985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926815dbd0 con 0x7f92680b88b0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926caa4640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 0x7f92680ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f925ffff640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 0x7f926815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926d2a5640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f92600ab040 tx=0x7f9260019490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f925d7fa640 1 -- 172.21.15.83:0/4117531985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600b7070 con 0x7f92680b88b0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f925d7fa640 1 -- 172.21.15.83:0/4117531985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92600a7670 con 0x7f92680b88b0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92680b0d90 con 0x7f92680b88b0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f925d7fa640 1 -- 172.21.15.83:0/4117531985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260009510 con 0x7f92680b88b0 2024-05-17T22:05:10.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.620+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92680b1350 con 0x7f92680b88b0 2024-05-17T22:05:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f925d7fa640 1 -- 172.21.15.83:0/4117531985 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f92600af020 con 0x7f92680b88b0 2024-05-17T22:05:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f925d7fa640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 0x7f9254071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f925d7fa640 1 -- 172.21.15.83:0/4117531985 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f92601294e0 con 0x7f92680b88b0 2024-05-17T22:05:10.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926caa4640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 0x7f9254071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926caa4640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 0x7f9254071940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f926400a850 tx=0x7f9264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 Processor -- start 2024-05-17T22:05:10.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 -- start start 2024-05-17T22:05:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x557331098210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331098750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557330f32a90 con 0x557330f1b090 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557330ea6120 con 0x557330f1a880 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557330e95cc0 con 0x557330f2df80 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60424/0 (socket says 172.21.15.83:60424) 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 -- 172.21.15.83:0/2290075488 learned_addr learned my addr 172.21.15.83:0/2290075488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f925ffff640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331098750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 -- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 msgr2=0x557331098210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926caa4640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x557331098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x557331098210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 -- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 msgr2=0x557331098750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331098750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f926d2a5640 1 -- 172.21.15.83:0/2290075488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557330e52110 con 0x557330f1b090 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.624+0000 7f925ffff640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331098750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f926d2a5640 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f92600f5c80 tx=0x7f92600093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86725a56614a744f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f924e7fc640 1 -- 172.21.15.83:0/2290075488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600fe410 con 0x557330f1b090 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f924e7fc640 1 -- 172.21.15.83:0/2290075488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92600f63f0 con 0x557330f1b090 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f924e7fc640 1 -- 172.21.15.83:0/2290075488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260129240 con 0x557330f1b090 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- 172.21.15.83:0/2290075488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 msgr2=0x557331098c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f92600f5c80 tx=0x7f92600093a0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- 172.21.15.83:0/2290075488 shutdown_connections 2024-05-17T22:05:10.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1b090 0x557331098c90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331098750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- 172.21.15.83:0/2290075488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x557331098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- 172.21.15.83:0/2290075488 >> 172.21.15.83:0/2290075488 conn(0x557330f69010 msgr2=0x5573310495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- 172.21.15.83:0/2290075488 shutdown_connections 2024-05-17T22:05:10.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- 172.21.15.83:0/2290075488 wait complete. 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- start start 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 Processor -- start 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331085c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557330e51ec0 con 0x557330f2df80 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557330e99d00 con 0x557330f1a880 2024-05-17T22:05:10.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557330b5f680 con 0x557330f1b090 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f9275f82000 10 client.0 _ll_register_callbacks cb 0x557330f4d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60438/0 (socket says 172.21.15.83:60438) 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.628+0000 7f926d2a5640 1 -- 172.21.15.83:0/303978056 learned_addr learned my addr 172.21.15.83:0/303978056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926caa4640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331085c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f925ffff640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 -- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 msgr2=0x5573310861d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310861d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 -- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 msgr2=0x557331085c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331085c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 -- 172.21.15.83:0/303978056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557330e54890 con 0x557330f2df80 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926d2a5640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f92600a6610 tx=0x7f9260009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd003d34419172b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600fe410 con 0x557330f2df80 2024-05-17T22:05:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f92600f4f50 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92600f63f0 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9260009af0 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 10 client.7224 fetch_fsmap learned FSMap version 72 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557330e56650 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92600f63f0 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f9260127ce0 con 0x557330f2df80 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 10 client.7224 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 10 client.7224 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 20 client.7224 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b3c930 0x5573310a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 4 client.7224 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557330e31c40 con 0x557330b3c930 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926caa4640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b3c930 0x5573310a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 10 client.7224 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f926caa4640 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b3c930 0x5573310a95c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdf3a84917f93d server_cookie=bae21c8c39faf300 in_seq=0 out_seq=0 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 1 -- 172.21.15.83:0/303978056 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557330e31c40 con 0x557330b3c930 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 10 client.7224 handle_command_reply: tid=0 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 client.7224 shutdown 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 msgr2=0x557331086710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f92600a6610 tx=0x7f9260009810 comp rx=0 tx=0).stop 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 shutdown_connections 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b3c930 0x5573310a95c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x557331086710 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f92477fe640 0 client.7224 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 --2- 172.21.15.83:0/303978056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x557331085c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 >> 172.21.15.83:0/303978056 conn(0x557330f69010 msgr2=0x557331046bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 shutdown_connections 2024-05-17T22:05:10.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 1 -- 172.21.15.83:0/303978056 wait complete. 2024-05-17T22:05:10.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.632+0000 7f9275f82000 20 client.7224 trim_cache size 0 max 16384 2024-05-17T22:05:10.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 Processor -- start 2024-05-17T22:05:10.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 -- start start 2024-05-17T22:05:10.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1b090 0x557331084db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x5573310852f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9268201680 con 0x557330f1a880 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f926815bbf0 con 0x557330f1b090 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f926806ed60 con 0x557330f2df80 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60444/0 (socket says 172.21.15.83:60444) 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 -- 172.21.15.83:0/3028956801 learned_addr learned my addr 172.21.15.83:0/3028956801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 -- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 msgr2=0x5573310852f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x5573310852f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 -- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1b090 msgr2=0x557331084db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f925ffff640 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1b090 0x557331084db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1b090 0x557331084db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 -- 172.21.15.83:0/3028956801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557330e358c0 con 0x557330f1a880 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f926caa4640 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f926400c9c0 tx=0x7f9264012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af783c383f94d15d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f924dffb640 1 -- 172.21.15.83:0/3028956801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926400ce80 con 0x557330f1a880 2024-05-17T22:05:10.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f924dffb640 1 -- 172.21.15.83:0/3028956801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92640228f0 con 0x557330f1a880 2024-05-17T22:05:10.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.652+0000 7f924dffb640 1 -- 172.21.15.83:0/3028956801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264022cc0 con 0x557330f1a880 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/3028956801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 msgr2=0x5573310aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f926400c9c0 tx=0x7f9264012000 comp rx=0 tx=0).stop 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/3028956801 shutdown_connections 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f2df80 0x5573310852f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1b090 0x557331084db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- 172.21.15.83:0/3028956801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f1a880 0x5573310aca10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/3028956801 >> 172.21.15.83:0/3028956801 conn(0x557330f69010 msgr2=0x5573310495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/3028956801 shutdown_connections 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/3028956801 wait complete. 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- start start 2024-05-17T22:05:10.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 Processor -- start 2024-05-17T22:05:10.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x5573310ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f926806f3b0 con 0x557330f2df80 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557330e9e330 con 0x557330f1a880 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557330e95cc0 con 0x557330f1b090 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 10 client.0 _ll_register_callbacks cb 0x7f92600f5640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60458/0 (socket says 172.21.15.83:60458) 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 -- 172.21.15.83:0/1819850030 learned_addr learned my addr 172.21.15.83:0/1819850030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f925ffff640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 -- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 msgr2=0x5573310acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 -- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 msgr2=0x5573310ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926caa4640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x5573310ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x5573310ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 -- 172.21.15.83:0/1819850030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557330e50fa0 con 0x557330f2df80 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f926d2a5640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f92600f2f60 tx=0x7f92600a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d546e720eed98df server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260129250 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f9260109b90 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92640228f0 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9260114050 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 10 client.7230 fetch_fsmap learned FSMap version 72 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x557330b50220 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92640228f0 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7f9260117020 con 0x557330f2df80 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 10 client.7230 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.656+0000 7f9275f82000 10 client.7230 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 20 client.7230 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b2a990 0x5573310c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 4 client.7230 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557330ea8070 con 0x557330b2a990 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926caa4640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b2a990 0x5573310c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f92477fe640 10 client.7230 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926caa4640 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b2a990 0x5573310c7e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98dd56de73716029 server_cookie=fd5c8d83992f4a2e in_seq=0 out_seq=0 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f92477fe640 1 -- 172.21.15.83:0/1819850030 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557330ea8070 con 0x557330b2a990 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f92477fe640 10 client.7230 handle_command_reply: tid=0 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 client.7230 shutdown 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 msgr2=0x5573310c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f92600f2f60 tx=0x7f92600a9040 comp rx=0 tx=0).stop 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 shutdown_connections 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x557330b2a990 0x5573310c7e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557330f2df80 0x5573310c7490 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557330f1b090 0x5573310acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f92477fe640 0 client.7230 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 --2- 172.21.15.83:0/1819850030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557330f1a880 0x5573310ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 >> 172.21.15.83:0/1819850030 conn(0x557330f69010 msgr2=0x557331083da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 shutdown_connections 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 1 -- 172.21.15.83:0/1819850030 wait complete. 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f9275f82000 20 client.7230 trim_cache size 0 max 16384 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:05:10.655 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 msgr2=0x7f9254071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 0x7f9254071940 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f926400a850 tx=0x7f9264008040 comp rx=0 tx=0).stop 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 msgr2=0x7f9268152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f92600ab040 tx=0x7f9260019490 comp rx=0 tx=0).stop 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 shutdown_connections 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92680b88b0 0x7f9268152980 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f925406f430 0x7f9254071940 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f926809ba00 0x7f926815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 --2- 172.21.15.83:0/4117531985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9268071400 0x7f92680ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 >> 172.21.15.83:0/4117531985 conn(0x7f92680d3620 msgr2=0x7f92680d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 shutdown_connections 2024-05-17T22:05:10.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.660+0000 7f926ed08640 1 -- 172.21.15.83:0/4117531985 wait complete. 2024-05-17T22:05:10.670 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:05:10.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:10.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 Processor -- start 2024-05-17T22:05:10.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 -- start start 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e200d1040 0x7f0e20144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 0x7f0e2006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2015ea40 con 0x7f0e2015e080 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2015ed20 con 0x7f0e200d1040 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e27b91640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2015f000 con 0x7f0e20073540 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e200d1040 0x7f0e20144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2512c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0e2015e080 0x7f0e20072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33604/0 (socket says 172.21.15.83:33604) 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e200d1040 0x7f0e20144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60462/0 (socket says 172.21.15.83:60462) 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2512c640 1 -- 172.21.15.83:0/1794350304 learned_addr learned my addr 172.21.15.83:0/1794350304 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2612e640 1 --2- 172.21.15.83:0/1794350304 >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 0x7f0e2006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 -- 172.21.15.83:0/1794350304 >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 msgr2=0x7f0e2006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 --2- 172.21.15.83:0/1794350304 >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 0x7f0e2006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 -- 172.21.15.83:0/1794350304 >> v1:172.21.15.83:6789/0 conn(0x7f0e2015e080 legacy=0x7f0e20072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2592d640 1 -- 172.21.15.83:0/1794350304 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2006ef30 con 0x7f0e200d1040 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2492b640 1 -- 172.21.15.83:0/1794350304 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375313547 0 0) 0x7f0e2015ea40 con 0x7f0e2015e080 2024-05-17T22:05:10.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.820+0000 7f0e2612e640 1 --2- 172.21.15.83:0/1794350304 >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 0x7f0e2006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:10.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e2592d640 1 --2- 172.21.15.83:0/1794350304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e200d1040 0x7f0e20144080 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0e20072c30 tx=0x7f0e1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84897eebabb38611 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e2492b640 1 -- 172.21.15.83:0/1794350304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100b2070 con 0x7f0e200d1040 2024-05-17T22:05:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e2492b640 1 -- 172.21.15.83:0/1794350304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e100a5900 con 0x7f0e200d1040 2024-05-17T22:05:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e2492b640 1 -- 172.21.15.83:0/1794350304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100a5cd0 con 0x7f0e200d1040 2024-05-17T22:05:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 -- 172.21.15.83:0/1794350304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e200d1040 msgr2=0x7f0e20144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 --2- 172.21.15.83:0/1794350304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e200d1040 0x7f0e20144080 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0e20072c30 tx=0x7f0e1001c930 comp rx=0 tx=0).stop 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 -- 172.21.15.83:0/1794350304 shutdown_connections 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 --2- 172.21.15.83:0/1794350304 >> v2:172.21.15.155:3300/0 conn(0x7f0e20073540 0x7f0e2006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 --2- 172.21.15.83:0/1794350304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e200d1040 0x7f0e20144080 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 -- 172.21.15.83:0/1794350304 >> 172.21.15.83:0/1794350304 conn(0x7f0e2006cd90 msgr2=0x7f0e200b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 -- 172.21.15.83:0/1794350304 shutdown_connections 2024-05-17T22:05:10.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.824+0000 7f0e27b91640 1 -- 172.21.15.83:0/1794350304 wait complete. 2024-05-17T22:05:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 Processor -- start 2024-05-17T22:05:10.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- start start 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e200d1040 0x7f0e2013f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e2015e080 0x7f0e20143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20070810 con 0x7f0e2015c210 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20070690 con 0x7f0e200d1040 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0e20070990 con 0x7f0e2015e080 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60472/0 (socket says 172.21.15.83:60472) 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 -- 172.21.15.83:0/586300142 learned_addr learned my addr 172.21.15.83:0/586300142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:10.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2592d640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e200d1040 0x7f0e2013f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2512c640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e2015e080 0x7f0e20143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 -- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e2015e080 msgr2=0x7f0e20143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e2015e080 0x7f0e20143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 -- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e200d1040 msgr2=0x7f0e2013f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e200d1040 0x7f0e2013f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 -- 172.21.15.83:0/586300142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2013e780 con 0x7f0e2015c210 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e2612e640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c5f0 tx=0x7f0e1800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18015070 con 0x7f0e2015c210 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18011820 con 0x7f0e2015c210 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e20143cc0 con 0x7f0e2015c210 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18011c10 con 0x7f0e2015c210 2024-05-17T22:05:10.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2013e9f0 con 0x7f0e2015c210 2024-05-17T22:05:10.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.828+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2006f730 con 0x7f0e2015c210 2024-05-17T22:05:10.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.832+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e18018070 con 0x7f0e2015c210 2024-05-17T22:05:10.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.832+0000 7f0e1e7fc640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 0x7f0e0c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:10.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.832+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f0e1808f4a0 con 0x7f0e2015c210 2024-05-17T22:05:10.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.832+0000 7f0e2592d640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 0x7f0e0c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.832+0000 7f0e2592d640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 0x7f0e0c071940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0e100af000 tx=0x7f0e100a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:10.836+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0e1805bd30 con 0x7f0e2015c210 2024-05-17T22:05:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.020+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e200aa480 con 0x7f0e2015c210 2024-05-17T22:05:11.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e1e7fc640 1 -- 172.21.15.83:0/586300142 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7f0e18060030 con 0x7f0e2015c210 2024-05-17T22:05:11.014 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:11.014 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:05:11.014 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 msgr2=0x7f0e0c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 0x7f0e0c071940 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0e100af000 tx=0x7f0e100a7070 comp rx=0 tx=0).stop 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 msgr2=0x7f0e201413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c5f0 tx=0x7f0e1800cab0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 shutdown_connections 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e2015c210 0x7f0e201413e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e2015e080 0x7f0e20143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0e0c06f430 0x7f0e0c071940 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 --2- 172.21.15.83:0/586300142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e200d1040 0x7f0e2013f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 >> 172.21.15.83:0/586300142 conn(0x7f0e2006cd90 msgr2=0x7f0e200b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 shutdown_connections 2024-05-17T22:05:11.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.024+0000 7f0e27b91640 1 -- 172.21.15.83:0/586300142 wait complete. 2024-05-17T22:05:11.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-17T22:05:11.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.180+0000 7fe68636d640 1 Processor -- start 2024-05-17T22:05:11.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.180+0000 7fe68636d640 1 -- start start 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe680073590 0x7fe6800d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe680143f20 0x7fe6801447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6800d10d0 con 0x7fe6800b3d00 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe6800b4110 con 0x7fe680073590 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe680070770 con 0x7fe680143f20 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe680073590 0x7fe6800d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe680073590 0x7fe6800d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60476/0 (socket says 172.21.15.83:60476) 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 -- 172.21.15.83:0/2354594292 learned_addr learned my addr 172.21.15.83:0/2354594292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68490a640 1 --2- 172.21.15.83:0/2354594292 >> v2:172.21.15.155:3300/0 conn(0x7fe680143f20 0x7fe6801447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 -- 172.21.15.83:0/2354594292 >> v2:172.21.15.155:3300/0 conn(0x7fe680143f20 msgr2=0x7fe6801447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/2354594292 >> v2:172.21.15.155:3300/0 conn(0x7fe680143f20 0x7fe6801447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 -- 172.21.15.83:0/2354594292 >> v1:172.21.15.155:6789/0 conn(0x7fe6800b3d00 legacy=0x7fe680072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 -- 172.21.15.83:0/2354594292 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe680070120 con 0x7fe680073590 2024-05-17T22:05:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/2354594292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680073590 0x7fe6800d0a10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe6800706e0 tx=0x7fe67401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc5a0aea85a5dba server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67effd640 1 -- 172.21.15.83:0/2354594292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740b0070 con 0x7fe680073590 2024-05-17T22:05:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67effd640 1 -- 172.21.15.83:0/2354594292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6740ab600 con 0x7fe680073590 2024-05-17T22:05:11.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe67effd640 1 -- 172.21.15.83:0/2354594292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740ab9d0 con 0x7fe680073590 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- 172.21.15.83:0/2354594292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680073590 msgr2=0x7fe6800d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 --2- 172.21.15.83:0/2354594292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680073590 0x7fe6800d0a10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe6800706e0 tx=0x7fe67401c520 comp rx=0 tx=0).stop 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- 172.21.15.83:0/2354594292 shutdown_connections 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 --2- 172.21.15.83:0/2354594292 >> v2:172.21.15.155:3300/0 conn(0x7fe680143f20 0x7fe6801447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 --2- 172.21.15.83:0/2354594292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680073590 0x7fe6800d0a10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- 172.21.15.83:0/2354594292 >> 172.21.15.83:0/2354594292 conn(0x7fe68006cd90 msgr2=0x7fe6800b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- 172.21.15.83:0/2354594292 shutdown_connections 2024-05-17T22:05:11.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.184+0000 7fe68636d640 1 -- 172.21.15.83:0/2354594292 wait complete. 2024-05-17T22:05:11.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 Processor -- start 2024-05-17T22:05:11.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- start start 2024-05-17T22:05:11.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 0x7fe68015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 0x7fe680142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe680070c00 con 0x7fe680143f20 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe680145540 con 0x7fe6800b3d00 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe680139b80 con 0x7fe680073590 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60492/0 (socket says 172.21.15.83:60492) 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 -- 172.21.15.83:0/2339023998 learned_addr learned my addr 172.21.15.83:0/2339023998 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 0x7fe680142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67ffff640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 0x7fe68015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 -- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 msgr2=0x7fe68015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 0x7fe68015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 -- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 msgr2=0x7fe680142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 0x7fe680142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 -- 172.21.15.83:0/2339023998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68015dab0 con 0x7fe680143f20 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 0x7fe680142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67ffff640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 0x7fe68015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68490a640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe678007e90 tx=0x7fe67800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67cff9640 1 -- 172.21.15.83:0/2339023998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678014420 con 0x7fe680143f20 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67cff9640 1 -- 172.21.15.83:0/2339023998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678019070 con 0x7fe680143f20 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe680139dd0 con 0x7fe680143f20 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe67cff9640 1 -- 172.21.15.83:0/2339023998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678014800 con 0x7fe680143f20 2024-05-17T22:05:11.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.188+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6800af540 con 0x7fe680143f20 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe67cff9640 1 -- 172.21.15.83:0/2339023998 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6780149a0 con 0x7fe680143f20 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe67cff9640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 0x7fe66c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe67cff9640 1 -- 172.21.15.83:0/2339023998 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fe67808fbd0 con 0x7fe680143f20 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe67ffff640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 0x7fe66c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe67ffff640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 0x7fe66c071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe67000a850 tx=0x7fe670008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe68d5e7000 1 Processor -- start 2024-05-17T22:05:11.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe68d5e7000 1 -- start start 2024-05-17T22:05:11.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.192+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b344020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b344560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56400b1e2fa0 con 0x56400b1de030 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56400b1572e0 con 0x56400b1f4460 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56400b146e80 con 0x56400b1fee00 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60498/0 (socket says 172.21.15.83:60498) 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 -- 172.21.15.83:0/1181000765 learned_addr learned my addr 172.21.15.83:0/1181000765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b344560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 -- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 msgr2=0x56400b344020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe67ffff640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b344020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b344020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 -- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 msgr2=0x56400b344560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b344560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 -- 172.21.15.83:0/1181000765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400b1f1b80 con 0x56400b1de030 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b344560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe678058950 tx=0x7fe67805bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95015388c05ddb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe665ffb640 1 -- 172.21.15.83:0/1181000765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678060710 con 0x56400b1de030 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe665ffb640 1 -- 172.21.15.83:0/1181000765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678065130 con 0x56400b1de030 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe665ffb640 1 -- 172.21.15.83:0/1181000765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67805b800 con 0x56400b1de030 2024-05-17T22:05:11.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1181000765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 msgr2=0x56400b344aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe678058950 tx=0x7fe67805bb90 comp rx=0 tx=0).stop 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1181000765 shutdown_connections 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b344aa0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b344560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1181000765 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b344020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1181000765 >> 172.21.15.83:0/1181000765 conn(0x56400ae08760 msgr2=0x56400b301270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1181000765 shutdown_connections 2024-05-17T22:05:11.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1181000765 wait complete. 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- start start 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 Processor -- start 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 0x56400b34b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1f4460 0x56400b34b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56400b102eb0 con 0x56400b1fee00 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56400b14aec0 con 0x56400b1de030 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56400ae10820 con 0x56400b1f4460 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68d5e7000 10 client.0 _ll_register_callbacks cb 0x56400b2e6d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.196+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60514/0 (socket says 172.21.15.83:60514) 2024-05-17T22:05:11.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 -- 172.21.15.83:0/1706820873 learned_addr learned my addr 172.21.15.83:0/1706820873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe67ffff640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 0x56400b34b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1f4460 0x56400b34b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 -- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1f4460 msgr2=0x56400b34b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1f4460 0x56400b34b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 -- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 msgr2=0x56400b34b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 0x56400b34b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 -- 172.21.15.83:0/1706820873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400b18fef0 con 0x56400b1fee00 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe67ffff640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 0x56400b34b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68490a640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe678058950 tx=0x7fe67800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b242f187ccbd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678079020 con 0x56400b1fee00 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fe678057cd0 con 0x56400b1fee00 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe678065130 con 0x56400b1fee00 2024-05-17T22:05:11.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fe6780589b0 con 0x56400b1fee00 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 10 client.7251 fetch_fsmap learned FSMap version 72 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56400b184910 con 0x56400b1fee00 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe678065130 con 0x56400b1fee00 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7fe678075ac0 con 0x56400b1fee00 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 10 client.7251 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 10 client.7251 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 20 client.7251 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400adedad0 0x56400b355680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 4 client.7251 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56400b0e2cf0 con 0x56400adedad0 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe67ffff640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400adedad0 0x56400b355680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 10 client.7251 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe67ffff640 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400adedad0 0x56400b355680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ca2f4b2aa7d88 server_cookie=2f47d628ed5b96a6 in_seq=0 out_seq=0 2024-05-17T22:05:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 1 -- 172.21.15.83:0/1706820873 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56400b0e2cf0 con 0x56400adedad0 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 10 client.7251 handle_command_reply: tid=0 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 client.7251 shutdown 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 msgr2=0x56400b34bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe678058950 tx=0x7fe67800efd0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 shutdown_connections 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400adedad0 0x56400b355680 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1fee00 0x56400b34bd20 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1f4460 0x56400b34b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe65effd640 0 client.7251 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/1706820873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1de030 0x56400b34b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 >> 172.21.15.83:0/1706820873 conn(0x56400ae08760 msgr2=0x56400b2f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 shutdown_connections 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 1 -- 172.21.15.83:0/1706820873 wait complete. 2024-05-17T22:05:11.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.200+0000 7fe68d5e7000 20 client.7251 trim_cache size 0 max 16384 2024-05-17T22:05:11.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.212+0000 7fe68d5e7000 1 Processor -- start 2024-05-17T22:05:11.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.212+0000 7fe68d5e7000 1 -- start start 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b3b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b3b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800b0a80 con 0x56400b1de030 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6802041c0 con 0x56400b1f4460 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56400b1cb8f0 con 0x56400b1fee00 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60524/0 (socket says 172.21.15.83:60524) 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 -- 172.21.15.83:0/588686199 learned_addr learned my addr 172.21.15.83:0/588686199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 -- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 msgr2=0x56400b3b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b3b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 -- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 msgr2=0x56400b3b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b3b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 -- 172.21.15.83:0/588686199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56400b104670 con 0x56400b1de030 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe67ffff640 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe670006fd0 tx=0x7fe670012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ac668ee8522ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe6657fa640 1 -- 172.21.15.83:0/588686199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670013cb0 con 0x56400b1de030 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe6657fa640 1 -- 172.21.15.83:0/588686199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670029030 con 0x56400b1de030 2024-05-17T22:05:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe6657fa640 1 -- 172.21.15.83:0/588686199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6700243d0 con 0x56400b1de030 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- 172.21.15.83:0/588686199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 msgr2=0x56400b3b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe670006fd0 tx=0x7fe670012000 comp rx=0 tx=0).stop 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- 172.21.15.83:0/588686199 shutdown_connections 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1fee00 0x56400b3b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1f4460 0x56400b3b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/588686199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1de030 0x56400b3b27c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- 172.21.15.83:0/588686199 >> 172.21.15.83:0/588686199 conn(0x56400ae08760 msgr2=0x56400b2f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- 172.21.15.83:0/588686199 shutdown_connections 2024-05-17T22:05:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- 172.21.15.83:0/588686199 wait complete. 2024-05-17T22:05:11.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 -- start start 2024-05-17T22:05:11.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.216+0000 7fe68d5e7000 1 Processor -- start 2024-05-17T22:05:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 0x56400b3b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1fee00 0x56400b3b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56400b14aec0 con 0x56400b1f4460 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56400b102eb0 con 0x56400b1fee00 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56400b0e2850 con 0x56400b1de030 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 10 client.0 _ll_register_callbacks cb 0x7fe678074070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60540/0 (socket says 172.21.15.83:60540) 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 -- 172.21.15.83:0/3245086653 learned_addr learned my addr 172.21.15.83:0/3245086653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67ffff640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 0x56400b3b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 -- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 msgr2=0x56400b3b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68490a640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1fee00 0x56400b3b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 0x56400b3b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 -- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1fee00 msgr2=0x56400b3b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1fee00 0x56400b3b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 -- 172.21.15.83:0/3245086653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400b103100 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67ffff640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 0x56400b3b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67f7fe640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe6740ae040 tx=0x7fe67401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb657a9bb91c4a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6740ba020 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fe6740abbb0 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe670029030 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fe6740c4050 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 10 client.7257 fetch_fsmap learned FSMap version 72 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56400b1cbd80 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670029030 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 72) ==== 3494+0+0 (secure 0 0 0) 0x7fe6740c7020 con 0x56400b1f4460 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 10 client.7257 fetch_fsmap finished waiting for FSMap version 72 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 10 client.7257 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:05:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 20 client.7257 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400addbb30 0x56400b373470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 4 client.7257 mds_command: new command op to 7083 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 --> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56400b159230 con 0x56400addbb30 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67ffff640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400addbb30 0x56400b373470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 10 client.7257 ms_handle_connect on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe67ffff640 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400addbb30 0x56400b373470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ecbb46dc4742d7 server_cookie=9bcd4e5e18756e2d in_seq=0 out_seq=0 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 1 -- 172.21.15.83:0/3245086653 <== mds.0 v2:172.21.15.83:6834/2140054867 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56400b159230 con 0x56400addbb30 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 10 client.7257 handle_command_reply: tid=0 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 client.7257 shutdown 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 msgr2=0x56400b3b7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe6740ae040 tx=0x7fe67401ad60 comp rx=0 tx=0).stop 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 shutdown_connections 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:6834/2140054867,v1:172.21.15.83:6836/2140054867] conn(0x56400addbb30 0x56400b373470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56400b1fee00 0x56400b3b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe65effd640 0 client.7257 ms_handle_reset on v2:172.21.15.83:6834/2140054867 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56400b1f4460 0x56400b3b7120 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 --2- 172.21.15.83:0/3245086653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56400b1de030 0x56400b3b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 >> 172.21.15.83:0/3245086653 conn(0x56400ae08760 msgr2=0x56400b2fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 shutdown_connections 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 1 -- 172.21.15.83:0/3245086653 wait complete. 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68d5e7000 20 client.7257 trim_cache size 0 max 16384 2024-05-17T22:05:11.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-05-17T22:05:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 msgr2=0x7fe66c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 0x7fe66c071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe67000a850 tx=0x7fe670008040 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 msgr2=0x7fe6801413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe678007e90 tx=0x7fe67800e4b0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 shutdown_connections 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe680143f20 0x7fe6801413d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe66c06f400 0x7fe66c071910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6800b3d00 0x7fe680142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 --2- 172.21.15.83:0/2339023998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe680073590 0x7fe68015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 >> 172.21.15.83:0/2339023998 conn(0x7fe68006cd90 msgr2=0x7fe680072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 shutdown_connections 2024-05-17T22:05:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.220+0000 7fe68636d640 1 -- 172.21.15.83:0/2339023998 wait complete. 2024-05-17T22:05:11.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-17T22:05:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 Processor -- start 2024-05-17T22:05:11.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 -- start start 2024-05-17T22:05:11.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17241461c0 0x7f172415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f172406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17241465d0 con 0x7f1724070510 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f172406ef30 con 0x7f17241461c0 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.368+0000 7f172a492640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f172406f210 con 0x7f172415e6b0 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17241461c0 0x7f172415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1728a2f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1724070510 0x7f17241605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33610/0 (socket says 172.21.15.83:33610) 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1728a2f640 1 -- 172.21.15.83:0/801885317 learned_addr learned my addr 172.21.15.83:0/801885317 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 -- 172.21.15.83:0/801885317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 msgr2=0x7f172406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 --2- 172.21.15.83:0/801885317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f172406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 -- 172.21.15.83:0/801885317 >> v1:172.21.15.83:6789/0 conn(0x7f1724070510 legacy=0x7f17241605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 -- 172.21.15.83:0/801885317 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1724070920 con 0x7f17241461c0 2024-05-17T22:05:11.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1722ffd640 1 -- 172.21.15.83:0/801885317 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745448935 0 0) 0x7f17241465d0 con 0x7f1724070510 2024-05-17T22:05:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1723fff640 1 --2- 172.21.15.83:0/801885317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f172415e170 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f17140097c0 tx=0x7f171401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bed8c41a17925d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1722ffd640 1 -- 172.21.15.83:0/801885317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17140a7070 con 0x7f17241461c0 2024-05-17T22:05:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1722ffd640 1 -- 172.21.15.83:0/801885317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17140b0070 con 0x7f17241461c0 2024-05-17T22:05:11.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f1722ffd640 1 -- 172.21.15.83:0/801885317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17140ab890 con 0x7f17241461c0 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 -- 172.21.15.83:0/801885317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 msgr2=0x7f172415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 --2- 172.21.15.83:0/801885317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f172415e170 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f17140097c0 tx=0x7f171401c470 comp rx=0 tx=0).stop 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 -- 172.21.15.83:0/801885317 shutdown_connections 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 --2- 172.21.15.83:0/801885317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f172406ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 --2- 172.21.15.83:0/801885317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f172415e170 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 -- 172.21.15.83:0/801885317 >> 172.21.15.83:0/801885317 conn(0x7f172406cd90 msgr2=0x7f17240b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 -- 172.21.15.83:0/801885317 shutdown_connections 2024-05-17T22:05:11.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.372+0000 7f172a492640 1 -- 172.21.15.83:0/801885317 wait complete. 2024-05-17T22:05:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 Processor -- start 2024-05-17T22:05:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 -- start start 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1724070510 0x7f1724155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f1724159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1724160e30 con 0x7f17241461c0 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1724160cb0 con 0x7f1724070510 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f172a492640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f17240d10a0 con 0x7f172415e6b0 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60548/0 (socket says 172.21.15.83:60548) 2024-05-17T22:05:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 -- 172.21.15.83:0/1006752728 learned_addr learned my addr 172.21.15.83:0/1006752728 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f1723fff640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1724070510 0x7f1724155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f1728a2f640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f1724159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 -- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 msgr2=0x7f1724159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f1724159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 -- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1724070510 msgr2=0x7f1724155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1724070510 0x7f1724155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 -- 172.21.15.83:0/1006752728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172413d530 con 0x7f17241461c0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f17237fe640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f171800b440 tx=0x7f171800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718015020 con 0x7f17241461c0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.376+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17180118b0 con 0x7f17241461c0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17240b5650 con 0x7f17241461c0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718011ca0 con 0x7f17241461c0 2024-05-17T22:05:11.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17240b5c10 con 0x7f17241461c0 2024-05-17T22:05:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17240b53c0 con 0x7f17241461c0 2024-05-17T22:05:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1718018020 con 0x7f17241461c0 2024-05-17T22:05:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1720ff9640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 0x7f1710071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f171808fa20 con 0x7f17241461c0 2024-05-17T22:05:11.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1723fff640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 0x7f1710071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.380+0000 7f1723fff640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 0x7f1710071910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f17140af000 tx=0x7f17140a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.384+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f171801f020 con 0x7f17241461c0 2024-05-17T22:05:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f172415e170 con 0x7f17241461c0 2024-05-17T22:05:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f1720ff9640 1 -- 172.21.15.83:0/1006752728 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f171800d040 con 0x7f17241461c0 2024-05-17T22:05:11.571 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:11.571 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-17T22:05:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 msgr2=0x7f1710071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 0x7f1710071910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f17140af000 tx=0x7f17140a7000 comp rx=0 tx=0).stop 2024-05-17T22:05:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 msgr2=0x7f17240b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f171800b440 tx=0x7f171800b900 comp rx=0 tx=0).stop 2024-05-17T22:05:11.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 shutdown_connections 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f172415e6b0 0x7f1724159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17241461c0 0x7f17240b4ae0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f171006f400 0x7f1710071910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 --2- 172.21.15.83:0/1006752728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1724070510 0x7f1724155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 >> 172.21.15.83:0/1006752728 conn(0x7f172406cd90 msgr2=0x7f172415edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 shutdown_connections 2024-05-17T22:05:11.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.580+0000 7f172a492640 1 -- 172.21.15.83:0/1006752728 wait complete. 2024-05-17T22:05:11.590 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-17T22:05:11.591 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:05:11.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:05:11.667 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:05:11.717 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:05:11.717 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:05:11.717 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:11.717 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:11.723 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:11.723 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:11.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:05:11.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 Processor -- start 2024-05-17T22:05:11.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 -- start start 2024-05-17T22:05:11.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f86f006d1a0 0x7f86f00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f00cdcb0 con 0x7f86f01633b0 2024-05-17T22:05:11.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f00cdf90 con 0x7f86f00d1e00 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f82a5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f86f00ce270 con 0x7f86f006d1a0 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f5840640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f86f01633b0 0x7f86f00d16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33626/0 (socket says 172.21.15.83:33626) 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f5840640 1 -- 172.21.15.83:0/276346245 learned_addr learned my addr 172.21.15.83:0/276346245 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f6041640 1 --2- 172.21.15.83:0/276346245 >> v2:172.21.15.155:3300/0 conn(0x7f86f006d1a0 0x7f86f00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768625815 0 0) 0x7f86f00cdcb0 con 0x7f86f01633b0 2024-05-17T22:05:11.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86dc003530 con 0x7f86f01633b0 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1979418299 0 0) 0x7f86dc003530 con 0x7f86f01633b0 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 >> v2:172.21.15.155:3300/0 conn(0x7f86f006d1a0 msgr2=0x7f86f00b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 --2- 172.21.15.83:0/276346245 >> v2:172.21.15.155:3300/0 conn(0x7f86f006d1a0 0x7f86f00b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 >> v1:172.21.15.155:6789/0 conn(0x7f86f00d1e00 legacy=0x7f86f00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f00b37a0 con 0x7f86f01633b0 2024-05-17T22:05:11.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f86ec001700 con 0x7f86f01633b0 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 >> v1:172.21.15.83:6789/0 conn(0x7f86f01633b0 legacy=0x7f86f00d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.972+0000 7f86f503f640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 0x7f86dc003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc004810 0x7f86dc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86dc005360 0x7f86dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00ce270 con 0x7f86dc003a60 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc00c290 con 0x7f86dc004810 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc00c490 con 0x7f86dc005360 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 0x7f86dc003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6842640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86dc005360 0x7f86dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f5840640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc004810 0x7f86dc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 -- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86dc005360 msgr2=0x7f86dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86dc005360 0x7f86dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 -- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc004810 msgr2=0x7f86dc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc004810 0x7f86dc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 -- 172.21.15.83:0/276346245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc003530 con 0x7f86dc003a60 2024-05-17T22:05:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6842640 1 -- 172.21.15.83:0/276346245 reap_dead start 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f6041640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 0x7f86dc003ed0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009ae0 tx=0x7f86e001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82c97e20ea05e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00096b0 con 0x7f86dc003a60 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e00a6450 con 0x7f86dc003a60 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f503f640 1 -- 172.21.15.83:0/276346245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e00a6840 con 0x7f86dc003a60 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 -- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 msgr2=0x7f86dc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 0x7f86dc003ed0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009ae0 tx=0x7f86e001ca70 comp rx=0 tx=0).stop 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 -- 172.21.15.83:0/276346245 shutdown_connections 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 --2- 172.21.15.83:0/276346245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86dc003a60 0x7f86dc003ed0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 -- 172.21.15.83:0/276346245 >> 172.21.15.83:0/276346245 conn(0x7f86f00b4090 msgr2=0x7f86f00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 -- 172.21.15.83:0/276346245 shutdown_connections 2024-05-17T22:05:11.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.976+0000 7f86f82a5640 1 -- 172.21.15.83:0/276346245 wait complete. 2024-05-17T22:05:11.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 Processor -- start 2024-05-17T22:05:11.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- start start 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc003a60 0x7f86f00c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86f00c6050 0x7f86f01f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00cd500 con 0x7f86f00cb370 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00cd380 con 0x7f86dc003a60 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86f00cd680 con 0x7f86f00c6050 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60578/0 (socket says 172.21.15.83:60578) 2024-05-17T22:05:11.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 -- 172.21.15.83:0/4106285263 learned_addr learned my addr 172.21.15.83:0/4106285263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 -- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86f00c6050 msgr2=0x7f86f01f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f5840640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86f00c6050 0x7f86f01f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86f00c6050 0x7f86f01f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6041640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc003a60 0x7f86f00c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 -- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc003a60 msgr2=0x7f86f00c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc003a60 0x7f86f00c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 -- 172.21.15.83:0/4106285263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f00cbcc0 con 0x7f86f00cb370 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f6842640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f86f00b5150 tx=0x7f86e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8006b60 con 0x7f86f00cb370 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e8006d00 con 0x7f86f00cb370 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f015a610 con 0x7f86f00cb370 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8003bc0 con 0x7f86f00cb370 2024-05-17T22:05:11.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.980+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f00c9770 con 0x7f86f00cb370 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f015a380 con 0x7f86f00cb370 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f86e8005070 con 0x7f86f00cb370 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86e6ffd640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 0x7f86d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86f6041640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 0x7f86d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f86e808eac0 con 0x7f86f00cb370 2024-05-17T22:05:11.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.984+0000 7f86f6041640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 0x7f86d4071910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ad000 tx=0x7f86e00a9040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:11.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:11.988+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f86f00cbcc0 con 0x7f86f00cb370 2024-05-17T22:05:12.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.160+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86f015ab30 con 0x7f86f00cb370 2024-05-17T22:05:12.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.160+0000 7f86e6ffd640 1 -- 172.21.15.83:0/4106285263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v85) ==== 80+0+242 (secure 0 0 0) 0x7f86f015ab30 con 0x7f86f00cb370 2024-05-17T22:05:12.151 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/165849096 2024-05-18T22:04:31.611283+0000 2024-05-17T22:05:12.151 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6834/165849096 2024-05-18T22:04:31.611283+0000 2024-05-17T22:05:12.151 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6836/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:05:12.151 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6834/3213750564 2024-05-18T22:03:12.435992+0000 2024-05-17T22:05:12.152 INFO:teuthology.orchestra.run.smithi083.stderr:listed 4 entries 2024-05-17T22:05:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 msgr2=0x7f86d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 0x7f86d4071910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ad000 tx=0x7f86e00a9040 comp rx=0 tx=0).stop 2024-05-17T22:05:12.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 msgr2=0x7f86f00cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f86f00b5150 tx=0x7f86e800eb90 comp rx=0 tx=0).stop 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 shutdown_connections 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86f00cb370 0x7f86f00cb780 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86f00c6050 0x7f86f01f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f86d406f400 0x7f86d4071910 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 --2- 172.21.15.83:0/4106285263 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86dc003a60 0x7f86f00c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 >> 172.21.15.83:0/4106285263 conn(0x7f86f00b4090 msgr2=0x7f86f00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 shutdown_connections 2024-05-17T22:05:12.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.164+0000 7f86f82a5640 1 -- 172.21.15.83:0/4106285263 wait complete. 2024-05-17T22:05:12.172 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:05:12.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:12.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:12.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[44949]: fuse finished with error 0 and tester_r 0 2024-05-17T22:05:12.184 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:05:12.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:12.185 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:05:12.344 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:05:12.344 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:05:12.344 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:12.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:05:12.351 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:05:12.351 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:12.351 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:12.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:05:12.390 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:05:12.390 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:05:12.390 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:12.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:05:12.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 Processor -- start 2024-05-17T22:05:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 -- start start 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0498144080 0x7f0498072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0498073370 0x7f049806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0498073780 con 0x7f04980d1040 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f049815e790 con 0x7f0498073370 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049fd83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f049815ea40 con 0x7f0498144080 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0498073370 0x7f049806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0498073370 0x7f049806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60588/0 (socket says 172.21.15.83:60588) 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 -- 172.21.15.83:0/3611209835 learned_addr learned my addr 172.21.15.83:0/3611209835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049db1f640 1 --1- 172.21.15.83:0/3611209835 >> v1:172.21.15.83:6789/0 conn(0x7f04980d1040 0x7f049815e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33634/0 (socket says 172.21.15.83:33634) 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049d31e640 1 --2- 172.21.15.83:0/3611209835 >> v2:172.21.15.155:3300/0 conn(0x7f0498144080 0x7f0498072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 -- 172.21.15.83:0/3611209835 >> v2:172.21.15.155:3300/0 conn(0x7f0498144080 msgr2=0x7f0498072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 --2- 172.21.15.83:0/3611209835 >> v2:172.21.15.155:3300/0 conn(0x7f0498144080 0x7f0498072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 -- 172.21.15.83:0/3611209835 >> v1:172.21.15.83:6789/0 conn(0x7f04980d1040 legacy=0x7f049815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049cb1d640 1 -- 172.21.15.83:0/3611209835 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740717675 0 0) 0x7f0490001120 con 0x7f04980d1040 2024-05-17T22:05:12.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.592+0000 7f049e320640 1 -- 172.21.15.83:0/3611209835 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049815ed20 con 0x7f0498073370 2024-05-17T22:05:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049e320640 1 --2- 172.21.15.83:0/3611209835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498073370 0x7f049806f730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f04980b6b10 tx=0x7f049401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e5d57cfeb213e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049cb1d640 1 -- 172.21.15.83:0/3611209835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940a5880 con 0x7f0498073370 2024-05-17T22:05:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049cb1d640 1 -- 172.21.15.83:0/3611209835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04940a5a20 con 0x7f0498073370 2024-05-17T22:05:12.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049cb1d640 1 -- 172.21.15.83:0/3611209835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940a5df0 con 0x7f0498073370 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 -- 172.21.15.83:0/3611209835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498073370 msgr2=0x7f049806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 --2- 172.21.15.83:0/3611209835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498073370 0x7f049806f730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f04980b6b10 tx=0x7f049401c9c0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 -- 172.21.15.83:0/3611209835 shutdown_connections 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 --2- 172.21.15.83:0/3611209835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498073370 0x7f049806f730 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 --2- 172.21.15.83:0/3611209835 >> v2:172.21.15.155:3300/0 conn(0x7f0498144080 0x7f0498072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 -- 172.21.15.83:0/3611209835 >> 172.21.15.83:0/3611209835 conn(0x7f049806cd90 msgr2=0x7f04980b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 -- 172.21.15.83:0/3611209835 shutdown_connections 2024-05-17T22:05:12.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.596+0000 7f049fd83640 1 -- 172.21.15.83:0/3611209835 wait complete. 2024-05-17T22:05:12.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 Processor -- start 2024-05-17T22:05:12.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- start start 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498073370 0x7f049815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04980d1040 0x7f049813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0498070480 con 0x7f0498144080 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0498144490 con 0x7f0498073370 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0498070600 con 0x7f04980d1040 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60596/0 (socket says 172.21.15.83:60596) 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 -- 172.21.15.83:0/1288083303 learned_addr learned my addr 172.21.15.83:0/1288083303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:12.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 -- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04980d1040 msgr2=0x7f049813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04980d1040 0x7f049813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049db1f640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498073370 0x7f049815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 -- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498073370 msgr2=0x7f049815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498073370 0x7f049815b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 -- 172.21.15.83:0/1288083303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049815bdf0 con 0x7f0498144080 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049e320640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f04940af040 tx=0x7f0494019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940bc020 con 0x7f0498144080 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04940b8690 con 0x7f0498144080 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940b8a80 con 0x7f0498144080 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04980b3640 con 0x7f0498144080 2024-05-17T22:05:12.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04980b54f0 con 0x7f0498144080 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.600+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498070990 con 0x7f0498144080 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.604+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04940bf020 con 0x7f0498144080 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.604+0000 7f048e7fc640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 0x7f047c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.604+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f049412a3c0 con 0x7f0498144080 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.604+0000 7f049db1f640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 0x7f047c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.604+0000 7f049db1f640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 0x7f047c071940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0490000f70 tx=0x7f0490009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.608+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f04940c6030 con 0x7f0498144080 2024-05-17T22:05:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.788+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f049815bff0 con 0x7f0498144080 2024-05-17T22:05:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.788+0000 7f048e7fc640 1 -- 172.21.15.83:0/1288083303 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7f04940165e0 con 0x7f0498144080 2024-05-17T22:05:12.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:12.780 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:05:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 msgr2=0x7f047c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 0x7f047c071940 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0490000f70 tx=0x7f0490009040 comp rx=0 tx=0).stop 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 msgr2=0x7f04980b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f04940af040 tx=0x7f0494019760 comp rx=0 tx=0).stop 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 shutdown_connections 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0498144080 0x7f04980b5a10 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04980d1040 0x7f049813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f047c06f430 0x7f047c071940 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 --2- 172.21.15.83:0/1288083303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0498073370 0x7f049815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 >> 172.21.15.83:0/1288083303 conn(0x7f049806cd90 msgr2=0x7f0498144fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 shutdown_connections 2024-05-17T22:05:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.792+0000 7f049fd83640 1 -- 172.21.15.83:0/1288083303 wait complete. 2024-05-17T22:05:12.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:12.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.948+0000 7fc03358b640 1 Processor -- start 2024-05-17T22:05:12.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.948+0000 7fc03358b640 1 -- start start 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc03358b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc02c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc03358b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02c1465d0 con 0x7fc02c1605a0 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc03358b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02c070920 con 0x7fc02c1461c0 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc03358b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc02c06ef30 con 0x7fc02c06ffd0 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031b28640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc02c1605a0 0x7fc02c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33650/0 (socket says 172.21.15.83:33650) 2024-05-17T22:05:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031b28640 1 -- 172.21.15.83:0/2360676251 learned_addr learned my addr 172.21.15.83:0/2360676251 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220174954 0 0) 0x7fc02c1465d0 con 0x7fc02c1605a0 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc030b26640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc02c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc018005740 con 0x7fc02c1605a0 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1383605964 0 0) 0x7fc018005740 con 0x7fc02c1605a0 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 msgr2=0x7fc02c0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc02c0703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 >> v1:172.21.15.155:6789/0 conn(0x7fc02c1461c0 legacy=0x7fc02c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c06f210 con 0x7fc02c1605a0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fc028001a80 con 0x7fc02c1605a0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc030b26640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc02c0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 >> v1:172.21.15.83:6789/0 conn(0x7fc02c1605a0 legacy=0x7fc02c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc0180060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc018006a20 0x7fc018006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc01800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c06ef30 con 0x7fc018005c50 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc018007770 con 0x7fc018006a20 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc01800e120 con 0x7fc02c06ffd0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc0180060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc030b26640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc018006a20 0x7fc018006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031b28640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc01800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 -- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 msgr2=0x7fc01800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c06ffd0 0x7fc01800dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 -- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc018006a20 msgr2=0x7fc018006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc018006a20 0x7fc018006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc031327640 1 -- 172.21.15.83:0/2360676251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc018005740 con 0x7fc018005c50 2024-05-17T22:05:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.952+0000 7fc030b26640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc018006a20 0x7fc018006e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc031327640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc0180060e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0b11f0 tx=0x7fc02401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220d959fcb7e84d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240013e0 con 0x7fc018005c50 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240052d0 con 0x7fc018005c50 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc023fff640 1 -- 172.21.15.83:0/2360676251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024003840 con 0x7fc018005c50 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- 172.21.15.83:0/2360676251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 msgr2=0x7fc0180060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 --2- 172.21.15.83:0/2360676251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc0180060e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0b11f0 tx=0x7fc02401c3a0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc031b28640 1 -- 172.21.15.83:0/2360676251 reap_dead start 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- 172.21.15.83:0/2360676251 shutdown_connections 2024-05-17T22:05:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- 172.21.15.83:0/2360676251 >> 172.21.15.83:0/2360676251 conn(0x7fc02c06cd90 msgr2=0x7fc02c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- 172.21.15.83:0/2360676251 shutdown_connections 2024-05-17T22:05:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- 172.21.15.83:0/2360676251 wait complete. 2024-05-17T22:05:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 Processor -- start 2024-05-17T22:05:12.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.956+0000 7fc03358b640 1 -- start start 2024-05-17T22:05:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c15a0e0 0x7fc02c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 0x7fc02c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c0d10a0 con 0x7fc018005c50 2024-05-17T22:05:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc02c15e880 con 0x7fc02c15a4f0 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc02c0d1220 con 0x7fc02c15a0e0 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60614/0 (socket says 172.21.15.83:60614) 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 -- 172.21.15.83:0/2045863053 learned_addr learned my addr 172.21.15.83:0/2045863053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031b28640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 0x7fc02c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 -- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c15a0e0 msgr2=0x7fc02c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c15a0e0 0x7fc02c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 -- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 msgr2=0x7fc02c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 0x7fc02c0c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 -- 172.21.15.83:0/2045863053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02c0b61b0 con 0x7fc018005c50 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031b28640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 0x7fc02c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc031327640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc024005040 tx=0x7fc018007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240bf070 con 0x7fc018005c50 2024-05-17T22:05:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240013c0 con 0x7fc018005c50 2024-05-17T22:05:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02c0b6490 con 0x7fc018005c50 2024-05-17T22:05:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024005480 con 0x7fc018005c50 2024-05-17T22:05:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02c0b6a50 con 0x7fc018005c50 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02c15e170 con 0x7fc018005c50 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc0240c2070 con 0x7fc018005c50 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.960+0000 7fc021ffb640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 0x7fc010071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.964+0000 7fc030b26640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 0x7fc010071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.964+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fc024128cf0 con 0x7fc018005c50 2024-05-17T22:05:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.964+0000 7fc030b26640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 0x7fc010071910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc02c133a50 tx=0x7fc01c006e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:12.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:12.968+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc02412d070 con 0x7fc018005c50 2024-05-17T22:05:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc02c0b41a0 con 0x7fc018005c50 2024-05-17T22:05:13.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc021ffb640 1 -- 172.21.15.83:0/2045863053 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7fc024016130 con 0x7fc018005c50 2024-05-17T22:05:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:13.151 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:05:13.151 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:05:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 msgr2=0x7fc010071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 0x7fc010071910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc02c133a50 tx=0x7fc01c006e20 comp rx=0 tx=0).stop 2024-05-17T22:05:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 msgr2=0x7fc02c0c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc024005040 tx=0x7fc018007a50 comp rx=0 tx=0).stop 2024-05-17T22:05:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 shutdown_connections 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc01006f400 0x7fc010071910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc02c15a4f0 0x7fc02c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc02c15a0e0 0x7fc02c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 --2- 172.21.15.83:0/2045863053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc018005c50 0x7fc02c0c7660 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 >> 172.21.15.83:0/2045863053 conn(0x7fc02c06cd90 msgr2=0x7fc02c15ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 shutdown_connections 2024-05-17T22:05:13.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.160+0000 7fc03358b640 1 -- 172.21.15.83:0/2045863053 wait complete. 2024-05-17T22:05:13.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:13.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 Processor -- start 2024-05-17T22:05:13.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 -- start start 2024-05-17T22:05:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 0x7fe6c0144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6c015e080 0x7fe6c0072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6c015ea40 con 0x7fe6c015e490 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c015ed20 con 0x7fe6c015e080 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6c48a9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c015f000 con 0x7fe6c00d1040 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bdd9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6c015e080 0x7fe6c0072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe6c015e490 0x7fe6c01616c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33656/0 (socket says 172.21.15.83:33656) 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bed9d640 1 -- 172.21.15.83:0/2143444835 learned_addr learned my addr 172.21.15.83:0/2143444835 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6be59c640 1 --2- 172.21.15.83:0/2143444835 >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 0x7fe6c0144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/2143444835 >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 msgr2=0x7fe6c0144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bdd9b640 1 --2- 172.21.15.83:0/2143444835 >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 0x7fe6c0144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/2143444835 >> v1:172.21.15.83:6789/0 conn(0x7fe6c015e490 legacy=0x7fe6c01616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/2143444835 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c006ef30 con 0x7fe6c015e080 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6bd59a640 1 -- 172.21.15.83:0/2143444835 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4107895421 0 0) 0x7fe6c015ea40 con 0x7fe6c015e490 2024-05-17T22:05:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.332+0000 7fe6be59c640 1 --2- 172.21.15.83:0/2143444835 >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 0x7fe6c0144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6bdd9b640 1 --2- 172.21.15.83:0/2143444835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c0072e30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0071d60 tx=0x7fe6b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f6d6f3415b80a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6bd59a640 1 -- 172.21.15.83:0/2143444835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40b1070 con 0x7fe6c015e080 2024-05-17T22:05:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6bd59a640 1 -- 172.21.15.83:0/2143444835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b40ac5f0 con 0x7fe6c015e080 2024-05-17T22:05:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6bd59a640 1 -- 172.21.15.83:0/2143444835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40ac9c0 con 0x7fe6c015e080 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 -- 172.21.15.83:0/2143444835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 msgr2=0x7fe6c0072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/2143444835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c0072e30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0071d60 tx=0x7fe6b401c480 comp rx=0 tx=0).stop 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 -- 172.21.15.83:0/2143444835 shutdown_connections 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/2143444835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c0072e30 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/2143444835 >> v2:172.21.15.155:3300/0 conn(0x7fe6c00d1040 0x7fe6c0144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 -- 172.21.15.83:0/2143444835 >> 172.21.15.83:0/2143444835 conn(0x7fe6c006cd90 msgr2=0x7fe6c00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 -- 172.21.15.83:0/2143444835 shutdown_connections 2024-05-17T22:05:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.336+0000 7fe6c48a9640 1 -- 172.21.15.83:0/2143444835 wait complete. 2024-05-17T22:05:13.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 Processor -- start 2024-05-17T22:05:13.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- start start 2024-05-17T22:05:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6c00d1040 0x7fe6c01581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6c015e490 0x7fe6c01f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c016abe0 con 0x7fe6c015e080 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c016aa60 con 0x7fe6c00d1040 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c016ad60 con 0x7fe6c015e490 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60632/0 (socket says 172.21.15.83:60632) 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/4011966700 learned_addr learned my addr 172.21.15.83:0/4011966700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6c015e490 msgr2=0x7fe6c01f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6c015e490 0x7fe6c01f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6c00d1040 msgr2=0x7fe6c01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6c00d1040 0x7fe6c01581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 -- 172.21.15.83:0/4011966700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c013e320 con 0x7fe6c015e080 2024-05-17T22:05:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6bdd9b640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe6b401c960 tx=0x7fe6b4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40b1040 con 0x7fe6c015e080 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b40ba070 con 0x7fe6c015e080 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01f80c0 con 0x7fe6c015e080 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40b5870 con 0x7fe6c015e080 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.340+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c013f900 con 0x7fe6c015e080 2024-05-17T22:05:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c01f7e30 con 0x7fe6c015e080 2024-05-17T22:05:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6b40ac4b0 con 0x7fe6c015e080 2024-05-17T22:05:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6aeffd640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 0x7fe6a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6be59c640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 0x7fe6a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fe6b4129210 con 0x7fe6c015e080 2024-05-17T22:05:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.344+0000 7fe6be59c640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 0x7fe6a0071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0008e00 tx=0x7fe6b0008330 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.348+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe6b40a9070 con 0x7fe6c015e080 2024-05-17T22:05:13.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.508+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6c0144080 con 0x7fe6c015e080 2024-05-17T22:05:13.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6aeffd640 1 -- 172.21.15.83:0/4011966700 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16991 (secure 0 0 0) 0x7fe6b4016070 con 0x7fe6c015e080 2024-05-17T22:05:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:13.503 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:04:31.611350+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:02:52.464989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-17T22:02:53.753100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6834/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6836/3213750564":"2024-05-18T22:03:12.435992+0000","172.21.15.83:6834/3213750564":"2024-05-18T22:03:12.435992+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 msgr2=0x7fe6a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 0x7fe6a0071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0008e00 tx=0x7fe6b0008330 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 msgr2=0x7fe6c01f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe6b401c960 tx=0x7fe6b4019220 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 shutdown_connections 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6c015e490 0x7fe6c01f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6c015e080 0x7fe6c01f73b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe6a006f400 0x7fe6a0071910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 --2- 172.21.15.83:0/4011966700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6c00d1040 0x7fe6c01581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 >> 172.21.15.83:0/4011966700 conn(0x7fe6c006cd90 msgr2=0x7fe6c015f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 shutdown_connections 2024-05-17T22:05:13.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.512+0000 7fe6c48a9640 1 -- 172.21.15.83:0/4011966700 wait complete. 2024-05-17T22:05:13.524 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:05:13.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:13.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 Processor -- start 2024-05-17T22:05:13.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 -- start start 2024-05-17T22:05:13.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa8580d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa85806e3d0 0x7fa8580d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8580d34e0 con 0x7fa8580d1150 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa8580d3780 con 0x7fa85806e3d0 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa860107640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580d3a60 con 0x7fa85815ffa0 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa85806e3d0 0x7fa8580d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa85806e3d0 0x7fa8580d4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60638/0 (socket says 172.21.15.83:60638) 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 -- 172.21.15.83:0/79985964 learned_addr learned my addr 172.21.15.83:0/79985964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85dea3640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa8580d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 -- 172.21.15.83:0/79985964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 msgr2=0x7fa8580d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa8580d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 -- 172.21.15.83:0/79985964 >> v1:172.21.15.155:6789/0 conn(0x7fa8580d1150 legacy=0x7fa85806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:13.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.668+0000 7fa85e6a4640 1 -- 172.21.15.83:0/79985964 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8580d3d40 con 0x7fa85806e3d0 2024-05-17T22:05:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa85e6a4640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa85806e3d0 0x7fa8580d4260 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244eb14f437b5660 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa85cea1640 1 -- 172.21.15.83:0/79985964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500a5670 con 0x7fa85806e3d0 2024-05-17T22:05:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa85cea1640 1 -- 172.21.15.83:0/79985964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500a5810 con 0x7fa85806e3d0 2024-05-17T22:05:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa85cea1640 1 -- 172.21.15.83:0/79985964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500a5be0 con 0x7fa85806e3d0 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 -- 172.21.15.83:0/79985964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa85806e3d0 msgr2=0x7fa8580d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa85806e3d0 0x7fa8580d4260 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa850009e50 tx=0x7fa85001ca30 comp rx=0 tx=0).stop 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 -- 172.21.15.83:0/79985964 shutdown_connections 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa85806e3d0 0x7fa8580d4260 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 --2- 172.21.15.83:0/79985964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa8580d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 -- 172.21.15.83:0/79985964 >> 172.21.15.83:0/79985964 conn(0x7fa8580d28f0 msgr2=0x7fa8580d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:13.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 -- 172.21.15.83:0/79985964 shutdown_connections 2024-05-17T22:05:13.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.672+0000 7fa860107640 1 -- 172.21.15.83:0/79985964 wait complete. 2024-05-17T22:05:13.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 Processor -- start 2024-05-17T22:05:13.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- start start 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa858157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa85815bb60 0x7fa85815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580d5340 con 0x7fa8580d1150 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa8580d51c0 con 0x7fa85815bb60 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa8580d54c0 con 0x7fa85815ffa0 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60648/0 (socket says 172.21.15.83:60648) 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 -- 172.21.15.83:0/1166614288 learned_addr learned my addr 172.21.15.83:0/1166614288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 -- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 msgr2=0x7fa858157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85d6a2640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa858157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85e6a4640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa85815bb60 0x7fa85815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa858157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 -- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa85815bb60 msgr2=0x7fa85815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa85815bb60 0x7fa85815bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 -- 172.21.15.83:0/1166614288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85815cea0 con 0x7fa8580d1150 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa85dea3640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa85400d620 tx=0x7fa85400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85400b620 con 0x7fa8580d1150 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa854019070 con 0x7fa8580d1150 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85815c4b0 con 0x7fa8580d1150 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85400b9e0 con 0x7fa8580d1150 2024-05-17T22:05:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8580ce2a0 con 0x7fa8580d1150 2024-05-17T22:05:13.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.676+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8580d4260 con 0x7fa8580d1150 2024-05-17T22:05:13.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.680+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa85400bb80 con 0x7fa8580d1150 2024-05-17T22:05:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.680+0000 7fa84e7fc640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.680+0000 7fa85d6a2640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.680+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7fa85408f850 con 0x7fa8580d1150 2024-05-17T22:05:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.680+0000 7fa85d6a2640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 0x7fa83c071910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa8581fc230 tx=0x7fa848008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:13.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.684+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa85815c4b0 con 0x7fa8580d1150 2024-05-17T22:05:13.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.864+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8580b5d40 con 0x7fa8580d1150 2024-05-17T22:05:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.864+0000 7fa84e7fc640 1 -- 172.21.15.83:0/1166614288 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7fa85405c650 con 0x7fa8580d1150 2024-05-17T22:05:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:13.856 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:05:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 msgr2=0x7fa83c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 0x7fa83c071910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa8581fc230 tx=0x7fa848008040 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 msgr2=0x7fa85815c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa85400d620 tx=0x7fa85400dae0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 shutdown_connections 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa83c06f400 0x7fa83c071910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa85815bb60 0x7fa85815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa85815ffa0 0x7fa858157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 --2- 172.21.15.83:0/1166614288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8580d1150 0x7fa85815c960 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 >> 172.21.15.83:0/1166614288 conn(0x7fa8580d28f0 msgr2=0x7fa8580b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 shutdown_connections 2024-05-17T22:05:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:13.868+0000 7fa860107640 1 -- 172.21.15.83:0/1166614288 wait complete. 2024-05-17T22:05:13.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:14.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.016+0000 7f1aeb6a9640 1 Processor -- start 2024-05-17T22:05:14.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.016+0000 7f1aeb6a9640 1 -- start start 2024-05-17T22:05:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1aeb6a9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ae41461c0 0x7f1ae415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1aeb6a9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 0x7f1ae406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1aeb6a9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae41465d0 con 0x7f1ae4070510 2024-05-17T22:05:14.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1aeb6a9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae406ef30 con 0x7f1ae41461c0 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1aeb6a9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae406f210 con 0x7f1ae415e6b0 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ae41461c0 0x7f1ae415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1ae41461c0 0x7f1ae415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60650/0 (socket says 172.21.15.83:60650) 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 -- 172.21.15.83:0/2557370053 learned_addr learned my addr 172.21.15.83:0/2557370053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae8c44640 1 --2- 172.21.15.83:0/2557370053 >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 0x7f1ae406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 -- 172.21.15.83:0/2557370053 >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 msgr2=0x7f1ae406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 --2- 172.21.15.83:0/2557370053 >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 0x7f1ae406ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 -- 172.21.15.83:0/2557370053 >> v1:172.21.15.155:6789/0 conn(0x7f1ae4070510 legacy=0x7f1ae41605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae9445640 1 -- 172.21.15.83:0/2557370053 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae4070920 con 0x7f1ae41461c0 2024-05-17T22:05:14.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.020+0000 7f1ae8c44640 1 --2- 172.21.15.83:0/2557370053 >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 0x7f1ae406ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:14.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1ae9445640 1 --2- 172.21.15.83:0/2557370053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae41461c0 0x7f1ae415e170 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4071230 tx=0x7f1adc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d05a481c6d07300 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1ae3fff640 1 -- 172.21.15.83:0/2557370053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0aa610 con 0x7f1ae41461c0 2024-05-17T22:05:14.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1ae3fff640 1 -- 172.21.15.83:0/2557370053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc0af070 con 0x7f1ae41461c0 2024-05-17T22:05:14.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1ae3fff640 1 -- 172.21.15.83:0/2557370053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0aa9d0 con 0x7f1ae41461c0 2024-05-17T22:05:14.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/2557370053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae41461c0 msgr2=0x7f1ae415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/2557370053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae41461c0 0x7f1ae415e170 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4071230 tx=0x7f1adc01c4f0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/2557370053 shutdown_connections 2024-05-17T22:05:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/2557370053 >> v2:172.21.15.155:3300/0 conn(0x7f1ae415e6b0 0x7f1ae406ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/2557370053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae41461c0 0x7f1ae415e170 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/2557370053 >> 172.21.15.83:0/2557370053 conn(0x7f1ae406cd90 msgr2=0x7f1ae40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/2557370053 shutdown_connections 2024-05-17T22:05:14.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/2557370053 wait complete. 2024-05-17T22:05:14.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 Processor -- start 2024-05-17T22:05:14.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.024+0000 7f1aeb6a9640 1 -- start start 2024-05-17T22:05:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ae41461c0 0x7f1ae415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ae415e6b0 0x7f1ae40b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4160e30 con 0x7f1ae4070510 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4160cb0 con 0x7f1ae41461c0 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae40d10a0 con 0x7f1ae415e6b0 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60658/0 (socket says 172.21.15.83:60658) 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 -- 172.21.15.83:0/3392502209 learned_addr learned my addr 172.21.15.83:0/3392502209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 -- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ae415e6b0 msgr2=0x7f1ae40b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ae415e6b0 0x7f1ae40b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 -- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ae41461c0 msgr2=0x7f1ae415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ae41461c0 0x7f1ae415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 -- 172.21.15.83:0/3392502209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae41f9520 con 0x7f1ae4070510 2024-05-17T22:05:14.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae9445640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1adc01aba0 tx=0x7f1adc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0bd5f0 con 0x7f1ae4070510 2024-05-17T22:05:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc0bd7b0 con 0x7f1ae4070510 2024-05-17T22:05:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae41f9800 con 0x7f1ae4070510 2024-05-17T22:05:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0bdba0 con 0x7f1ae4070510 2024-05-17T22:05:14.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41f9dc0 con 0x7f1ae4070510 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae415e170 con 0x7f1ae4070510 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1adc0b33f0 con 0x7f1ae4070510 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 0x7f1ad0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f1adc129a80 con 0x7f1ae4070510 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae8c44640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 0x7f1ad0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.028+0000 7f1ae8c44640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 0x7f1ad0071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ae41458c0 tx=0x7f1ad8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.032+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1adc0c3060 con 0x7f1ae4070510 2024-05-17T22:05:14.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.228+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae40b41a0 con 0x7f1ae4070510 2024-05-17T22:05:14.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.228+0000 7f1ae1ffb640 1 -- 172.21.15.83:0/3392502209 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5190 (secure 0 0 0) 0x7f1adc0a7070 con 0x7f1ae4070510 2024-05-17T22:05:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:14.221 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:02:54.779677+0000","modified":"2024-05-17T22:04:35.641883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7083},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7083":{"gid":7083,"name":"a","rank":0,"incarnation":69,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/2140054867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2140054867},{"type":"v1","addr":"172.21.15.83:6836","nonce":2140054867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7083,"qdb_cluster":[7083]},"id":6}]} 2024-05-17T22:05:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2024-05-17T22:05:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 msgr2=0x7f1ad0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 0x7f1ad0071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ae41458c0 tx=0x7f1ad8008040 comp rx=0 tx=0).stop 2024-05-17T22:05:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 msgr2=0x7f1ae4141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1adc01aba0 tx=0x7f1adc01ad30 comp rx=0 tx=0).stop 2024-05-17T22:05:14.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 shutdown_connections 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ae415e6b0 0x7f1ae40b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1ad006f400 0x7f1ad0071910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ae41461c0 0x7f1ae415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 --2- 172.21.15.83:0/3392502209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ae4070510 0x7f1ae4141ba0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 >> 172.21.15.83:0/3392502209 conn(0x7f1ae406cd90 msgr2=0x7f1ae41423f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 shutdown_connections 2024-05-17T22:05:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.232+0000 7f1aeb6a9640 1 -- 172.21.15.83:0/3392502209 wait complete. 2024-05-17T22:05:14.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 Processor -- start 2024-05-17T22:05:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 -- start start 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 0x7f4f18144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f18073540 0x7f4f1806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f1815ea40 con 0x7f4f1815e080 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1815ed20 con 0x7f4f18073540 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f200ac640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1815f000 con 0x7f4f180d1040 2024-05-17T22:05:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1e649640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f18073540 0x7f4f1806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1d647640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4f1815e080 0x7f4f18072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33664/0 (socket says 172.21.15.83:33664) 2024-05-17T22:05:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1d647640 1 -- 172.21.15.83:0/239285035 learned_addr learned my addr 172.21.15.83:0/239285035 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1de48640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 0x7f4f18144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1e649640 1 -- 172.21.15.83:0/239285035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 msgr2=0x7f4f18144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1e649640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 0x7f4f18144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1e649640 1 -- 172.21.15.83:0/239285035 >> v1:172.21.15.83:6789/0 conn(0x7f4f1815e080 legacy=0x7f4f18072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1e649640 1 -- 172.21.15.83:0/239285035 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1806ef30 con 0x7f4f18073540 2024-05-17T22:05:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.400+0000 7f4f1de48640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 0x7f4f18144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f1e649640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f18073540 0x7f4f1806f730 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009d90 tx=0x7f4f1001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827ccfdf167584da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f1ce46640 1 -- 172.21.15.83:0/239285035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100b2070 con 0x7f4f18073540 2024-05-17T22:05:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f1ce46640 1 -- 172.21.15.83:0/239285035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f100a5940 con 0x7f4f18073540 2024-05-17T22:05:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f1ce46640 1 -- 172.21.15.83:0/239285035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100a5d10 con 0x7f4f18073540 2024-05-17T22:05:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 -- 172.21.15.83:0/239285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f18073540 msgr2=0x7f4f1806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f18073540 0x7f4f1806f730 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009d90 tx=0x7f4f1001c970 comp rx=0 tx=0).stop 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 -- 172.21.15.83:0/239285035 shutdown_connections 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f18073540 0x7f4f1806f730 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 --2- 172.21.15.83:0/239285035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f180d1040 0x7f4f18144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 -- 172.21.15.83:0/239285035 >> 172.21.15.83:0/239285035 conn(0x7f4f1806cd90 msgr2=0x7f4f180b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 -- 172.21.15.83:0/239285035 shutdown_connections 2024-05-17T22:05:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 -- 172.21.15.83:0/239285035 wait complete. 2024-05-17T22:05:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.404+0000 7f4f200ac640 1 Processor -- start 2024-05-17T22:05:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- start start 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f1815e080 0x7f4f18142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 0x7f4f18142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18070810 con 0x7f4f180d1040 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18070690 con 0x7f4f1815e080 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4f18070990 con 0x7f4f18161c60 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60684/0 (socket says 172.21.15.83:60684) 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 -- 172.21.15.83:0/1553386278 learned_addr learned my addr 172.21.15.83:0/1553386278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1d647640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f1815e080 0x7f4f18142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1e649640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 0x7f4f18142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 -- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 msgr2=0x7f4f18142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 0x7f4f18142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 -- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f1815e080 msgr2=0x7f4f18142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f1815e080 0x7f4f18142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 -- 172.21.15.83:0/1553386278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1815d020 con 0x7f4f180d1040 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1e649640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 0x7f4f18142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f1de48640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c820 tx=0x7f4f0800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08015070 con 0x7f4f180d1040 2024-05-17T22:05:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f08007a30 con 0x7f4f180d1040 2024-05-17T22:05:14.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1815d300 con 0x7f4f180d1040 2024-05-17T22:05:14.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08007e40 con 0x7f4f180d1040 2024-05-17T22:05:14.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f180b3850 con 0x7f4f180d1040 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1806f730 con 0x7f4f180d1040 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.408+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4f08018070 con 0x7f4f180d1040 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.412+0000 7f4f167fc640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 0x7f4f04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.412+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7f4f0808f350 con 0x7f4f180d1040 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.412+0000 7f4f1d647640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 0x7f4f04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.412+0000 7f4f1d647640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 0x7f4f04071910 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f18143b30 tx=0x7f4f0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.416+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4f0805c150 con 0x7f4f180d1040 2024-05-17T22:05:14.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.564+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f181458a0 con 0x7f4f180d1040 2024-05-17T22:05:14.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f167fc640 1 -- 172.21.15.83:0/1553386278 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16991 (secure 0 0 0) 0x7f4f0805ba50 con 0x7f4f180d1040 2024-05-17T22:05:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:04:31.611350+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:02:52.464989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-17T22:02:53.753100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6834/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6836/3213750564":"2024-05-18T22:03:12.435992+0000","172.21.15.83:6834/3213750564":"2024-05-18T22:03:12.435992+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:14.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 msgr2=0x7f4f04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 0x7f4f04071910 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f18143b30 tx=0x7f4f0c009040 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 msgr2=0x7f4f1815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c820 tx=0x7f4f0800cce0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 shutdown_connections 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f18161c60 0x7f4f18142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4f0406f400 0x7f4f04071910 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f1815e080 0x7f4f18142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 --2- 172.21.15.83:0/1553386278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f180d1040 0x7f4f1815be60 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 >> 172.21.15.83:0/1553386278 conn(0x7f4f1806cd90 msgr2=0x7f4f18147050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 shutdown_connections 2024-05-17T22:05:14.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.568+0000 7f4f200ac640 1 -- 172.21.15.83:0/1553386278 wait complete. 2024-05-17T22:05:14.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:05:14.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.732+0000 7ff4839d3640 1 Processor -- start 2024-05-17T22:05:14.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.732+0000 7ff4839d3640 1 -- start start 2024-05-17T22:05:14.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 0x7ff47c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c15eae0 0x7ff47c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff47c143720 0x7ff47c0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff47c0b71b0 con 0x7ff47c143720 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff47c0ce530 con 0x7ff47c0b68c0 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff4839d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c143b30 con 0x7ff47c15eae0 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff47c143720 0x7ff47c0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff47c143720 0x7ff47c0b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60690/0 (socket says 172.21.15.83:60690) 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 -- 172.21.15.83:0/3906358918 learned_addr learned my addr 172.21.15.83:0/3906358918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff48176f640 1 --2- 172.21.15.83:0/3906358918 >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 0x7ff47c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff480f6e640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c15eae0 0x7ff47c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 -- 172.21.15.83:0/3906358918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c15eae0 msgr2=0x7ff47c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c15eae0 0x7ff47c1431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 -- 172.21.15.83:0/3906358918 >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 msgr2=0x7ff47c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 --2- 172.21.15.83:0/3906358918 >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 0x7ff47c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff481f70640 1 -- 172.21.15.83:0/3906358918 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c0b4db0 con 0x7ff47c143720 2024-05-17T22:05:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.736+0000 7ff48176f640 1 --2- 172.21.15.83:0/3906358918 >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 0x7ff47c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff481f70640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c0b0c10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0d2c70 tx=0x7ff47401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5950e0cc3497e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff473fff640 1 -- 172.21.15.83:0/3906358918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740a5780 con 0x7ff47c143720 2024-05-17T22:05:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff473fff640 1 -- 172.21.15.83:0/3906358918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4740a5920 con 0x7ff47c143720 2024-05-17T22:05:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff473fff640 1 -- 172.21.15.83:0/3906358918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740a5cf0 con 0x7ff47c143720 2024-05-17T22:05:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- 172.21.15.83:0/3906358918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 msgr2=0x7ff47c0b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c0b0c10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0d2c70 tx=0x7ff47401c9f0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- 172.21.15.83:0/3906358918 shutdown_connections 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c0b0c10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 --2- 172.21.15.83:0/3906358918 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c15eae0 0x7ff47c1431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 --2- 172.21.15.83:0/3906358918 >> v2:172.21.15.155:3300/0 conn(0x7ff47c0b68c0 0x7ff47c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- 172.21.15.83:0/3906358918 >> 172.21.15.83:0/3906358918 conn(0x7ff47c0ce6e0 msgr2=0x7ff47c0ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- 172.21.15.83:0/3906358918 shutdown_connections 2024-05-17T22:05:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- 172.21.15.83:0/3906358918 wait complete. 2024-05-17T22:05:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 Processor -- start 2024-05-17T22:05:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.740+0000 7ff4839d3640 1 -- start start 2024-05-17T22:05:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0b68c0 0x7ff47c146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c15eae0 0x7ff47c1f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c15eef0 con 0x7ff47c143720 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c0b1560 con 0x7ff47c15eae0 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c1f9aa0 con 0x7ff47c0b68c0 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60700/0 (socket says 172.21.15.83:60700) 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 -- 172.21.15.83:0/1193129308 learned_addr learned my addr 172.21.15.83:0/1193129308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 -- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0b68c0 msgr2=0x7ff47c146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0b68c0 0x7ff47c146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 -- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c15eae0 msgr2=0x7ff47c1f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c15eae0 0x7ff47c1f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 -- 172.21.15.83:0/1193129308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c145870 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff480f6e640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c640 tx=0x7ff46c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c015070 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c004220 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c145b50 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c003620 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c146110 con 0x7ff47c143720 2024-05-17T22:05:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c0b0c10 con 0x7ff47c143720 2024-05-17T22:05:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.744+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff46c0037c0 con 0x7ff47c143720 2024-05-17T22:05:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.748+0000 7ff471ffb640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 0x7ff460071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.748+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5939+0+0 (secure 0 0 0) 0x7ff46c08ef20 con 0x7ff47c143720 2024-05-17T22:05:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.748+0000 7ff48176f640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 0x7ff460071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.748+0000 7ff48176f640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 0x7ff460071940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff47c162710 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.752+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff46c05bdb0 con 0x7ff47c143720 2024-05-17T22:05:14.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.932+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff47c06da90 con 0x7ff47c143720 2024-05-17T22:05:14.923 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:05:14.932+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:05:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff471ffb640 1 -- 172.21.15.83:0/1193129308 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7ff46c05b6b0 con 0x7ff47c143720 2024-05-17T22:05:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 msgr2=0x7ff460071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 0x7ff460071940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff47c162710 tx=0x7ff478008040 comp rx=0 tx=0).stop 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 msgr2=0x7ff47c146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c640 tx=0x7ff46c00cb00 comp rx=0 tx=0).stop 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 shutdown_connections 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff46006f430 0x7ff460071940 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff47c15eae0 0x7ff47c1f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff47c143720 0x7ff47c146e50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 --2- 172.21.15.83:0/1193129308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff47c0b68c0 0x7ff47c146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 >> 172.21.15.83:0/1193129308 conn(0x7ff47c0ce6e0 msgr2=0x7ff47c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 shutdown_connections 2024-05-17T22:05:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:14.936+0000 7ff4839d3640 1 -- 172.21.15.83:0/1193129308 wait complete. 2024-05-17T22:05:14.933 INFO:tasks.ceph.mds.a.smithi083.stderr: -2133> 2024-05-17T22:04:38.625+0000 7f3030588640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:05:14.934 INFO:tasks.ceph.mds.a.smithi083.stderr: -1842> 2024-05-17T22:04:40.213+0000 7f302c580640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5558616c0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v23>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-17T22:05:14.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:05:15.006 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:05:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.100+0000 7fefb8c4c640 1 Processor -- start 2024-05-17T22:05:15.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.100+0000 7fefb8c4c640 1 -- start start 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb8c4c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fefb40b4c10 0x7fefb40cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb8c4c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb415e040 con 0x7fefb40b5020 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb8c4c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb415e300 con 0x7fefb41438f0 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb8c4c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fefb415e5e0 con 0x7fefb40b4c10 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefa9d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fefb40b5020 0x7fefb41431e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33672/0 (socket says 172.21.15.83:33672) 2024-05-17T22:05:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefa9d9b640 1 -- 172.21.15.83:0/2026055655 learned_addr learned my addr 172.21.15.83:0/2026055655 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb259c640 1 --2- 172.21.15.83:0/2026055655 >> v2:172.21.15.155:3300/0 conn(0x7fefb40b4c10 0x7fefb40cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076589166 0 0) 0x7fefb415e040 con 0x7fefb40b5020 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa0003530 con 0x7fefb40b5020 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2871318561 0 0) 0x7fefa0003530 con 0x7fefb40b5020 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 >> v2:172.21.15.155:3300/0 conn(0x7fefb40b4c10 msgr2=0x7fefb40cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 --2- 172.21.15.83:0/2026055655 >> v2:172.21.15.155:3300/0 conn(0x7fefb40b4c10 0x7fefb40cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 >> v1:172.21.15.155:6789/0 conn(0x7fefb41438f0 legacy=0x7fefb40b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb415e8c0 con 0x7fefb40b5020 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fef9c001700 con 0x7fefb40b5020 2024-05-17T22:05:15.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 >> v1:172.21.15.83:6789/0 conn(0x7fefb40b5020 legacy=0x7fefb41431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa0003a60 0x7fefa0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0004810 0x7fefa0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 0x7fefa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefb415e5e0 con 0x7fefa0005360 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefa000c290 con 0x7fefa0003a60 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fefa000c490 con 0x7fefa0004810 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 0x7fefa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefa9d9b640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0004810 0x7fefa0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb259c640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa0003a60 0x7fefa0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 -- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0004810 msgr2=0x7fefa0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0004810 0x7fefa0004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 -- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa0003a60 msgr2=0x7fefa0003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa0003a60 0x7fefa0003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb2d9d640 1 -- 172.21.15.83:0/2026055655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa0003530 con 0x7fefa0005360 2024-05-17T22:05:15.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.104+0000 7fefb259c640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa0003a60 0x7fefa0003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb2d9d640 1 -- 172.21.15.83:0/2026055655 reap_dead start 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb2d9d640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 0x7fefa000b9d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fefb415eec0 tx=0x7fefac01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb48f8bbfa62e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac001ae0 con 0x7fefa0005360 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefac003b80 con 0x7fefa0005360 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb1d9b640 1 -- 172.21.15.83:0/2026055655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac005020 con 0x7fefa0005360 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 msgr2=0x7fefa000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 0x7fefa000b9d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fefb415eec0 tx=0x7fefac01c710 comp rx=0 tx=0).stop 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- 172.21.15.83:0/2026055655 shutdown_connections 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/2026055655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefa0005360 0x7fefa000b9d0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- 172.21.15.83:0/2026055655 >> 172.21.15.83:0/2026055655 conn(0x7fefb40ce6e0 msgr2=0x7fefb40ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- 172.21.15.83:0/2026055655 shutdown_connections 2024-05-17T22:05:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- 172.21.15.83:0/2026055655 wait complete. 2024-05-17T22:05:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 Processor -- start 2024-05-17T22:05:15.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.108+0000 7fefb8c4c640 1 -- start start 2024-05-17T22:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0005360 0x7fefb40afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb41f71a0 0x7fefb41f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40cddc0 con 0x7fefb415cf90 2024-05-17T22:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40689d0 con 0x7fefb41f71a0 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fefb40cdf40 con 0x7fefa0005360 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60736/0 (socket says 172.21.15.83:60736) 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 -- 172.21.15.83:0/3060788823 learned_addr learned my addr 172.21.15.83:0/3060788823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb259c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0005360 0x7fefb40afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb2d9d640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb41f71a0 0x7fefb41f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 -- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0005360 msgr2=0x7fefb40afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0005360 0x7fefb40afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 -- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb41f71a0 msgr2=0x7fefb41f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb41f71a0 0x7fefb41f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 -- 172.21.15.83:0/3060788823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb409fd20 con 0x7fefb415cf90 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefa9d9b640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000fc0 tx=0x7fef9c00e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c01b070 con 0x7fefb415cf90 2024-05-17T22:05:15.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c011040 con 0x7fefb415cf90 2024-05-17T22:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41f44e0 con 0x7fefb415cf90 2024-05-17T22:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41f3f20 con 0x7fefb415cf90 2024-05-17T22:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0143a0 con 0x7fefb415cf90 2024-05-17T22:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb40ce320 con 0x7fefb415cf90 2024-05-17T22:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fef9c01e070 con 0x7fefb415cf90 2024-05-17T22:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.112+0000 7fefab7fe640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 0x7fef94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.116+0000 7fefb259c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 0x7fef94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.116+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6025+0+0 (secure 0 0 0) 0x7fef9c091ef0 con 0x7fefb415cf90 2024-05-17T22:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.116+0000 7fefb259c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 0x7fef94071910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009a20 tx=0x7fefa4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.120+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fef9c05e650 con 0x7fefb415cf90 2024-05-17T22:05:15.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.312+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fefb4160ee0 con 0x7fefb415cf90 2024-05-17T22:05:15.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefab7fe640 1 -- 172.21.15.83:0/3060788823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fef9c062950 con 0x7fefb415cf90 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 msgr2=0x7fef94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 0x7fef94071910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009a20 tx=0x7fefa4008040 comp rx=0 tx=0).stop 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 msgr2=0x7fefb409f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000fc0 tx=0x7fef9c00e100 comp rx=0 tx=0).stop 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 shutdown_connections 2024-05-17T22:05:15.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef9406f400 0x7fef94071910 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb41f71a0 0x7fefb41f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefb415cf90 0x7fefb409f410 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 --2- 172.21.15.83:0/3060788823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefa0005360 0x7fefb40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.948+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 >> 172.21.15.83:0/3060788823 conn(0x7fefb40ce6e0 msgr2=0x7fefb40d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.952+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 shutdown_connections 2024-05-17T22:05:15.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:15.952+0000 7fefb8c4c640 1 -- 172.21.15.83:0/3060788823 wait complete. 2024-05-17T22:05:15.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:05:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.124+0000 7fc14f7eb640 1 Processor -- start 2024-05-17T22:05:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.124+0000 7fc14f7eb640 1 -- start start 2024-05-17T22:05:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14f7eb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc148160420 0x7fc148160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14f7eb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc148160d70 con 0x7fc1480b4f20 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14f7eb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1480d0820 con 0x7fc1481570e0 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14f7eb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc1480b4a90 con 0x7fc148160420 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14dd88640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1480b4f20 0x7fc1480b5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33674/0 (socket says 172.21.15.83:33674) 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14dd88640 1 -- 172.21.15.83:0/1886965057 learned_addr learned my addr 172.21.15.83:0/1886965057 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962069866 0 0) 0x7fc1400014a0 con 0x7fc1480b4f20 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14cd86640 1 --2- 172.21.15.83:0/1886965057 >> v2:172.21.15.155:3300/0 conn(0x7fc148160420 0x7fc148160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc134003530 con 0x7fc1480b4f20 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 520372246 0 0) 0x7fc134003530 con 0x7fc1480b4f20 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 >> v2:172.21.15.155:3300/0 conn(0x7fc148160420 msgr2=0x7fc148160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 --2- 172.21.15.83:0/1886965057 >> v2:172.21.15.155:3300/0 conn(0x7fc148160420 0x7fc148160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 >> v1:172.21.15.155:6789/0 conn(0x7fc1481570e0 legacy=0x7fc1480b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480d0b00 con 0x7fc1480b4f20 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fc140001c50 con 0x7fc1480b4f20 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 >> v1:172.21.15.83:6789/0 conn(0x7fc1480b4f20 legacy=0x7fc1480b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 0x7fc134003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134004810 0x7fc134004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc134005260 0x7fc13400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b4a90 con 0x7fc134003a60 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc13400c1f0 con 0x7fc134004810 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc13400c3f0 con 0x7fc134005260 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 0x7fc134003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14cd86640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134004810 0x7fc134004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14dd88640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc134005260 0x7fc13400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 -- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc134005260 msgr2=0x7fc13400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc134005260 0x7fc13400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 -- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134004810 msgr2=0x7fc134004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134004810 0x7fc134004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14d587640 1 -- 172.21.15.83:0/1886965057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134003530 con 0x7fc134003a60 2024-05-17T22:05:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.128+0000 7fc14dd88640 1 -- 172.21.15.83:0/1886965057 reap_dead start 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14d587640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 0x7fc134003ed0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc144001790 tx=0x7fc14401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286eaf65d3c0509c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440013e0 con 0x7fc134003a60 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1440a4e50 con 0x7fc134003a60 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc13ffff640 1 -- 172.21.15.83:0/1886965057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144003600 con 0x7fc134003a60 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 -- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 msgr2=0x7fc134003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 0x7fc134003ed0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc144001790 tx=0x7fc14401c3a0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 -- 172.21.15.83:0/1886965057 shutdown_connections 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/1886965057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc134003a60 0x7fc134003ed0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 -- 172.21.15.83:0/1886965057 >> 172.21.15.83:0/1886965057 conn(0x7fc14806cd90 msgr2=0x7fc1480b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 -- 172.21.15.83:0/1886965057 shutdown_connections 2024-05-17T22:05:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.132+0000 7fc14f7eb640 1 -- 172.21.15.83:0/1886965057 wait complete. 2024-05-17T22:05:16.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 Processor -- start 2024-05-17T22:05:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- start start 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134003a60 0x7fc148142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc14814ea60 0x7fc14815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b5330 con 0x7fc14814f4e0 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480d05c0 con 0x7fc134003a60 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480b54b0 con 0x7fc14814ea60 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60762/0 (socket says 172.21.15.83:60762) 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 -- 172.21.15.83:0/3288171999 learned_addr learned my addr 172.21.15.83:0/3288171999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14d587640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134003a60 0x7fc148142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14dd88640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc14814ea60 0x7fc14815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 -- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc14814ea60 msgr2=0x7fc14815f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc14814ea60 0x7fc14815f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 -- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134003a60 msgr2=0x7fc148142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134003a60 0x7fc148142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 -- 172.21.15.83:0/3288171999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc148151b20 con 0x7fc14814f4e0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14cd86640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc138011690 tx=0x7fc138011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800f9c0 con 0x7fc14814f4e0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13800fb60 con 0x7fc14814f4e0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138015460 con 0x7fc14814f4e0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc148158570 con 0x7fc14814f4e0 2024-05-17T22:05:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.136+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc148158b30 con 0x7fc14814f4e0 2024-05-17T22:05:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1480b5840 con 0x7fc14814f4e0 2024-05-17T22:05:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc138015600 con 0x7fc14814f4e0 2024-05-17T22:05:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc13dffb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 0x7fc12c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6025+0+0 (secure 0 0 0) 0x7fc13808fa10 con 0x7fc14814f4e0 2024-05-17T22:05:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc14d587640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 0x7fc12c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.140+0000 7fc14d587640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 0x7fc12c071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc1440ac000 tx=0x7fc144009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.144+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc138019030 con 0x7fc14814f4e0 2024-05-17T22:05:16.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:16.308+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1480b0820 con 0x7fc14814f4e0 2024-05-17T22:05:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.288+0000 7fc13dffb640 1 -- 172.21.15.83:0/3288171999 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7fc13805c850 con 0x7fc14814f4e0 2024-05-17T22:05:17.280 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:05:17.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 msgr2=0x7fc12c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 0x7fc12c071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc1440ac000 tx=0x7fc144009040 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 msgr2=0x7fc14815ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc138011690 tx=0x7fc138011b50 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 shutdown_connections 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc14814ea60 0x7fc14815f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc14814f4e0 0x7fc14815ebd0 secure :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc138011690 tx=0x7fc138011b50 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc12c06f400 0x7fc12c071910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 --2- 172.21.15.83:0/3288171999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc134003a60 0x7fc148142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.292+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 >> 172.21.15.83:0/3288171999 conn(0x7fc14806cd90 msgr2=0x7fc14806f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:17.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.296+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 shutdown_connections 2024-05-17T22:05:17.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.296+0000 7fc14f7eb640 1 -- 172.21.15.83:0/3288171999 wait complete. 2024-05-17T22:05:17.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:05:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.480+0000 7fdf97e01640 1 Processor -- start 2024-05-17T22:05:17.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.480+0000 7fdf97e01640 1 -- start start 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf97e01640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf90160420 0x7fdf90160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf97e01640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf900b4f20 0x7fdf900b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf97e01640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf90160d70 con 0x7fdf901570e0 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf97e01640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdf900b4a90 con 0x7fdf90160420 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf97e01640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdf900d0820 con 0x7fdf900b4f20 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf90160420 0x7fdf90160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf90160420 0x7fdf90160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56586/0 (socket says 172.21.15.83:56586) 2024-05-17T22:05:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 -- 172.21.15.83:0/1581894388 learned_addr learned my addr 172.21.15.83:0/1581894388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:17.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9639e640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf900b4f20 0x7fdf900b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 -- 172.21.15.83:0/1581894388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf900b4f20 msgr2=0x7fdf900b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:17.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf900b4f20 0x7fdf900b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 -- 172.21.15.83:0/1581894388 >> v1:172.21.15.155:6789/0 conn(0x7fdf901570e0 legacy=0x7fdf900b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.484+0000 7fdf9539c640 1 -- 172.21.15.83:0/1581894388 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf900d0b00 con 0x7fdf90160420 2024-05-17T22:05:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf9539c640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90160420 0x7fdf90160830 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400bb10 tx=0x7fdf8401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4451cdf31c0a2deb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf94b9b640 1 -- 172.21.15.83:0/1581894388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf840b4070 con 0x7fdf90160420 2024-05-17T22:05:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf94b9b640 1 -- 172.21.15.83:0/1581894388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf840a7bf0 con 0x7fdf90160420 2024-05-17T22:05:17.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf94b9b640 1 -- 172.21.15.83:0/1581894388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf840af440 con 0x7fdf90160420 2024-05-17T22:05:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- 172.21.15.83:0/1581894388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90160420 msgr2=0x7fdf90160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90160420 0x7fdf90160830 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400bb10 tx=0x7fdf8401ec30 comp rx=0 tx=0).stop 2024-05-17T22:05:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- 172.21.15.83:0/1581894388 shutdown_connections 2024-05-17T22:05:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf900b4f20 0x7fdf900b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 --2- 172.21.15.83:0/1581894388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf90160420 0x7fdf90160830 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- 172.21.15.83:0/1581894388 >> 172.21.15.83:0/1581894388 conn(0x7fdf9006cd90 msgr2=0x7fdf900b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- 172.21.15.83:0/1581894388 shutdown_connections 2024-05-17T22:05:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- 172.21.15.83:0/1581894388 wait complete. 2024-05-17T22:05:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 Processor -- start 2024-05-17T22:05:17.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.488+0000 7fdf97e01640 1 -- start start 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf901570e0 0x7fdf9014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90160420 0x7fdf9014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf90072e50 con 0x7fdf900b4f20 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900d0600 con 0x7fdf90160420 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdf90072fd0 con 0x7fdf901570e0 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56592/0 (socket says 172.21.15.83:56592) 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 -- 172.21.15.83:0/3079529463 learned_addr learned my addr 172.21.15.83:0/3079529463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf9539c640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf901570e0 0x7fdf9014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf9639e640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90160420 0x7fdf9014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 -- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf901570e0 msgr2=0x7fdf9014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf901570e0 0x7fdf9014df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 -- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90160420 msgr2=0x7fdf9014e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90160420 0x7fdf9014e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 -- 172.21.15.83:0/3079529463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf90152580 con 0x7fdf900b4f20 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf95b9d640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c3c0 tx=0x7fdf8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c017070 con 0x7fdf900b4f20 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0052d0 con 0x7fdf900b4f20 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf90151610 con 0x7fdf900b4f20 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0054c0 con 0x7fdf900b4f20 2024-05-17T22:05:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf901508d0 con 0x7fdf900b4f20 2024-05-17T22:05:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90152380 con 0x7fdf900b4f20 2024-05-17T22:05:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdf8c01a070 con 0x7fdf900b4f20 2024-05-17T22:05:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 0x7fdf74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5587+0+0 (secure 0 0 0) 0x7fdf8c08e4f0 con 0x7fdf900b4f20 2024-05-17T22:05:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.492+0000 7fdf9539c640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 0x7fdf74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.496+0000 7fdf9539c640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 0x7fdf74071910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf840b1000 tx=0x7fdf840a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:17.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.500+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdf90152380 con 0x7fdf900b4f20 2024-05-17T22:05:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:17.668+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf900b5330 con 0x7fdf900b4f20 2024-05-17T22:05:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.292+0000 7fdf827fc640 1 -- 172.21.15.83:0/3079529463 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7fdf90160d70 con 0x7fdf900b4f20 2024-05-17T22:05:18.284 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 msgr2=0x7fdf74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 0x7fdf74071910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf840b1000 tx=0x7fdf840a9070 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 msgr2=0x7fdf901510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c3c0 tx=0x7fdf8c00c880 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 shutdown_connections 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdf7406f400 0x7fdf74071910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf90160420 0x7fdf9014e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf901570e0 0x7fdf9014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 --2- 172.21.15.83:0/3079529463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf900b4f20 0x7fdf901510d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 >> 172.21.15.83:0/3079529463 conn(0x7fdf9006cd90 msgr2=0x7fdf9015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:18.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 shutdown_connections 2024-05-17T22:05:18.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.296+0000 7fdf97e01640 1 -- 172.21.15.83:0/3079529463 wait complete. 2024-05-17T22:05:18.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T22:05:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 Processor -- start 2024-05-17T22:05:18.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 -- start start 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb2e409ba00 0x7fb2e409be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e409c350 con 0x7fb2e40b88b0 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e409c710 con 0x7fb2e4071400 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2ea0d3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e40d1a40 con 0x7fb2e409ba00 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb2e40b88b0 0x7fb2e40d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36558/0 (socket says 172.21.15.83:36558) 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e37fe640 1 -- 172.21.15.83:0/3984218299 learned_addr learned my addr 172.21.15.83:0/3984218299 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:18.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e2ffd640 1 --2- 172.21.15.83:0/3984218299 >> v2:172.21.15.83:3300/0 conn(0x7fb2e409ba00 0x7fb2e409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e2ffd640 1 -- 172.21.15.83:0/3984218299 >> v1:172.21.15.155:6789/0 conn(0x7fb2e4071400 legacy=0x7fb2e4071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e27fc640 1 -- 172.21.15.83:0/3984218299 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 921327234 0 0) 0x7fb2d4001120 con 0x7fb2e40b88b0 2024-05-17T22:05:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e2ffd640 1 -- 172.21.15.83:0/3984218299 >> v1:172.21.15.83:6789/0 conn(0x7fb2e40b88b0 legacy=0x7fb2e40d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.480+0000 7fb2e2ffd640 1 -- 172.21.15.83:0/3984218299 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e409c9f0 con 0x7fb2e409ba00 2024-05-17T22:05:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2e2ffd640 1 --2- 172.21.15.83:0/3984218299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e409ba00 0x7fb2e409be10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d90 tx=0x7fb2cc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcc2d80984f911a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2e27fc640 1 -- 172.21.15.83:0/3984218299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0b2070 con 0x7fb2e409ba00 2024-05-17T22:05:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2e27fc640 1 -- 172.21.15.83:0/3984218299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc0a5820 con 0x7fb2e409ba00 2024-05-17T22:05:18.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2e27fc640 1 -- 172.21.15.83:0/3984218299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4072730 con 0x7fb2e409ba00 2024-05-17T22:05:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/3984218299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e409ba00 msgr2=0x7fb2e409be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/3984218299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e409ba00 0x7fb2e409be10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009d90 tx=0x7fb2cc01c850 comp rx=0 tx=0).stop 2024-05-17T22:05:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/3984218299 shutdown_connections 2024-05-17T22:05:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/3984218299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e409ba00 0x7fb2e409be10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/3984218299 >> 172.21.15.83:0/3984218299 conn(0x7fb2e40d3620 msgr2=0x7fb2e40d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/3984218299 shutdown_connections 2024-05-17T22:05:18.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.484+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/3984218299 wait complete. 2024-05-17T22:05:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 Processor -- start 2024-05-17T22:05:18.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- start start 2024-05-17T22:05:18.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 0x7fb2e40c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 0x7fb2e40b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e416b910 con 0x7fb2e40b88b0 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e409d040 con 0x7fb2e4071400 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e416ba90 con 0x7fb2e409ba00 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56616/0 (socket says 172.21.15.83:56616) 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 -- 172.21.15.83:0/1001585633 learned_addr learned my addr 172.21.15.83:0/1001585633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e37fe640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 0x7fb2e40c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e2ffd640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 0x7fb2e40b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 -- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 msgr2=0x7fb2e40b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 0x7fb2e40b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 -- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 msgr2=0x7fb2e40c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 0x7fb2e40c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 -- 172.21.15.83:0/1001585633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e40b4ad0 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e37fe640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 0x7fb2e40c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e2ffd640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 0x7fb2e40b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2e3fff640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c3c0 tx=0x7fb2dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc017070 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0052d0 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e40b32b0 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc003800 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e40b3870 con 0x7fb2e40b88b0 2024-05-17T22:05:18.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.488+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e40b1c40 con 0x7fb2e40b88b0 2024-05-17T22:05:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.492+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb2dc01a070 con 0x7fb2e40b88b0 2024-05-17T22:05:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.492+0000 7fb2dbfff640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 0x7fb2c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.492+0000 7fb2e37fe640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 0x7fb2c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.492+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fb2dc08e1f0 con 0x7fb2e40b88b0 2024-05-17T22:05:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.492+0000 7fb2e37fe640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 0x7fb2c4071940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000f70 tx=0x7fb2d4007dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.496+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb2e40b4ad0 con 0x7fb2e40b88b0 2024-05-17T22:05:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.664+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb2e4071a20 con 0x7fb2e40b88b0 2024-05-17T22:05:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.668+0000 7fb2dbfff640 1 -- 172.21.15.83:0/1001585633 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fb2dc005480 con 0x7fb2e40b88b0 2024-05-17T22:05:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 msgr2=0x7fb2c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 0x7fb2c4071940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000f70 tx=0x7fb2d4007dd0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 msgr2=0x7fb2e40b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c3c0 tx=0x7fb2dc00c880 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 shutdown_connections 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2e40b88b0 0x7fb2e40b4590 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e409ba00 0x7fb2e40b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb2c406f430 0x7fb2c4071940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 --2- 172.21.15.83:0/1001585633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4071400 0x7fb2e40c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 >> 172.21.15.83:0/1001585633 conn(0x7fb2e40d3620 msgr2=0x7fb2e4072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 shutdown_connections 2024-05-17T22:05:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.672+0000 7fb2ea0d3640 1 -- 172.21.15.83:0/1001585633 wait complete. 2024-05-17T22:05:18.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-17T22:05:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 Processor -- start 2024-05-17T22:05:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 -- start start 2024-05-17T22:05:18.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5cac1461c0 0x7f5cac15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cac0706e0 0x7f5cac1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cac1465d0 con 0x7f5cac15e6b0 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5cac06ef30 con 0x7f5cac0706e0 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb31d4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f5cac06f210 con 0x7f5cac1461c0 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cac0706e0 0x7f5cac1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cac0706e0 0x7f5cac1605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56632/0 (socket says 172.21.15.83:56632) 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 -- 172.21.15.83:0/2505266171 learned_addr learned my addr 172.21.15.83:0/2505266171 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 -- 172.21.15.83:0/2505266171 >> v2:172.21.15.155:3300/0 conn(0x7f5cac1461c0 msgr2=0x7f5cac15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb0f70640 1 --2- 172.21.15.83:0/2505266171 >> v2:172.21.15.155:3300/0 conn(0x7f5cac1461c0 0x7f5cac15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 --2- 172.21.15.83:0/2505266171 >> v2:172.21.15.155:3300/0 conn(0x7f5cac1461c0 0x7f5cac15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 -- 172.21.15.83:0/2505266171 >> v1:172.21.15.155:6789/0 conn(0x7f5cac15e6b0 legacy=0x7f5cac06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.844+0000 7f5cb1771640 1 -- 172.21.15.83:0/2505266171 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac06f4f0 con 0x7f5cac0706e0 2024-05-17T22:05:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb1771640 1 --2- 172.21.15.83:0/2505266171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac0706e0 0x7f5cac1605a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06d5a0 tx=0x7f5ca401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2570fd34a43bb861 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5ca37fe640 1 -- 172.21.15.83:0/2505266171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40a5750 con 0x7f5cac0706e0 2024-05-17T22:05:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5ca37fe640 1 -- 172.21.15.83:0/2505266171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca40a58f0 con 0x7f5cac0706e0 2024-05-17T22:05:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5ca37fe640 1 -- 172.21.15.83:0/2505266171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40a5cc0 con 0x7f5cac0706e0 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 -- 172.21.15.83:0/2505266171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac0706e0 msgr2=0x7f5cac1605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/2505266171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac0706e0 0x7f5cac1605a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cac06d5a0 tx=0x7f5ca401c9c0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 -- 172.21.15.83:0/2505266171 shutdown_connections 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/2505266171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac0706e0 0x7f5cac1605a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/2505266171 >> v2:172.21.15.155:3300/0 conn(0x7f5cac1461c0 0x7f5cac15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 -- 172.21.15.83:0/2505266171 >> 172.21.15.83:0/2505266171 conn(0x7f5cac06cd90 msgr2=0x7f5cac0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 -- 172.21.15.83:0/2505266171 shutdown_connections 2024-05-17T22:05:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.848+0000 7f5cb31d4640 1 -- 172.21.15.83:0/2505266171 wait complete. 2024-05-17T22:05:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 Processor -- start 2024-05-17T22:05:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- start start 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cac0706e0 0x7f5cac0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cac1461c0 0x7f5cac15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac169940 con 0x7f5cac15e6b0 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac0d12b0 con 0x7f5cac1461c0 2024-05-17T22:05:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac169ac0 con 0x7f5cac0706e0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56646/0 (socket says 172.21.15.83:56646) 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 -- 172.21.15.83:0/3842149493 learned_addr learned my addr 172.21.15.83:0/3842149493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb0f70640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cac0706e0 0x7f5cac0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5ca3fff640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cac1461c0 0x7f5cac15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 -- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cac0706e0 msgr2=0x7f5cac0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cac0706e0 0x7f5cac0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 -- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cac1461c0 msgr2=0x7f5cac15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cac1461c0 0x7f5cac15aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 -- 172.21.15.83:0/3842149493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac15b050 con 0x7f5cac15e6b0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb1771640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40af040 tx=0x7f5ca4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40b8670 con 0x7f5cac15e6b0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca40be070 con 0x7f5cac15e6b0 2024-05-17T22:05:18.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac15b2f0 con 0x7f5cac15e6b0 2024-05-17T22:05:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40b8a50 con 0x7f5cac15e6b0 2024-05-17T22:05:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac15b8b0 con 0x7f5cac15e6b0 2024-05-17T22:05:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.852+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac1605a0 con 0x7f5cac15e6b0 2024-05-17T22:05:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.856+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ca40b8bf0 con 0x7f5cac15e6b0 2024-05-17T22:05:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.856+0000 7f5ca17fa640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 0x7f5c98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.856+0000 7f5cb0f70640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 0x7f5c98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.856+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f5ca412a360 con 0x7f5cac15e6b0 2024-05-17T22:05:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.856+0000 7f5cb0f70640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 0x7f5c98071910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0b11f0 tx=0x7f5ca8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:18.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:18.860+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5ca40165e0 con 0x7f5cac15e6b0 2024-05-17T22:05:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.084+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f5cac0b41a0 con 0x7f5cac15e6b0 2024-05-17T22:05:19.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.308+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3842149493 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v358) ==== 167+0+0 (secure 0 0 0) 0x7f5ca40f74a0 con 0x7f5cac15e6b0 2024-05-17T22:05:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 msgr2=0x7f5c98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 0x7f5c98071910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0b11f0 tx=0x7f5ca8008040 comp rx=0 tx=0).stop 2024-05-17T22:05:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 msgr2=0x7f5cac1412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40af040 tx=0x7f5ca4019760 comp rx=0 tx=0).stop 2024-05-17T22:05:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 shutdown_connections 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cac15e6b0 0x7f5cac1412c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5c9806f400 0x7f5c98071910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cac1461c0 0x7f5cac15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 --2- 172.21.15.83:0/3842149493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5cac0706e0 0x7f5cac0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 >> 172.21.15.83:0/3842149493 conn(0x7f5cac06cd90 msgr2=0x7f5cac13cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 shutdown_connections 2024-05-17T22:05:19.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.312+0000 7f5cb31d4640 1 -- 172.21.15.83:0/3842149493 wait complete. 2024-05-17T22:05:19.326 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-17T22:05:19.327 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T22:05:19.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 Processor -- start 2024-05-17T22:05:19.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 -- start start 2024-05-17T22:05:19.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 0x7fa9100cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91006f7e0 con 0x7fa9100ced00 2024-05-17T22:05:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa9100af780 con 0x7fa9100b7ce0 2024-05-17T22:05:19.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa915b95640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100af450 con 0x7fa9100cd740 2024-05-17T22:05:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 0x7fa9100cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 0x7fa9100cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:41876/0 (socket says 172.21.15.83:41876) 2024-05-17T22:05:19.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90f7fe640 1 -- 172.21.15.83:0/1016502698 learned_addr learned my addr 172.21.15.83:0/1016502698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:33736/0 (socket says 172.21.15.83:33736) 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.488+0000 7fa90ffff640 1 --1- 172.21.15.83:0/1016502698 >> v1:172.21.15.155:6789/0 conn(0x7fa9100ced00 0x7fa9100cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:42222/0 (socket says 172.21.15.83:42222) 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90effd640 1 -- 172.21.15.83:0/1016502698 >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 msgr2=0x7fa9100cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90effd640 1 --2- 172.21.15.83:0/1016502698 >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 0x7fa9100cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90effd640 1 -- 172.21.15.83:0/1016502698 >> v1:172.21.15.155:6789/0 conn(0x7fa9100ced00 legacy=0x7fa9100cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90effd640 1 -- 172.21.15.83:0/1016502698 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100d0f40 con 0x7fa9100cd740 2024-05-17T22:05:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90effd640 1 --2- 172.21.15.83:0/1016502698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9000097c0 tx=0x7fa90001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae68618f541b6212 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90e7fc640 1 -- 172.21.15.83:0/1016502698 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000b5070 con 0x7fa9100cd740 2024-05-17T22:05:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90e7fc640 1 -- 172.21.15.83:0/1016502698 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9000a8990 con 0x7fa9100cd740 2024-05-17T22:05:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.492+0000 7fa90e7fc640 1 -- 172.21.15.83:0/1016502698 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000a8d60 con 0x7fa9100cd740 2024-05-17T22:05:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- 172.21.15.83:0/1016502698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 msgr2=0x7fa9100cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- 172.21.15.83:0/1016502698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9000097c0 tx=0x7fa90001ec90 comp rx=0 tx=0).stop 2024-05-17T22:05:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- 172.21.15.83:0/1016502698 shutdown_connections 2024-05-17T22:05:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- 172.21.15.83:0/1016502698 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100cd740 0x7fa9100cdb50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- 172.21.15.83:0/1016502698 >> v2:172.21.15.155:3300/0 conn(0x7fa9100b7ce0 0x7fa9100cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- 172.21.15.83:0/1016502698 >> 172.21.15.83:0/1016502698 conn(0x7fa9100b4170 msgr2=0x7fa9100b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- 172.21.15.83:0/1016502698 shutdown_connections 2024-05-17T22:05:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- 172.21.15.83:0/1016502698 wait complete. 2024-05-17T22:05:19.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 Processor -- start 2024-05-17T22:05:19.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- start start 2024-05-17T22:05:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100b7ce0 0x7fa91014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9100cd740 0x7fa9100c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100b80f0 con 0x7fa9100ced00 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d1500 con 0x7fa9100cd740 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa915b95640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100ce090 con 0x7fa9100b7ce0 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56654/0 (socket says 172.21.15.83:56654) 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 -- 172.21.15.83:0/168271098 learned_addr learned my addr 172.21.15.83:0/168271098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90f7fe640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100b7ce0 0x7fa91014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90effd640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9100cd740 0x7fa9100c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 -- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100b7ce0 msgr2=0x7fa91014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100b7ce0 0x7fa91014eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 -- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9100cd740 msgr2=0x7fa9100c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9100cd740 0x7fa9100c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.496+0000 7fa90ffff640 1 -- 172.21.15.83:0/168271098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100c9e60 con 0x7fa9100ced00 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa90ffff640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c760 tx=0x7fa90800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908017070 con 0x7fa9100ced00 2024-05-17T22:05:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90800a5e0 con 0x7fa9100ced00 2024-05-17T22:05:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9100ca140 con 0x7fa9100ced00 2024-05-17T22:05:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90800a990 con 0x7fa9100ced00 2024-05-17T22:05:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9100ca700 con 0x7fa9100ced00 2024-05-17T22:05:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9100d0b60 con 0x7fa9100ced00 2024-05-17T22:05:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa90801a070 con 0x7fa9100ced00 2024-05-17T22:05:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 0x7fa8fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fa90808e4d0 con 0x7fa9100ced00 2024-05-17T22:05:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa90f7fe640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 0x7fa8fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.500+0000 7fa90f7fe640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 0x7fa8fc071940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa90400bce0 tx=0x7fa90400b450 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.504+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa90805b050 con 0x7fa9100ced00 2024-05-17T22:05:19.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.676+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9100c9b60 con 0x7fa9100ced00 2024-05-17T22:05:19.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.676+0000 7fa8f7fff640 1 -- 172.21.15.83:0/168271098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+13255 (secure 0 0 0) 0x7fa90805f350 con 0x7fa9100ced00 2024-05-17T22:05:19.668 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:19.668 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:05:18.292741+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6834/2140054867":"2024-05-18T22:05:14.936239+0000","172.21.15.83:6836/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6834/165849096":"2024-05-18T22:04:31.611283+0000","172.21.15.83:6836/2140054867":"2024-05-18T22:05:14.936239+0000","172.21.15.83:6836/3213750564":"2024-05-18T22:03:12.435992+0000","172.21.15.83:6834/3213750564":"2024-05-18T22:03:12.435992+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 msgr2=0x7fa8fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 0x7fa8fc071940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa90400bce0 tx=0x7fa90400b450 comp rx=0 tx=0).stop 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 msgr2=0x7fa9101f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c760 tx=0x7fa90800cc20 comp rx=0 tx=0).stop 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 shutdown_connections 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9100ced00 0x7fa9101f1730 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa8fc06f430 0x7fa8fc071940 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9100cd740 0x7fa9100c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 --2- 172.21.15.83:0/168271098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9100b7ce0 0x7fa91014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 >> 172.21.15.83:0/168271098 conn(0x7fa9100b4170 msgr2=0x7fa910070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:19.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 shutdown_connections 2024-05-17T22:05:19.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.680+0000 7fa915b95640 1 -- 172.21.15.83:0/168271098 wait complete. 2024-05-17T22:05:19.690 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T22:05:19.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:05:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 Processor -- start 2024-05-17T22:05:19.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 -- start start 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f76c40b4810 0x7f76c40d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76c40d2000 0x7f76c40d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c40d4170 con 0x7f76c40d0290 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f76c40d4450 con 0x7f76c40d2000 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76cb1fc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f76c40d4730 con 0x7f76c40b4810 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c9799640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76c40d2000 0x7f76c40d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f76c40d0290 0x7f76c415fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36560/0 (socket says 172.21.15.83:36560) 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c3fff640 1 -- 172.21.15.83:0/25221213 learned_addr learned my addr 172.21.15.83:0/25221213 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c8f98640 1 --2- 172.21.15.83:0/25221213 >> v2:172.21.15.155:3300/0 conn(0x7f76c40b4810 0x7f76c40d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c9799640 1 -- 172.21.15.83:0/25221213 >> v2:172.21.15.155:3300/0 conn(0x7f76c40b4810 msgr2=0x7f76c40d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c9799640 1 --2- 172.21.15.83:0/25221213 >> v2:172.21.15.155:3300/0 conn(0x7f76c40b4810 0x7f76c40d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c9799640 1 -- 172.21.15.83:0/25221213 >> v1:172.21.15.83:6789/0 conn(0x7f76c40d0290 legacy=0x7f76c415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c9799640 1 -- 172.21.15.83:0/25221213 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c40d4a10 con 0x7f76c40d2000 2024-05-17T22:05:19.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.848+0000 7f76c37fe640 1 -- 172.21.15.83:0/25221213 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2843179287 0 0) 0x7f76c40d4170 con 0x7f76c40d0290 2024-05-17T22:05:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76c9799640 1 --2- 172.21.15.83:0/25221213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d2000 0x7f76c40d2410 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f76c408e620 tx=0x7f76bc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b48890a813d05f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76c37fe640 1 -- 172.21.15.83:0/25221213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0b2070 con 0x7f76c40d2000 2024-05-17T22:05:19.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76c37fe640 1 -- 172.21.15.83:0/25221213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76bc0a5a10 con 0x7f76c40d2000 2024-05-17T22:05:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76c37fe640 1 -- 172.21.15.83:0/25221213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0a5de0 con 0x7f76c40d2000 2024-05-17T22:05:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 -- 172.21.15.83:0/25221213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d2000 msgr2=0x7f76c40d2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/25221213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d2000 0x7f76c40d2410 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f76c408e620 tx=0x7f76bc01c930 comp rx=0 tx=0).stop 2024-05-17T22:05:19.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 -- 172.21.15.83:0/25221213 shutdown_connections 2024-05-17T22:05:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/25221213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d2000 0x7f76c40d2410 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/25221213 >> v2:172.21.15.155:3300/0 conn(0x7f76c40b4810 0x7f76c40d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 -- 172.21.15.83:0/25221213 >> 172.21.15.83:0/25221213 conn(0x7f76c406e010 msgr2=0x7f76c406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 -- 172.21.15.83:0/25221213 shutdown_connections 2024-05-17T22:05:19.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.852+0000 7f76cb1fc640 1 -- 172.21.15.83:0/25221213 wait complete. 2024-05-17T22:05:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 Processor -- start 2024-05-17T22:05:19.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- start start 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76c40b4810 0x7f76c414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76c40d2000 0x7f76c40b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76c40b9670 con 0x7f76c40d0290 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f76c40b94f0 con 0x7f76c40b4810 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f76c40b97f0 con 0x7f76c40d2000 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56670/0 (socket says 172.21.15.83:56670) 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 -- 172.21.15.83:0/2092865974 learned_addr learned my addr 172.21.15.83:0/2092865974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 -- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76c40d2000 msgr2=0x7f76c40b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76c40d2000 0x7f76c40b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 -- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76c40b4810 msgr2=0x7f76c414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:19.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76c40b4810 0x7f76c414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 -- 172.21.15.83:0/2092865974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c41575a0 con 0x7f76c40d0290 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c3fff640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c3f0 tx=0x7f76b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b80074e0 con 0x7f76c40d0290 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b80047f0 con 0x7f76c40d0290 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c415e220 con 0x7f76c40d0290 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8007680 con 0x7f76c40d0290 2024-05-17T22:05:19.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.856+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c415c2d0 con 0x7f76c40d0290 2024-05-17T22:05:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c4157310 con 0x7f76c40d0290 2024-05-17T22:05:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76b8011070 con 0x7f76c40d0290 2024-05-17T22:05:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76c17fa640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 0x7f76b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:19.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f76b808f1b0 con 0x7f76c40d0290 2024-05-17T22:05:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76c8f98640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 0x7f76b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:19.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.860+0000 7f76c8f98640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 0x7f76b0071910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f76b400b850 tx=0x7f76b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:19.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:19.864+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f76b805c3d0 con 0x7f76c40d0290 2024-05-17T22:05:20.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.012+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76c40d5210 con 0x7f76c40d0290 2024-05-17T22:05:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.012+0000 7f76c17fa640 1 -- 172.21.15.83:0/2092865974 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f76b805bcd0 con 0x7f76c40d0290 2024-05-17T22:05:20.004 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 msgr2=0x7f76b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 0x7f76b0071910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f76b400b850 tx=0x7f76b4008040 comp rx=0 tx=0).stop 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 msgr2=0x7f76c4156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f76b800c3f0 tx=0x7f76b800c8b0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 shutdown_connections 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76c40d2000 0x7f76c40b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c40d0290 0x7f76c4156a30 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f76b006f400 0x7f76b0071910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 --2- 172.21.15.83:0/2092865974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76c40b4810 0x7f76c414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 >> 172.21.15.83:0/2092865974 conn(0x7f76c406e010 msgr2=0x7f76c406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 shutdown_connections 2024-05-17T22:05:20.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.016+0000 7f76cb1fc640 1 -- 172.21.15.83:0/2092865974 wait complete. 2024-05-17T22:05:20.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-17T22:05:20.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 Processor -- start 2024-05-17T22:05:20.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 -- start start 2024-05-17T22:05:20.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a601461c0 0x7f9a6015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9a6015e6b0 0x7f9a6006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a60070510 0x7f9a601605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9a601465d0 con 0x7f9a60070510 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9a60070920 con 0x7f9a6015e6b0 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a65984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6006ef30 con 0x7f9a601461c0 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a60070510 0x7f9a601605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a60070510 0x7f9a601605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56672/0 (socket says 172.21.15.83:56672) 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/2705019653 learned_addr learned my addr 172.21.15.83:0/2705019653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5e7fc640 1 --2- 172.21.15.83:0/2705019653 >> v2:172.21.15.155:3300/0 conn(0x7f9a6015e6b0 0x7f9a6006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5effd640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a601461c0 0x7f9a6015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/2705019653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a601461c0 msgr2=0x7f9a6015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:20.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a601461c0 0x7f9a6015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/2705019653 >> v2:172.21.15.155:3300/0 conn(0x7f9a6015e6b0 msgr2=0x7f9a6006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/2705019653 >> v2:172.21.15.155:3300/0 conn(0x7f9a6015e6b0 0x7f9a6006ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.188+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/2705019653 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6006f210 con 0x7f9a60070510 2024-05-17T22:05:20.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a60070510 0x7f9a601605a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9a60070f80 tx=0x7f9a5801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a850fc5b188e3d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:20.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a5dffb640 1 -- 172.21.15.83:0/2705019653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580a5750 con 0x7f9a60070510 2024-05-17T22:05:20.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a5dffb640 1 -- 172.21.15.83:0/2705019653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a580a58f0 con 0x7f9a60070510 2024-05-17T22:05:20.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a5dffb640 1 -- 172.21.15.83:0/2705019653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580a5cc0 con 0x7f9a60070510 2024-05-17T22:05:20.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 -- 172.21.15.83:0/2705019653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a60070510 msgr2=0x7f9a601605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:20.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a60070510 0x7f9a601605a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9a60070f80 tx=0x7f9a5801c9c0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 -- 172.21.15.83:0/2705019653 shutdown_connections 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a60070510 0x7f9a601605a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 --2- 172.21.15.83:0/2705019653 >> v2:172.21.15.155:3300/0 conn(0x7f9a6015e6b0 0x7f9a6006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 --2- 172.21.15.83:0/2705019653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a601461c0 0x7f9a6015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 -- 172.21.15.83:0/2705019653 >> 172.21.15.83:0/2705019653 conn(0x7f9a6006cd90 msgr2=0x7f9a600b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 -- 172.21.15.83:0/2705019653 shutdown_connections 2024-05-17T22:05:20.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.192+0000 7f9a65984640 1 -- 172.21.15.83:0/2705019653 wait complete. 2024-05-17T22:05:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 Processor -- start 2024-05-17T22:05:20.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- start start 2024-05-17T22:05:20.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a60070510 0x7f9a6015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a601461c0 0x7f9a60162980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600d1220 con 0x7f9a6015e6b0 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600d10a0 con 0x7f9a601461c0 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9a600d13a0 con 0x7f9a60070510 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56688/0 (socket says 172.21.15.83:56688) 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/4161289305 learned_addr learned my addr 172.21.15.83:0/4161289305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5e7fc640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a601461c0 0x7f9a60162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5effd640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a60070510 0x7f9a6015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a60070510 msgr2=0x7f9a6015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a60070510 0x7f9a6015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a601461c0 msgr2=0x7f9a60162980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:20.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a601461c0 0x7f9a60162980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 -- 172.21.15.83:0/4161289305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a60140bf0 con 0x7f9a6015e6b0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a5f7fe640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af040 tx=0x7f9a58019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580bc020 con 0x7f9a6015e6b0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a580b8670 con 0x7f9a6015e6b0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a580b8a60 con 0x7f9a6015e6b0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a601636f0 con 0x7f9a6015e6b0 2024-05-17T22:05:20.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a60141190 con 0x7f9a6015e6b0 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.196+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a6015e170 con 0x7f9a6015e6b0 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.200+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9a580bf020 con 0x7f9a6015e6b0 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.200+0000 7f9a47fff640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 0x7f9a4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.200+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f9a58129a20 con 0x7f9a6015e6b0 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.200+0000 7f9a5effd640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 0x7f9a4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:20.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.200+0000 7f9a5effd640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 0x7f9a4c071940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b11f0 tx=0x7f9a50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:20.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.204+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9a580c7020 con 0x7f9a6015e6b0 2024-05-17T22:05:20.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:20.420+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f9a60073d20 con 0x7f9a6015e6b0 2024-05-17T22:05:21.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.324+0000 7f9a47fff640 1 -- 172.21.15.83:0/4161289305 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v360) ==== 149+0+0 (secure 0 0 0) 0x7f9a580165e0 con 0x7f9a6015e6b0 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 msgr2=0x7f9a4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 0x7f9a4c071940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b11f0 tx=0x7f9a50008040 comp rx=0 tx=0).stop 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 msgr2=0x7f9a60072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af040 tx=0x7f9a58019760 comp rx=0 tx=0).stop 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 shutdown_connections 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a6015e6b0 0x7f9a60072e30 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9a4c06f430 0x7f9a4c071940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a601461c0 0x7f9a60162980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 --2- 172.21.15.83:0/4161289305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a60070510 0x7f9a6015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 >> 172.21.15.83:0/4161289305 conn(0x7f9a6006cd90 msgr2=0x7f9a60141e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 shutdown_connections 2024-05-17T22:05:21.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.328+0000 7f9a65984640 1 -- 172.21.15.83:0/4161289305 wait complete. 2024-05-17T22:05:21.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T22:05:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.508+0000 7f6cbff2c640 1 Processor -- start 2024-05-17T22:05:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.508+0000 7f6cbff2c640 1 -- start start 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbff2c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6cb80cf830 0x7f6cb80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbff2c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6cb815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbff2c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb80b5640 con 0x7f6cb815d530 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbff2c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb80b58e0 con 0x7f6cb80cf830 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbff2c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb806f060 con 0x7f6cb80cdf40 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbe4c9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6cb815d530 0x7f6cb806f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36562/0 (socket says 172.21.15.83:36562) 2024-05-17T22:05:21.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/1991595189 learned_addr learned my addr 172.21.15.83:0/1991595189 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/1991595189 >> v2:172.21.15.155:3300/0 conn(0x7f6cb80cf830 0x7f6cb80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbd4c7640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6cb815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602935866 0 0) 0x7f6cb80b5640 con 0x7f6cb815d530 2024-05-17T22:05:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ca4003530 con 0x7f6cb815d530 2024-05-17T22:05:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1622470193 0 0) 0x7f6ca4003530 con 0x7f6cb815d530 2024-05-17T22:05:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 msgr2=0x7f6cb815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6cb815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 >> v2:172.21.15.155:3300/0 conn(0x7f6cb80cf830 msgr2=0x7f6cb80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 --2- 172.21.15.83:0/1991595189 >> v2:172.21.15.155:3300/0 conn(0x7f6cb80cf830 0x7f6cb80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb806f340 con 0x7f6cb815d530 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f6cb0001aa0 con 0x7f6cb815d530 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 >> v1:172.21.15.83:6789/0 conn(0x7f6cb815d530 legacy=0x7f6cb806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4003a10 0x7f6ca4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6ca4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4004d50 0x7f6ca4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80b58e0 con 0x7f6ca4003a10 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb806f060 con 0x7f6ca4004d50 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca400ba80 con 0x7f6cb80cdf40 2024-05-17T22:05:21.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4003a10 0x7f6ca4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbd4c7640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6ca4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbe4c9640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4004d50 0x7f6ca4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 -- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 msgr2=0x7f6ca4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb80cdf40 0x7f6ca4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 -- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4004d50 msgr2=0x7f6ca4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4004d50 0x7f6ca4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbdcc8640 1 -- 172.21.15.83:0/1991595189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca4003530 con 0x7f6ca4003a10 2024-05-17T22:05:21.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.512+0000 7f6cbe4c9640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4004d50 0x7f6ca4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4003a10 0x7f6ca4003e80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009ae0 tx=0x7f6cb401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a059242bf9da27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40b2070 con 0x7f6ca4003a10 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb40a5590 con 0x7f6ca4003a10 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbccc6640 1 -- 172.21.15.83:0/1991595189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40096b0 con 0x7f6ca4003a10 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 -- 172.21.15.83:0/1991595189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4003a10 msgr2=0x7f6ca4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/1991595189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca4003a10 0x7f6ca4003e80 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009ae0 tx=0x7f6cb401cab0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/1991595189 reap_dead start 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 -- 172.21.15.83:0/1991595189 shutdown_connections 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 -- 172.21.15.83:0/1991595189 >> 172.21.15.83:0/1991595189 conn(0x7f6cb806cd90 msgr2=0x7f6cb80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 -- 172.21.15.83:0/1991595189 shutdown_connections 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.516+0000 7f6cbff2c640 1 -- 172.21.15.83:0/1991595189 wait complete. 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 Processor -- start 2024-05-17T22:05:21.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- start start 2024-05-17T22:05:21.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 0x7f6cb8154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb8155960 0x7f6cb81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8167770 con 0x7f6cb8154ec0 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81675f0 con 0x7f6ca4003a10 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb81678f0 con 0x7f6cb8155960 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56710/0 (socket says 172.21.15.83:56710) 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/4198786771 learned_addr learned my addr 172.21.15.83:0/4198786771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbd4c7640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb8155960 0x7f6cb81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 0x7f6cb8154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb8155960 msgr2=0x7f6cb81571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb8155960 0x7f6cb81571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 msgr2=0x7f6cb8154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 0x7f6cb8154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 -- 172.21.15.83:0/4198786771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8157730 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 0x7f6cb8154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbe4c9640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6cb815a970 tx=0x7f6cb000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0004d70 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb000bb20 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb815fc50 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0007070 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb80b1340 con 0x7f6cb8154ec0 2024-05-17T22:05:21.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.520+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb8157900 con 0x7f6cb8154ec0 2024-05-17T22:05:21.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.524+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6cb001b020 con 0x7f6cb8154ec0 2024-05-17T22:05:21.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.524+0000 7f6cae7fc640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 0x7f6c9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:21.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.524+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 0x7f6c9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:21.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.524+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f6cb008fa20 con 0x7f6cb8154ec0 2024-05-17T22:05:21.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.524+0000 7f6cbdcc8640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 0x7f6c9c071940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009ae0 tx=0x7f6cb40046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:21.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.528+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6cb0022070 con 0x7f6cb8154ec0 2024-05-17T22:05:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.684+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6cb8154ce0 con 0x7f6cb8154ec0 2024-05-17T22:05:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cae7fc640 1 -- 172.21.15.83:0/4198786771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f6cb8154ce0 con 0x7f6cb8154ec0 2024-05-17T22:05:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 msgr2=0x7f6c9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 0x7f6c9c071940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009ae0 tx=0x7f6cb40046d0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 msgr2=0x7f6cb8160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6cb815a970 tx=0x7f6cb000b4a0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 shutdown_connections 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cb8154ec0 0x7f6cb8160930 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb8155960 0x7f6cb81571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6c9c06f430 0x7f6c9c071940 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 --2- 172.21.15.83:0/4198786771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ca4003a10 0x7f6cb8154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 >> 172.21.15.83:0/4198786771 conn(0x7f6cb806cd90 msgr2=0x7f6cb815a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 shutdown_connections 2024-05-17T22:05:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:21.688+0000 7f6cbff2c640 1 -- 172.21.15.83:0/4198786771 wait complete. 2024-05-17T22:05:21.701 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:05:21.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:05:21.743 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:05:21.781 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:05:21.782 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:21.782 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:21.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:05:21.790 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:05:21.790 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:21.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:05:21.829 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:21.829 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:21.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:05:21.837 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:05:21.837 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:21.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:05:21.876 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T22:05:21.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.040+0000 7fb2a3434640 1 Processor -- start 2024-05-17T22:05:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.040+0000 7fb2a3434640 1 -- start start 2024-05-17T22:05:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a3434640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb29c15ffa0 0x7fb29c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a3434640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c06e3d0 0x7fb29c0d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a3434640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29c0d34e0 con 0x7fb29c0d1150 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a3434640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb29c0d3780 con 0x7fb29c15ffa0 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a3434640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0d3a60 con 0x7fb29c06e3d0 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb29c15ffa0 0x7fb29c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb29c15ffa0 0x7fb29c0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56722/0 (socket says 172.21.15.83:56722) 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 -- 172.21.15.83:0/1984665733 learned_addr learned my addr 172.21.15.83:0/1984665733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a19d1640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c06e3d0 0x7fb29c0d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 -- 172.21.15.83:0/1984665733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c06e3d0 msgr2=0x7fb29c0d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c06e3d0 0x7fb29c0d4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 -- 172.21.15.83:0/1984665733 >> v1:172.21.15.155:6789/0 conn(0x7fb29c0d1150 legacy=0x7fb29c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.044+0000 7fb2a11d0640 1 -- 172.21.15.83:0/1984665733 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c0d3d40 con 0x7fb29c15ffa0 2024-05-17T22:05:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a11d0640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0d0c10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb29c08e380 tx=0x7fb29801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab93aede2b31f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb293fff640 1 -- 172.21.15.83:0/1984665733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980b0070 con 0x7fb29c15ffa0 2024-05-17T22:05:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb293fff640 1 -- 172.21.15.83:0/1984665733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2980ab600 con 0x7fb29c15ffa0 2024-05-17T22:05:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb293fff640 1 -- 172.21.15.83:0/1984665733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2980ab9d0 con 0x7fb29c15ffa0 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- 172.21.15.83:0/1984665733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 msgr2=0x7fb29c0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0d0c10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb29c08e380 tx=0x7fb29801c520 comp rx=0 tx=0).stop 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- 172.21.15.83:0/1984665733 shutdown_connections 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c06e3d0 0x7fb29c0d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 --2- 172.21.15.83:0/1984665733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0d0c10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- 172.21.15.83:0/1984665733 >> 172.21.15.83:0/1984665733 conn(0x7fb29c0d28f0 msgr2=0x7fb29c0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- 172.21.15.83:0/1984665733 shutdown_connections 2024-05-17T22:05:22.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- 172.21.15.83:0/1984665733 wait complete. 2024-05-17T22:05:22.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 Processor -- start 2024-05-17T22:05:22.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.048+0000 7fb2a3434640 1 -- start start 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c0d1150 0x7fb29c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c168fe0 0x7fb29c1693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0d5340 con 0x7fb29c15ffa0 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c0d51c0 con 0x7fb29c168fe0 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb29c0d54c0 con 0x7fb29c0d1150 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56724/0 (socket says 172.21.15.83:56724) 2024-05-17T22:05:22.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 -- 172.21.15.83:0/738024713 learned_addr learned my addr 172.21.15.83:0/738024713 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a11d0640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c0d1150 0x7fb29c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a19d1640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c168fe0 0x7fb29c1693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 -- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c0d1150 msgr2=0x7fb29c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c0d1150 0x7fb29c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 -- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c168fe0 msgr2=0x7fb29c1693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c168fe0 0x7fb29c1693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 -- 172.21.15.83:0/738024713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c169930 con 0x7fb29c15ffa0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a09cf640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00b410 tx=0x7fb28c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c015020 con 0x7fb29c15ffa0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c011830 con 0x7fb29c15ffa0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c160a80 con 0x7fb29c15ffa0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c011c20 con 0x7fb29c15ffa0 2024-05-17T22:05:22.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c161040 con 0x7fb29c15ffa0 2024-05-17T22:05:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c0d4260 con 0x7fb29c15ffa0 2024-05-17T22:05:22.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.052+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb28c018020 con 0x7fb29c15ffa0 2024-05-17T22:05:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.056+0000 7fb291ffb640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 0x7fb280071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.056+0000 7fb2a11d0640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 0x7fb280071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.056+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7fb28c08ef20 con 0x7fb29c15ffa0 2024-05-17T22:05:22.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.056+0000 7fb2a11d0640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 0x7fb280071910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ad000 tx=0x7fb298008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.060+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb28c01f020 con 0x7fb29c15ffa0 2024-05-17T22:05:22.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29c0bb7f0 con 0x7fb29c15ffa0 2024-05-17T22:05:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb291ffb640 1 -- 172.21.15.83:0/738024713 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3872 (secure 0 0 0) 0x7fb29c169930 con 0x7fb29c15ffa0 2024-05-17T22:05:22.247 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:22.247 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2203339192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2203339192},{"type":"v1","addr":"172.21.15.83:6837","nonce":2203339192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2643842472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2643842472},{"type":"v1","addr":"172.21.15.155:6833","nonce":2643842472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5234,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1454212441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1454212441},{"type":"v1","addr":"172.21.15.155:6835","nonce":1454212441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/520885559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":520885559},{"type":"v1","addr":"172.21.15.83:6836","nonce":520885559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[]} 2024-05-17T22:05:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 74 2024-05-17T22:05:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 msgr2=0x7fb280071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 0x7fb280071910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ad000 tx=0x7fb298008040 comp rx=0 tx=0).stop 2024-05-17T22:05:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 msgr2=0x7fb29c0cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00b410 tx=0x7fb28c00b8d0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 shutdown_connections 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb28006f400 0x7fb280071910 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c168fe0 0x7fb29c1693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb29c15ffa0 0x7fb29c0cf1c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.256+0000 7fb2a3434640 1 --2- 172.21.15.83:0/738024713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c0d1150 0x7fb29c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.260+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 >> 172.21.15.83:0/738024713 conn(0x7fb29c0d28f0 msgr2=0x7fb29c0b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.260+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 shutdown_connections 2024-05-17T22:05:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.260+0000 7fb2a3434640 1 -- 172.21.15.83:0/738024713 wait complete. 2024-05-17T22:05:22.274 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T22:05:22.274 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T22:05:22.274 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T22:05:22.274 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:22.274 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T22:05:22.274 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T22:05:22.275 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T22:05:22.275 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:22.275 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T22:05:22.275 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T22:05:22.275 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T22:05:22.275 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:22.275 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T22:05:22.275 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T22:05:22.276 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T22:05:22.276 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:22.291 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T22:05:22.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T22:05:22.292 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T22:05:22.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T22:05:22.294 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T22:05:22.294 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T22:05:22.296 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T22:05:22.296 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T22:05:22.335 INFO:tasks.ceph.mds.c:Started 2024-05-17T22:05:22.335 DEBUG:teuthology.parallel:result is None 2024-05-17T22:05:22.335 INFO:tasks.ceph.mds.a:Started 2024-05-17T22:05:22.336 DEBUG:teuthology.parallel:result is None 2024-05-17T22:05:22.340 INFO:tasks.ceph.mds.d:Started 2024-05-17T22:05:22.341 DEBUG:teuthology.parallel:result is None 2024-05-17T22:05:22.342 INFO:tasks.ceph.mds.b:Started 2024-05-17T22:05:22.342 DEBUG:teuthology.parallel:result is None 2024-05-17T22:05:22.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T22:05:22.424 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:05:22.424 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:05:22.437 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T22:05:22.444 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T22:05:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 Processor -- start 2024-05-17T22:05:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 -- start start 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b500cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b500ce450 con 0x7f2b500b3be0 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b500ce730 con 0x7f2b500b7d90 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b54d04640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b500cea10 con 0x7f2b500b4bb0 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2b500b3be0 0x7f2b500af370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36590/0 (socket says 172.21.15.83:36590) 2024-05-17T22:05:22.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 -- 172.21.15.83:0/3119272340 learned_addr learned my addr 172.21.15.83:0/3119272340 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b500cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110795842 0 0) 0x7f2b500ce450 con 0x7f2b500b3be0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b3c003530 con 0x7f2b500b3be0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977513118 0 0) 0x7f2b500ce730 con 0x7f2b500b7d90 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b500ce450 con 0x7f2b500b7d90 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462638616 0 0) 0x7f2b3c003530 con 0x7f2b500b3be0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 msgr2=0x7f2b500cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b500cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 >> v1:172.21.15.155:6789/0 conn(0x7f2b500b7d90 legacy=0x7f2b5006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b500cecf0 con 0x7f2b500b3be0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2b40001700 con 0x7f2b500b3be0 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b500cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:05:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 >> v1:172.21.15.83:6789/0 conn(0x7f2b500b3be0 legacy=0x7f2b500af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b3c003c20 0x7f2b3c004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b3c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500cea10 con 0x7f2b3c004810 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00bbe0 con 0x7f2b3c003c20 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c00bde0 con 0x7f2b500b4bb0 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b3c003c20 0x7f2b3c004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4ed9d640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b3c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 -- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 msgr2=0x7f2b3c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b500b4bb0 0x7f2b3c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 -- 172.21.15.83:0/3119272340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b3c004810 msgr2=0x7f2b3c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b4e59c640 1 -- 172.21.15.83:0/3119272340 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c003530 con 0x7f2b3c003c20 2024-05-17T22:05:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.492+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b3c004810 0x7f2b3c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b3c003c20 0x7f2b3c004070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c1c0 tx=0x7f2b4001cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6324eba78d4482dc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007cd0 con 0x7f2b3c003c20 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b400a9070 con 0x7f2b3c003c20 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b4dd9b640 1 -- 172.21.15.83:0/3119272340 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40005d20 con 0x7f2b3c003c20 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 -- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b3c003c20 msgr2=0x7f2b3c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 --2- 172.21.15.83:0/3119272340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b3c003c20 0x7f2b3c004070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c1c0 tx=0x7f2b4001cb40 comp rx=0 tx=0).stop 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b4ed9d640 1 -- 172.21.15.83:0/3119272340 reap_dead start 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 -- 172.21.15.83:0/3119272340 shutdown_connections 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 -- 172.21.15.83:0/3119272340 >> 172.21.15.83:0/3119272340 conn(0x7f2b500b4090 msgr2=0x7f2b500b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 -- 172.21.15.83:0/3119272340 shutdown_connections 2024-05-17T22:05:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.496+0000 7f2b54d04640 1 -- 172.21.15.83:0/3119272340 wait complete. 2024-05-17T22:05:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 Processor -- start 2024-05-17T22:05:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- start start 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b3c003c20 0x7f2b5009fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 0x7f2b500c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500afa80 con 0x7f2b500c5cc0 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500ce040 con 0x7f2b500c7d00 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f2b500c63c0 con 0x7f2b3c003c20 2024-05-17T22:05:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56778/0 (socket says 172.21.15.83:56778) 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 -- 172.21.15.83:0/2556617899 learned_addr learned my addr 172.21.15.83:0/2556617899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b4ed9d640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 0x7f2b500c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b3c003c20 0x7f2b5009fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 -- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b3c003c20 msgr2=0x7f2b5009fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b3c003c20 0x7f2b5009fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 -- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 msgr2=0x7f2b500c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 0x7f2b500c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 -- 172.21.15.83:0/2556617899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b500c8650 con 0x7f2b500c5cc0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b4ed9d640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 0x7f2b500c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b45d9b640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2b38011c50 tx=0x7f2b3800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800fd20 con 0x7f2b500c5cc0 2024-05-17T22:05:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38004020 con 0x7f2b500c5cc0 2024-05-17T22:05:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b501f3e80 con 0x7f2b500c5cc0 2024-05-17T22:05:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380043d0 con 0x7f2b500c5cc0 2024-05-17T22:05:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b501f4440 con 0x7f2b500c5cc0 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.500+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b5006ed20 con 0x7f2b500c5cc0 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.504+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b38012040 con 0x7f2b500c5cc0 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.504+0000 7f2b477fe640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 0x7f2b30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.504+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5115+0+0 (secure 0 0 0) 0x7f2b3808f910 con 0x7f2b500c5cc0 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.504+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 0x7f2b30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.504+0000 7f2b4e59c640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 0x7f2b30071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ab000 tx=0x7f2b400a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:22.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.508+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f2b500c8650 con 0x7f2b500c5cc0 2024-05-17T22:05:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:22.664+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b500af5c0 con 0x7f2b500c5cc0 2024-05-17T22:05:23.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.348+0000 7f2b477fe640 1 -- 172.21.15.83:0/2556617899 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) ==== 97+0+0 (secure 0 0 0) 0x7f2b38019030 con 0x7f2b500c5cc0 2024-05-17T22:05:23.339 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 msgr2=0x7f2b30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 0x7f2b30071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2b400ab000 tx=0x7f2b400a7070 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 msgr2=0x7f2b501528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2b38011c50 tx=0x7f2b3800f610 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 shutdown_connections 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2b3006f400 0x7f2b30071910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b500c7d00 0x7f2b500c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b500c5cc0 0x7f2b501528b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 --2- 172.21.15.83:0/2556617899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b3c003c20 0x7f2b5009fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 >> 172.21.15.83:0/2556617899 conn(0x7f2b500b4090 msgr2=0x7f2b500b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:23.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 shutdown_connections 2024-05-17T22:05:23.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.352+0000 7f2b54d04640 1 -- 172.21.15.83:0/2556617899 wait complete. 2024-05-17T22:05:23.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T22:05:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 Processor -- start 2024-05-17T22:05:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 -- start start 2024-05-17T22:05:23.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac240d1040 0x7fac24144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac24073540 0x7fac2406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac2415ea40 con 0x7fac2415e080 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fac2415ed20 con 0x7fac240d1040 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac28dc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac2415f000 con 0x7fac24073540 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac2259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac240d1040 0x7fac24144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac2259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac240d1040 0x7fac24144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56788/0 (socket says 172.21.15.83:56788) 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.528+0000 7fac2259c640 1 -- 172.21.15.83:0/3025353733 learned_addr learned my addr 172.21.15.83:0/3025353733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac22d9d640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac24073540 0x7fac2406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac2259c640 1 -- 172.21.15.83:0/3025353733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac24073540 msgr2=0x7fac2406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac2259c640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac24073540 0x7fac2406f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac2259c640 1 -- 172.21.15.83:0/3025353733 >> v1:172.21.15.155:6789/0 conn(0x7fac2415e080 legacy=0x7fac24072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac2259c640 1 -- 172.21.15.83:0/3025353733 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2406ef30 con 0x7fac240d1040 2024-05-17T22:05:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac2259c640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac240d1040 0x7fac24144080 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac1401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c1595f62a116c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac21d9b640 1 -- 172.21.15.83:0/3025353733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140a7070 con 0x7fac240d1040 2024-05-17T22:05:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac21d9b640 1 -- 172.21.15.83:0/3025353733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac140b0070 con 0x7fac240d1040 2024-05-17T22:05:23.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac21d9b640 1 -- 172.21.15.83:0/3025353733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140ab890 con 0x7fac240d1040 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 -- 172.21.15.83:0/3025353733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac240d1040 msgr2=0x7fac24144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac240d1040 0x7fac24144080 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac1401c560 comp rx=0 tx=0).stop 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 -- 172.21.15.83:0/3025353733 shutdown_connections 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac24073540 0x7fac2406f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 --2- 172.21.15.83:0/3025353733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac240d1040 0x7fac24144080 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.532+0000 7fac28dc2640 1 -- 172.21.15.83:0/3025353733 >> 172.21.15.83:0/3025353733 conn(0x7fac2406cd90 msgr2=0x7fac240b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- 172.21.15.83:0/3025353733 shutdown_connections 2024-05-17T22:05:23.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- 172.21.15.83:0/3025353733 wait complete. 2024-05-17T22:05:23.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 Processor -- start 2024-05-17T22:05:23.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- start start 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d1040 0x7fac24145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac24145dd0 0x7fac241411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac24070810 con 0x7fac2415e080 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fac24070690 con 0x7fac24145dd0 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac28dc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fac24070990 con 0x7fac240d1040 2024-05-17T22:05:23.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56792/0 (socket says 172.21.15.83:56792) 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 -- 172.21.15.83:0/4006674025 learned_addr learned my addr 172.21.15.83:0/4006674025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac2259c640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d1040 0x7fac24145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac22d9d640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac24145dd0 0x7fac241411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 -- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d1040 msgr2=0x7fac24145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d1040 0x7fac24145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 -- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac24145dd0 msgr2=0x7fac241411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac24145dd0 0x7fac241411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 -- 172.21.15.83:0/4006674025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac241473a0 con 0x7fac2415e080 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.536+0000 7fac19d9b640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c3f0 tx=0x7fac0c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c017070 con 0x7fac2415e080 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c0052d0 con 0x7fac2415e080 2024-05-17T22:05:23.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac241431f0 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c003820 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac24144b50 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2406f730 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac0c01a070 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 0x7fac04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7fac0c08e690 con 0x7fac2415e080 2024-05-17T22:05:23.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac2259c640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 0x7fac04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.540+0000 7fac2259c640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 0x7fac04071910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac140af000 tx=0x7fac140a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.544+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fac0c021020 con 0x7fac2415e080 2024-05-17T22:05:23.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.764+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fac24143880 con 0x7fac2415e080 2024-05-17T22:05:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.764+0000 7fac1b7fe640 1 -- 172.21.15.83:0/4006674025 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fac0c005480 con 0x7fac2415e080 2024-05-17T22:05:23.755 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T22:05:23.759 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.761 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:05:23.763 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T22:05:23.764 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:05:23.765 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.766 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:05:23.767 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T22:05:23.768 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 msgr2=0x7fac04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 0x7fac04071910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac140af000 tx=0x7fac140a7000 comp rx=0 tx=0).stop 2024-05-17T22:05:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 msgr2=0x7fac24145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c3f0 tx=0x7fac0c00c8b0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 shutdown_connections 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fac0406f400 0x7fac04071910 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac24145dd0 0x7fac241411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac2415e080 0x7fac24145890 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 --2- 172.21.15.83:0/4006674025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac240d1040 0x7fac24145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 >> 172.21.15.83:0/4006674025 conn(0x7fac2406cd90 msgr2=0x7fac240b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 shutdown_connections 2024-05-17T22:05:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.768+0000 7fac28dc2640 1 -- 172.21.15.83:0/4006674025 wait complete. 2024-05-17T22:05:23.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T22:05:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 Processor -- start 2024-05-17T22:05:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 -- start start 2024-05-17T22:05:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 0x7f4194144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419415ea40 con 0x7f419415e490 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f419415ed20 con 0x7f41940d1040 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4199cc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f419415f000 con 0x7f419415e080 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f41937fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 0x7f4194144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:33904/0 (socket says 172.21.15.83:33904) 2024-05-17T22:05:23.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 -- 172.21.15.83:0/1692309956 learned_addr learned my addr 172.21.15.83:0/1692309956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:23.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 -- 172.21.15.83:0/1692309956 >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 msgr2=0x7f4194144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:23.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 --2- 172.21.15.83:0/1692309956 >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 0x7f4194144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 -- 172.21.15.83:0/1692309956 >> v1:172.21.15.155:6789/0 conn(0x7f419415e490 legacy=0x7f41941616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f4192ffd640 1 -- 172.21.15.83:0/1692309956 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419406ef30 con 0x7f419415e080 2024-05-17T22:05:23.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.940+0000 7f41937fe640 1 --2- 172.21.15.83:0/1692309956 >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 0x7f4194144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:05:23.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4192ffd640 1 --2- 172.21.15.83:0/1692309956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4188009870 tx=0x7f418801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70666e5e6817fe5b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f41927fc640 1 -- 172.21.15.83:0/1692309956 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880a7070 con 0x7f419415e080 2024-05-17T22:05:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f41927fc640 1 -- 172.21.15.83:0/1692309956 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41880b0070 con 0x7f419415e080 2024-05-17T22:05:23.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f41927fc640 1 -- 172.21.15.83:0/1692309956 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880ab890 con 0x7f419415e080 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 -- 172.21.15.83:0/1692309956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 msgr2=0x7f4194072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 --2- 172.21.15.83:0/1692309956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4188009870 tx=0x7f418801c560 comp rx=0 tx=0).stop 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 -- 172.21.15.83:0/1692309956 shutdown_connections 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 --2- 172.21.15.83:0/1692309956 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f4194072e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 --2- 172.21.15.83:0/1692309956 >> v2:172.21.15.155:3300/0 conn(0x7f41940d1040 0x7f4194144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 -- 172.21.15.83:0/1692309956 >> 172.21.15.83:0/1692309956 conn(0x7f419406cd90 msgr2=0x7f41940b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 -- 172.21.15.83:0/1692309956 shutdown_connections 2024-05-17T22:05:23.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.944+0000 7f4199cc2640 1 -- 172.21.15.83:0/1692309956 wait complete. 2024-05-17T22:05:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 Processor -- start 2024-05-17T22:05:23.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 -- start start 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 0x7f419415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f41940b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f419416abe0 con 0x7f419415e490 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f419416aa60 con 0x7f41940d1040 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4199cc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f419416ad60 con 0x7f419415e080 2024-05-17T22:05:23.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56796/0 (socket says 172.21.15.83:56796) 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.948+0000 7f4193fff640 1 -- 172.21.15.83:0/2674871187 learned_addr learned my addr 172.21.15.83:0/2674871187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4192ffd640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f41940b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f41937fe640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 0x7f419415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 -- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 msgr2=0x7f41940b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f41940b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 -- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 msgr2=0x7f419415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 0x7f419415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 -- 172.21.15.83:0/2674871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419415b390 con 0x7f419415e490 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4192ffd640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f41940b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f41937fe640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 0x7f419415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4193fff640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c3c0 tx=0x7f418c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c017070 con 0x7f419415e490 2024-05-17T22:05:23.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c0052d0 con 0x7f419415e490 2024-05-17T22:05:23.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c003800 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41940b4880 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419413ada0 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419413e370 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f418c01a070 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 0x7f4180071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7f418c08e1c0 con 0x7f419415e490 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.952+0000 7f41937fe640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 0x7f4180071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:23.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.956+0000 7f41937fe640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 0x7f4180071940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4184009fd0 tx=0x7f418400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:23.960+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f418c005480 con 0x7f419415e490 2024-05-17T22:05:24.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.120+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f419413c640 con 0x7f419415e490 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.124+0000 7f417bfff640 1 -- 172.21.15.83:0/2674871187 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) ==== 84+0+65 (secure 0 0 0) 0x7f419413c640 con 0x7f419415e490 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:24.114 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 msgr2=0x7f4180071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 0x7f4180071940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4184009fd0 tx=0x7f418400b040 comp rx=0 tx=0).stop 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 msgr2=0x7f41940b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c3c0 tx=0x7f418c00c880 comp rx=0 tx=0).stop 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 shutdown_connections 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419415e490 0x7f41940b1520 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f419415e080 0x7f41940b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f418006f430 0x7f4180071940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 --2- 172.21.15.83:0/2674871187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41940d1040 0x7f419415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 >> 172.21.15.83:0/2674871187 conn(0x7f419406cd90 msgr2=0x7f419413d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:24.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 shutdown_connections 2024-05-17T22:05:24.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.128+0000 7f4199cc2640 1 -- 172.21.15.83:0/2674871187 wait complete. 2024-05-17T22:05:24.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T22:05:24.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T22:05:24.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 Processor -- start 2024-05-17T22:05:24.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 -- start start 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb118072e50 0x7fb118073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb1180d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1180737a0 con 0x7fb118159da0 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb1180b0e90 con 0x7fb118072e50 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.304+0000 7fb11fc8a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1180cf050 con 0x7fb1180cfd90 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb118159da0 0x7fb11815cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:36606/0 (socket says 172.21.15.83:36606) 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 -- 172.21.15.83:0/3677506595 learned_addr learned my addr 172.21.15.83:0/3677506595 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11d225640 1 --2- 172.21.15.83:0/3677506595 >> v2:172.21.15.155:3300/0 conn(0x7fb118072e50 0x7fb118073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11e227640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb1180d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128854410 0 0) 0x7fb114001120 con 0x7fb118159da0 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb104003530 con 0x7fb118159da0 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1482610188 0 0) 0x7fb104003530 con 0x7fb118159da0 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 msgr2=0x7fb1180d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb1180d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 >> v2:172.21.15.155:3300/0 conn(0x7fb118072e50 msgr2=0x7fb118073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 --2- 172.21.15.83:0/3677506595 >> v2:172.21.15.155:3300/0 conn(0x7fb118072e50 0x7fb118073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1180cf2f0 con 0x7fb118159da0 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fb1140018d0 con 0x7fb118159da0 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 >> v1:172.21.15.83:6789/0 conn(0x7fb118159da0 legacy=0x7fb11815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb104003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb104004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb104004d50 0x7fb104005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1180b0e90 con 0x7fb104003a10 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1180cf050 con 0x7fb104004d50 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb10400dcc0 con 0x7fb1180cfd90 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb104003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11d225640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb104004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11e227640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb104004d50 0x7fb104005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 -- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 msgr2=0x7fb104004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1180cfd90 0x7fb104004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 -- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb104004d50 msgr2=0x7fb104005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb104004d50 0x7fb104005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.308+0000 7fb11da26640 1 -- 172.21.15.83:0/3677506595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb104003530 con 0x7fb104003a10 2024-05-17T22:05:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11da26640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb104003e80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c4c0 tx=0x7fb11401cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ece0a9056f96db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11400c0e0 con 0x7fb104003a10 2024-05-17T22:05:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1140a9070 con 0x7fb104003a10 2024-05-17T22:05:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11ca24640 1 -- 172.21.15.83:0/3677506595 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114005d20 con 0x7fb104003a10 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- 172.21.15.83:0/3677506595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 msgr2=0x7fb104003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/3677506595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb104003e80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c4c0 tx=0x7fb11401cb30 comp rx=0 tx=0).stop 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11e227640 1 -- 172.21.15.83:0/3677506595 reap_dead start 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- 172.21.15.83:0/3677506595 shutdown_connections 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- 172.21.15.83:0/3677506595 >> 172.21.15.83:0/3677506595 conn(0x7fb11806cd90 msgr2=0x7fb1180b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- 172.21.15.83:0/3677506595 shutdown_connections 2024-05-17T22:05:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- 172.21.15.83:0/3677506595 wait complete. 2024-05-17T22:05:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 Processor -- start 2024-05-17T22:05:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.312+0000 7fb11fc8a640 1 -- start start 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 0x7fb118162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 0x7fb118162f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1180b0810 con 0x7fb104003a10 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb118070520 con 0x7fb11814bea0 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb1180b0990 con 0x7fb11814b4d0 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56810/0 (socket says 172.21.15.83:56810) 2024-05-17T22:05:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 -- 172.21.15.83:0/167769853 learned_addr learned my addr 172.21.15.83:0/167769853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11d225640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 0x7fb118162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11e227640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 0x7fb118162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 -- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 msgr2=0x7fb118162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 0x7fb118162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 -- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 msgr2=0x7fb118162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 0x7fb118162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 -- 172.21.15.83:0/167769853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11815fb40 con 0x7fb104003a10 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11d225640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 0x7fb118162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11e227640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 0x7fb118162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11da26640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb1140985d0 tx=0x7fb104005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114001120 con 0x7fb104003a10 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb114005d00 con 0x7fb104003a10 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11815fe20 con 0x7fb104003a10 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114001890 con 0x7fb104003a10 2024-05-17T22:05:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1181648a0 con 0x7fb104003a10 2024-05-17T22:05:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb118073260 con 0x7fb104003a10 2024-05-17T22:05:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.316+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1140a58b0 con 0x7fb104003a10 2024-05-17T22:05:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.320+0000 7fb10e7fc640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 0x7fb0fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.320+0000 7fb11d225640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 0x7fb0fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.320+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 4857+0+0 (secure 0 0 0) 0x7fb1141284d0 con 0x7fb104003a10 2024-05-17T22:05:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.320+0000 7fb11d225640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 0x7fb0fc071910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb10800a270 tx=0x7fb108008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.324+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb1140bf060 con 0x7fb104003a10 2024-05-17T22:05:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:24.492+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb1180cd940 con 0x7fb104003a10 2024-05-17T22:05:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.352+0000 7fb10e7fc640 1 -- 172.21.15.83:0/167769853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) ==== 138+0+0 (secure 0 0 0) 0x7fb1180737a0 con 0x7fb104003a10 2024-05-17T22:05:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 msgr2=0x7fb0fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 0x7fb0fc071910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb10800a270 tx=0x7fb108008040 comp rx=0 tx=0).stop 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 msgr2=0x7fb11814af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb1140985d0 tx=0x7fb104005840 comp rx=0 tx=0).stop 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 shutdown_connections 2024-05-17T22:05:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb0fc06f400 0x7fb0fc071910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb11814bea0 0x7fb118162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb11814b4d0 0x7fb118162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 --2- 172.21.15.83:0/167769853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb104003a10 0x7fb11814af90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 >> 172.21.15.83:0/167769853 conn(0x7fb11806cd90 msgr2=0x7fb11806f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 shutdown_connections 2024-05-17T22:05:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.356+0000 7fb11fc8a640 1 -- 172.21.15.83:0/167769853 wait complete. 2024-05-17T22:05:25.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T22:05:25.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.552+0000 7fc616ee4640 1 Processor -- start 2024-05-17T22:05:25.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.552+0000 7fc616ee4640 1 -- start start 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc616ee4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 0x7fc6100b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc616ee4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6100cdf40 0x7fc61015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc616ee4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6100b5640 con 0x7fc61015d530 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc616ee4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc6100b58e0 con 0x7fc6100cdf40 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc616ee4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc61006f060 con 0x7fc6100cf830 2024-05-17T22:05:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6100cdf40 0x7fc61015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc6100cdf40 0x7fc61015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56824/0 (socket says 172.21.15.83:56824) 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 -- 172.21.15.83:0/3056076956 learned_addr learned my addr 172.21.15.83:0/3056076956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 -- 172.21.15.83:0/3056076956 >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 msgr2=0x7fc6100b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc614c80640 1 --2- 172.21.15.83:0/3056076956 >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 0x7fc6100b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 --2- 172.21.15.83:0/3056076956 >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 0x7fc6100b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 -- 172.21.15.83:0/3056076956 >> v1:172.21.15.155:6789/0 conn(0x7fc61015d530 legacy=0x7fc61006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc60ffff640 1 -- 172.21.15.83:0/3056076956 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61006f340 con 0x7fc6100cdf40 2024-05-17T22:05:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.556+0000 7fc614c80640 1 --2- 172.21.15.83:0/3056076956 >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 0x7fc6100b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc60ffff640 1 --2- 172.21.15.83:0/3056076956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cdf40 0x7fc61015cff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc6100cefe0 tx=0x7fc60401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb72a5c2d9f980d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc60f7fe640 1 -- 172.21.15.83:0/3056076956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040b1070 con 0x7fc6100cdf40 2024-05-17T22:05:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc60f7fe640 1 -- 172.21.15.83:0/3056076956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6040ac630 con 0x7fc6100cdf40 2024-05-17T22:05:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc60f7fe640 1 -- 172.21.15.83:0/3056076956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040a8050 con 0x7fc6100cdf40 2024-05-17T22:05:25.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- 172.21.15.83:0/3056076956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cdf40 msgr2=0x7fc61015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:25.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3056076956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cdf40 0x7fc61015cff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc6100cefe0 tx=0x7fc60401c500 comp rx=0 tx=0).stop 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- 172.21.15.83:0/3056076956 shutdown_connections 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3056076956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cdf40 0x7fc61015cff0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3056076956 >> v2:172.21.15.155:3300/0 conn(0x7fc6100cf830 0x7fc6100b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- 172.21.15.83:0/3056076956 >> 172.21.15.83:0/3056076956 conn(0x7fc61006cd90 msgr2=0x7fc6100b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- 172.21.15.83:0/3056076956 shutdown_connections 2024-05-17T22:05:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- 172.21.15.83:0/3056076956 wait complete. 2024-05-17T22:05:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 Processor -- start 2024-05-17T22:05:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.560+0000 7fc616ee4640 1 -- start start 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 0x7fc6101f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc61015d530 0x7fc610072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc610169780 con 0x7fc6100cf830 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc610169600 con 0x7fc6100cdf40 2024-05-17T22:05:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc610169900 con 0x7fc61015d530 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56840/0 (socket says 172.21.15.83:56840) 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 -- 172.21.15.83:0/3033797122 learned_addr learned my addr 172.21.15.83:0/3033797122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc614c80640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 0x7fc6101f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc615481640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc61015d530 0x7fc610072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 -- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc61015d530 msgr2=0x7fc610072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc61015d530 0x7fc610072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 -- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 msgr2=0x7fc6101f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 0x7fc6101f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 -- 172.21.15.83:0/3033797122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6100aae90 con 0x7fc6100cf830 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc614c80640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 0x7fc6101f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60ffff640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc6040af040 tx=0x7fc6040192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040bc070 con 0x7fc6100cf830 2024-05-17T22:05:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6040ab940 con 0x7fc6100cf830 2024-05-17T22:05:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040b1070 con 0x7fc6100cf830 2024-05-17T22:05:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc610158690 con 0x7fc6100cf830 2024-05-17T22:05:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc610158c50 con 0x7fc6100cf830 2024-05-17T22:05:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61015cff0 con 0x7fc6100cf830 2024-05-17T22:05:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc6040bf070 con 0x7fc6100cf830 2024-05-17T22:05:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc60d7fa640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 0x7fc5fc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.564+0000 7fc614c80640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 0x7fc5fc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.568+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5227+0+0 (secure 0 0 0) 0x7fc604128cc0 con 0x7fc6100cf830 2024-05-17T22:05:25.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.568+0000 7fc614c80640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 0x7fc5fc071940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc600007f40 tx=0x7fc600007490 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.572+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc6040160f0 con 0x7fc6100cf830 2024-05-17T22:05:25.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:25.732+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc610154a40 con 0x7fc6100cf830 2024-05-17T22:05:26.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.356+0000 7fc60d7fa640 1 -- 172.21.15.83:0/3033797122 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) ==== 170+0+0 (secure 0 0 0) 0x7fc6040f5f00 con 0x7fc6100cf830 2024-05-17T22:05:26.348 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T22:05:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 msgr2=0x7fc5fc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 0x7fc5fc071940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc600007f40 tx=0x7fc600007490 comp rx=0 tx=0).stop 2024-05-17T22:05:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 msgr2=0x7fc61014f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc6040af040 tx=0x7fc6040192a0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 shutdown_connections 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc61015d530 0x7fc610072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6100cf830 0x7fc61014f260 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc5fc06f430 0x7fc5fc071940 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 --2- 172.21.15.83:0/3033797122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6100cdf40 0x7fc6101f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 >> 172.21.15.83:0/3033797122 conn(0x7fc61006cd90 msgr2=0x7fc6100b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.360+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 shutdown_connections 2024-05-17T22:05:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.364+0000 7fc616ee4640 1 -- 172.21.15.83:0/3033797122 wait complete. 2024-05-17T22:05:26.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T22:05:26.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 Processor -- start 2024-05-17T22:05:26.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 -- start start 2024-05-17T22:05:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c06d170 0x7f580c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580c0ce210 0x7f580c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580c0b3a30 con 0x7f580c0cef60 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f580c0b3d10 con 0x7f580c0ce210 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f58118dd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f580c0b5610 con 0x7f580c06d170 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580c0ce210 0x7f580c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f580c0ce210 0x7f580c0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56854/0 (socket says 172.21.15.83:56854) 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 -- 172.21.15.83:0/2543473394 learned_addr learned my addr 172.21.15.83:0/2543473394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580affd640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c06d170 0x7f580c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 -- 172.21.15.83:0/2543473394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c06d170 msgr2=0x7f580c0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c06d170 0x7f580c0b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 -- 172.21.15.83:0/2543473394 >> v1:172.21.15.155:6789/0 conn(0x7f580c0cef60 legacy=0x7f580c0cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.560+0000 7f580a7fc640 1 -- 172.21.15.83:0/2543473394 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c0b58f0 con 0x7f580c0ce210 2024-05-17T22:05:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f580a7fc640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c0ce210 0x7f580c0ce620 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5800009870 tx=0x7f580001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8bd46eb820a575 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f5809ffb640 1 -- 172.21.15.83:0/2543473394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000b2070 con 0x7f580c0ce210 2024-05-17T22:05:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f5809ffb640 1 -- 172.21.15.83:0/2543473394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58000a9070 con 0x7f580c0ce210 2024-05-17T22:05:26.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f5809ffb640 1 -- 172.21.15.83:0/2543473394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000ad880 con 0x7f580c0ce210 2024-05-17T22:05:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 -- 172.21.15.83:0/2543473394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c0ce210 msgr2=0x7f580c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c0ce210 0x7f580c0ce620 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5800009870 tx=0x7f580001c560 comp rx=0 tx=0).stop 2024-05-17T22:05:26.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 -- 172.21.15.83:0/2543473394 shutdown_connections 2024-05-17T22:05:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c0ce210 0x7f580c0ce620 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 --2- 172.21.15.83:0/2543473394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c06d170 0x7f580c0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 -- 172.21.15.83:0/2543473394 >> 172.21.15.83:0/2543473394 conn(0x7f580c0b4170 msgr2=0x7f580c0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 -- 172.21.15.83:0/2543473394 shutdown_connections 2024-05-17T22:05:26.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.564+0000 7f58118dd640 1 -- 172.21.15.83:0/2543473394 wait complete. 2024-05-17T22:05:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 Processor -- start 2024-05-17T22:05:26.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- start start 2024-05-17T22:05:26.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 0x7f580c1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 0x7f580c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f580c16b0c0 con 0x7f580c06d170 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f580c15df80 con 0x7f580c0cef60 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f580c16b240 con 0x7f580c0ce210 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56868/0 (socket says 172.21.15.83:56868) 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 -- 172.21.15.83:0/120701797 learned_addr learned my addr 172.21.15.83:0/120701797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580b7fe640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 0x7f580c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 -- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 msgr2=0x7f580c1416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580a7fc640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 0x7f580c1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 0x7f580c1416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 -- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 msgr2=0x7f580c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 0x7f580c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 -- 172.21.15.83:0/120701797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c0c8080 con 0x7f580c06d170 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580b7fe640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 0x7f580c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580a7fc640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 0x7f580c1416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f580affd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d620 tx=0x7f57fc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0125f0 con 0x7f580c06d170 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc019070 con 0x7f580c06d170 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c155a10 con 0x7f580c06d170 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0129b0 con 0x7f580c06d170 2024-05-17T22:05:26.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c141ee0 con 0x7f580c06d170 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.568+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580c0ce620 con 0x7f580c06d170 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.572+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57fc012b50 con 0x7f580c06d170 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.572+0000 7f57f3fff640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 0x7f57f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.572+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5609+0+0 (secure 0 0 0) 0x7f57fc08f650 con 0x7f580c06d170 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.572+0000 7f580a7fc640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 0x7f57f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:26.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.572+0000 7f580a7fc640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 0x7f57f8071910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f58000af000 tx=0x7f5800008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:26.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.576+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f57fc05c630 con 0x7f580c06d170 2024-05-17T22:05:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:26.772+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f580c0d34b0 con 0x7f580c06d170 2024-05-17T22:05:27.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.452+0000 7f57f3fff640 1 -- 172.21.15.83:0/120701797 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v77) ==== 174+0+0 (secure 0 0 0) 0x7f57fc05bf30 con 0x7f580c06d170 2024-05-17T22:05:27.444 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 14 and data pool 15 2024-05-17T22:05:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.456+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 msgr2=0x7f57f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.456+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 0x7f57f8071910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f58000af000 tx=0x7f5800008040 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.456+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 msgr2=0x7f580c1441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.456+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00d620 tx=0x7f57fc00dae0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 shutdown_connections 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f57f806f400 0x7f57f8071910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c0cef60 0x7f580c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f580c0ce210 0x7f580c1416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 --2- 172.21.15.83:0/120701797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f580c06d170 0x7f580c1441a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 >> 172.21.15.83:0/120701797 conn(0x7f580c0b4170 msgr2=0x7f580c15c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 shutdown_connections 2024-05-17T22:05:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.460+0000 7f58118dd640 1 -- 172.21.15.83:0/120701797 wait complete. 2024-05-17T22:05:27.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 Processor -- start 2024-05-17T22:05:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 -- start start 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc4ac06dcc0 0x7fc4ac06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4ac0d4260 0x7fc4ac0d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ac06e610 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ac0d34e0 con 0x7fc4ac0d4260 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b3b62640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ac0d3780 con 0x7fc4ac06dcc0 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b18fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc4ac15ffa0 0x7fc4ac0d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56104/0 (socket says 172.21.15.83:56104) 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b20ff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc4ac0d4260 0x7fc4ac0d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b18fe640 1 -- 172.21.15.83:0/3308717992 learned_addr learned my addr 172.21.15.83:0/3308717992 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b10fd640 1 --2- 172.21.15.83:0/3308717992 >> v2:172.21.15.155:3300/0 conn(0x7fc4ac06dcc0 0x7fc4ac06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b20ff640 1 -- 172.21.15.83:0/3308717992 >> v2:172.21.15.155:3300/0 conn(0x7fc4ac06dcc0 msgr2=0x7fc4ac06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b20ff640 1 --2- 172.21.15.83:0/3308717992 >> v2:172.21.15.155:3300/0 conn(0x7fc4ac06dcc0 0x7fc4ac06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b20ff640 1 -- 172.21.15.83:0/3308717992 >> v1:172.21.15.83:6789/0 conn(0x7fc4ac15ffa0 legacy=0x7fc4ac0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b08fc640 1 -- 172.21.15.83:0/3308717992 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720278669 0 0) 0x7fc49c001120 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.636+0000 7fc4b20ff640 1 -- 172.21.15.83:0/3308717992 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac0d3a60 con 0x7fc4ac0d4260 2024-05-17T22:05:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b20ff640 1 --2- 172.21.15.83:0/3308717992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac0d4260 0x7fc4ac0d4670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0b5260 tx=0x7fc4a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210523d8acd4cb40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b08fc640 1 -- 172.21.15.83:0/3308717992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b2070 con 0x7fc4ac0d4260 2024-05-17T22:05:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b08fc640 1 -- 172.21.15.83:0/3308717992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a40a5870 con 0x7fc4ac0d4260 2024-05-17T22:05:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b08fc640 1 -- 172.21.15.83:0/3308717992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40a5c40 con 0x7fc4ac0d4260 2024-05-17T22:05:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 -- 172.21.15.83:0/3308717992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac0d4260 msgr2=0x7fc4ac0d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/3308717992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac0d4260 0x7fc4ac0d4670 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0b5260 tx=0x7fc4a401c930 comp rx=0 tx=0).stop 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 -- 172.21.15.83:0/3308717992 shutdown_connections 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/3308717992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac0d4260 0x7fc4ac0d4670 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/3308717992 >> v2:172.21.15.155:3300/0 conn(0x7fc4ac06dcc0 0x7fc4ac06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 -- 172.21.15.83:0/3308717992 >> 172.21.15.83:0/3308717992 conn(0x7fc4ac0d28f0 msgr2=0x7fc4ac0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 -- 172.21.15.83:0/3308717992 shutdown_connections 2024-05-17T22:05:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.640+0000 7fc4b3b62640 1 -- 172.21.15.83:0/3308717992 wait complete. 2024-05-17T22:05:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 Processor -- start 2024-05-17T22:05:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- start start 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 0x7fc4ac14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4ac0d4260 0x7fc4ac0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac0d17e0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac0d1660 con 0x7fc4ac06dcc0 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ac16cb80 con 0x7fc4ac0d4260 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55958/0 (socket says 172.21.15.83:55958) 2024-05-17T22:05:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 -- 172.21.15.83:0/184693971 learned_addr learned my addr 172.21.15.83:0/184693971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b18fe640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 0x7fc4ac14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b10fd640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4ac0d4260 0x7fc4ac0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 -- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4ac0d4260 msgr2=0x7fc4ac0cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4ac0d4260 0x7fc4ac0cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 -- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 msgr2=0x7fc4ac14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 0x7fc4ac14c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 -- 172.21.15.83:0/184693971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac0cd910 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b18fe640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 0x7fc4ac14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b20ff640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4019540 tx=0x7fc4a40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b2050 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a40b3040 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac0cdbf0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4003dc0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.644+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac0ce1b0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ac06e0d0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4a40c1060 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4a27fc640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 0x7fc490071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fc4a41294e0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4b18fe640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 0x7fc490071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.648+0000 7fc4b18fe640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 0x7fc490071910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000f70 tx=0x7fc49c007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.652+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc4a4016520 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.812+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4ac164440 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4a27fc640 1 -- 172.21.15.83:0/184693971 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16597 (secure 0 0 0) 0x7fc4a40f64b0 con 0x7fc4ac15ffa0 2024-05-17T22:05:27.806 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":95,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:05:26.778135+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:05:24.506575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-17T22:05:25.749261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 msgr2=0x7fc490071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 0x7fc490071910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000f70 tx=0x7fc49c007cb0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 msgr2=0x7fc4ac15c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4019540 tx=0x7fc4a40196d0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 shutdown_connections 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4ac15ffa0 0x7fc4ac15c7b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4ac0d4260 0x7fc4ac0cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc49006f400 0x7fc490071910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 --2- 172.21.15.83:0/184693971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4ac06dcc0 0x7fc4ac14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 >> 172.21.15.83:0/184693971 conn(0x7fc4ac0d28f0 msgr2=0x7fc4ac0b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 shutdown_connections 2024-05-17T22:05:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.816+0000 7fc4b3b62640 1 -- 172.21.15.83:0/184693971 wait complete. 2024-05-17T22:05:27.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:27.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.984+0000 7f272c9e5640 1 Processor -- start 2024-05-17T22:05:27.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.984+0000 7f272c9e5640 1 -- start start 2024-05-17T22:05:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272c9e5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f272806ffd0 0x7f27280703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272c9e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f27281605a0 0x7f27281609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272c9e5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27281465d0 con 0x7f27281461c0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272c9e5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2728160ef0 con 0x7f27281605a0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272c9e5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f272806ef30 con 0x7f272806ffd0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272659c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f27281461c0 0x7f272815e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56112/0 (socket says 172.21.15.83:56112) 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272659c640 1 -- 172.21.15.83:0/2301679532 learned_addr learned my addr 172.21.15.83:0/2301679532 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2726d9d640 1 --2- 172.21.15.83:0/2301679532 >> v2:172.21.15.83:3300/0 conn(0x7f27281605a0 0x7f27281609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2725d9b640 1 --2- 172.21.15.83:0/2301679532 >> v2:172.21.15.155:3300/0 conn(0x7f272806ffd0 0x7f27280703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272559a640 1 -- 172.21.15.83:0/2301679532 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252712218 0 0) 0x7f27281465d0 con 0x7f27281461c0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f272559a640 1 -- 172.21.15.83:0/2301679532 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2710003530 con 0x7f27281461c0 2024-05-17T22:05:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2726d9d640 1 -- 172.21.15.83:0/2301679532 >> v2:172.21.15.155:3300/0 conn(0x7f272806ffd0 msgr2=0x7f27280703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2726d9d640 1 --2- 172.21.15.83:0/2301679532 >> v2:172.21.15.155:3300/0 conn(0x7f272806ffd0 0x7f27280703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2726d9d640 1 -- 172.21.15.83:0/2301679532 >> v1:172.21.15.83:6789/0 conn(0x7f27281461c0 legacy=0x7f272815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.988+0000 7f2726d9d640 1 -- 172.21.15.83:0/2301679532 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2728070920 con 0x7f27281605a0 2024-05-17T22:05:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f2726d9d640 1 --2- 172.21.15.83:0/2301679532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27281605a0 0x7f27281609b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f2728071230 tx=0x7f272001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf88283f29f74493 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272559a640 1 -- 172.21.15.83:0/2301679532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27200b2070 con 0x7f27281605a0 2024-05-17T22:05:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272559a640 1 -- 172.21.15.83:0/2301679532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27200a59c0 con 0x7f27281605a0 2024-05-17T22:05:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272559a640 1 -- 172.21.15.83:0/2301679532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27200a5d90 con 0x7f27281605a0 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- 172.21.15.83:0/2301679532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27281605a0 msgr2=0x7f27281609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2301679532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27281605a0 0x7f27281609b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f2728071230 tx=0x7f272001c900 comp rx=0 tx=0).stop 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- 172.21.15.83:0/2301679532 shutdown_connections 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2301679532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27281605a0 0x7f27281609b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2301679532 >> v2:172.21.15.155:3300/0 conn(0x7f272806ffd0 0x7f27280703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- 172.21.15.83:0/2301679532 >> 172.21.15.83:0/2301679532 conn(0x7f272806cd90 msgr2=0x7f27280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- 172.21.15.83:0/2301679532 shutdown_connections 2024-05-17T22:05:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- 172.21.15.83:0/2301679532 wait complete. 2024-05-17T22:05:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 Processor -- start 2024-05-17T22:05:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.992+0000 7f272c9e5640 1 -- start start 2024-05-17T22:05:27.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27281461c0 0x7f2728072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 0x7f2728073370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f272815e880 con 0x7f272806ffd0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f272806fab0 con 0x7f27281605a0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f27280d10a0 con 0x7f27281461c0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55974/0 (socket says 172.21.15.83:55974) 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 -- 172.21.15.83:0/2217262823 learned_addr learned my addr 172.21.15.83:0/2217262823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2726d9d640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 0x7f2728073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 -- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27281461c0 msgr2=0x7f2728072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2725d9b640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27281461c0 0x7f2728072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27281461c0 0x7f2728072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 -- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 msgr2=0x7f2728073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 0x7f2728073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 -- 172.21.15.83:0/2217262823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272813caf0 con 0x7f272806ffd0 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2726d9d640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 0x7f2728073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:27.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272659c640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f271800c430 tx=0x7f271800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718017070 con 0x7f272806ffd0 2024-05-17T22:05:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2718005cf0 con 0x7f272806ffd0 2024-05-17T22:05:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272813d0b0 con 0x7f272806ffd0 2024-05-17T22:05:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180072f0 con 0x7f272806ffd0 2024-05-17T22:05:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2728142cc0 con 0x7f272806ffd0 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27280703e0 con 0x7f272806ffd0 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f271801a070 con 0x7f272806ffd0 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:27.996+0000 7f2716ffd640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 0x7f2708071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.000+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f271808e600 con 0x7f272806ffd0 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.000+0000 7f2725d9b640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 0x7f2708071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.000+0000 7f2725d9b640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 0x7f2708071910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f272813a750 tx=0x7f271c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.004+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f271805b580 con 0x7f272806ffd0 2024-05-17T22:05:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.164+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27280b41a0 con 0x7f272806ffd0 2024-05-17T22:05:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f2716ffd640 1 -- 172.21.15.83:0/2217262823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16597 (secure 0 0 0) 0x7f271805ae80 con 0x7f272806ffd0 2024-05-17T22:05:28.158 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:28.159 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":95,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:05:26.778135+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:05:24.506575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-17T22:05:25.749261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 msgr2=0x7f2708071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 0x7f2708071910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f272813a750 tx=0x7f271c008040 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 msgr2=0x7f2728158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f271800c430 tx=0x7f271800c8f0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 shutdown_connections 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f270806f400 0x7f2708071910 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27281605a0 0x7f2728073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27281461c0 0x7f2728072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 --2- 172.21.15.83:0/2217262823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f272806ffd0 0x7f2728158350 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 >> 172.21.15.83:0/2217262823 conn(0x7f272806cd90 msgr2=0x7f27281437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:28.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 shutdown_connections 2024-05-17T22:05:28.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.168+0000 7f272c9e5640 1 -- 172.21.15.83:0/2217262823 wait complete. 2024-05-17T22:05:28.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T22:05:28.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 Processor -- start 2024-05-17T22:05:28.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 -- start start 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240b3d00 0x7f85240d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8524143f20 0x7f8524072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85240737d0 con 0x7f85240733c0 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f85240b4110 con 0x7f8524143f20 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f852a20a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8524070770 con 0x7f85240b3d00 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8522ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8524143f20 0x7f8524072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8523fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85240733c0 0x7f852406eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56116/0 (socket says 172.21.15.83:56116) 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8523fff640 1 -- 172.21.15.83:0/4125008044 learned_addr learned my addr 172.21.15.83:0/4125008044 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f85237fe640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240b3d00 0x7f85240d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8522ffd640 1 -- 172.21.15.83:0/4125008044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240b3d00 msgr2=0x7f85240d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:28.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8522ffd640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240b3d00 0x7f85240d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8522ffd640 1 -- 172.21.15.83:0/4125008044 >> v1:172.21.15.83:6789/0 conn(0x7f85240733c0 legacy=0x7f852406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.348+0000 7f8522ffd640 1 -- 172.21.15.83:0/4125008044 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85240d10d0 con 0x7f8524143f20 2024-05-17T22:05:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f8522ffd640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f8524072e80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097c0 tx=0x7f850c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f70593a943178a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f85227fc640 1 -- 172.21.15.83:0/4125008044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b2070 con 0x7f8524143f20 2024-05-17T22:05:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f85227fc640 1 -- 172.21.15.83:0/4125008044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c0ad640 con 0x7f8524143f20 2024-05-17T22:05:28.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f85227fc640 1 -- 172.21.15.83:0/4125008044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0ada10 con 0x7f8524143f20 2024-05-17T22:05:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 -- 172.21.15.83:0/4125008044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 msgr2=0x7f8524072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f8524072e80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f850c0097c0 tx=0x7f850c01c4a0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 -- 172.21.15.83:0/4125008044 shutdown_connections 2024-05-17T22:05:28.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f8524072e80 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 --2- 172.21.15.83:0/4125008044 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240b3d00 0x7f85240d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 -- 172.21.15.83:0/4125008044 >> 172.21.15.83:0/4125008044 conn(0x7f852406cd90 msgr2=0x7f85240b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:28.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 -- 172.21.15.83:0/4125008044 shutdown_connections 2024-05-17T22:05:28.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.352+0000 7f852a20a640 1 -- 172.21.15.83:0/4125008044 wait complete. 2024-05-17T22:05:28.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 Processor -- start 2024-05-17T22:05:28.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- start start 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 0x7f8524142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85240b3d00 0x7f85240aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8524070c00 con 0x7f8524143f20 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8524144330 con 0x7f85240b3d00 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f852415b460 con 0x7f85240733c0 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55986/0 (socket says 172.21.15.83:55986) 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 -- 172.21.15.83:0/4108881864 learned_addr learned my addr 172.21.15.83:0/4108881864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:28.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8522ffd640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85240b3d00 0x7f85240aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f85237fe640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 0x7f8524142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 -- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 msgr2=0x7f8524142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 0x7f8524142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 -- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85240b3d00 msgr2=0x7f85240aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85240b3d00 0x7f85240aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 -- 172.21.15.83:0/4108881864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85240af270 con 0x7f8524143f20 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f85237fe640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 0x7f8524142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f8523fff640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f851400c3f0 tx=0x7f851400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514017070 con 0x7f8524143f20 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8514005cf0 con 0x7f8524143f20 2024-05-17T22:05:28.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85240adf90 con 0x7f8524143f20 2024-05-17T22:05:28.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85140072d0 con 0x7f8524143f20 2024-05-17T22:05:28.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.356+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85240ae250 con 0x7f8524143f20 2024-05-17T22:05:28.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85240d0a10 con 0x7f8524143f20 2024-05-17T22:05:28.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f851401a070 con 0x7f8524143f20 2024-05-17T22:05:28.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f851bfff640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 0x7f8504071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:28.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f85237fe640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 0x7f8504071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:28.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f851408e4e0 con 0x7f8524143f20 2024-05-17T22:05:28.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.360+0000 7f85237fe640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 0x7f8504071940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f851c009f50 tx=0x7f851c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:28.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.364+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f851405ad80 con 0x7f8524143f20 2024-05-17T22:05:28.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:28.572+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f85240b1970 con 0x7f8524143f20 2024-05-17T22:05:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f851bfff640 1 -- 172.21.15.83:0/4108881864 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7f851405f080 con 0x7f8524143f20 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 msgr2=0x7f8504071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 0x7f8504071940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f851c009f50 tx=0x7f851c00b040 comp rx=0 tx=0).stop 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 msgr2=0x7f852415a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f851400c3f0 tx=0x7f851400c8b0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 shutdown_connections 2024-05-17T22:05:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8524143f20 0x7f852415a850 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f850406f430 0x7f8504071940 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85240b3d00 0x7f85240aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 --2- 172.21.15.83:0/4108881864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85240733c0 0x7f8524142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 >> 172.21.15.83:0/4108881864 conn(0x7f852406cd90 msgr2=0x7f8524145970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 shutdown_connections 2024-05-17T22:05:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.452+0000 7f852a20a640 1 -- 172.21.15.83:0/4108881864 wait complete. 2024-05-17T22:05:29.473 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T22:05:29.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T22:05:29.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 Processor -- start 2024-05-17T22:05:29.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 -- start start 2024-05-17T22:05:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc348073590 0x7fc3480d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc3481447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3480d10d0 con 0x7fc3480b3d00 2024-05-17T22:05:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc3480b4110 con 0x7fc348073590 2024-05-17T22:05:29.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc3502b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc348070770 con 0x7fc348143f20 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc34e04c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc3480b3d00 0x7fc348072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56130/0 (socket says 172.21.15.83:56130) 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.640+0000 7fc34e04c640 1 -- 172.21.15.83:0/1464308286 learned_addr learned my addr 172.21.15.83:0/1464308286 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1464308286 >> v2:172.21.15.155:3300/0 conn(0x7fc348073590 0x7fc3480d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34e84d640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc3481447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766149440 0 0) 0x7fc3480d10d0 con 0x7fc3480b3d00 2024-05-17T22:05:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc334003530 con 0x7fc3480b3d00 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1712133603 0 0) 0x7fc334003530 con 0x7fc3480b3d00 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 msgr2=0x7fc3481447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc3481447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 >> v2:172.21.15.155:3300/0 conn(0x7fc348073590 msgr2=0x7fc3480d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 --2- 172.21.15.83:0/1464308286 >> v2:172.21.15.155:3300/0 conn(0x7fc348073590 0x7fc3480d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc348070120 con 0x7fc3480b3d00 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fc344001700 con 0x7fc3480b3d00 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 >> v1:172.21.15.83:6789/0 conn(0x7fc3480b3d00 legacy=0x7fc348072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc334003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc334003fe0 0x7fc334004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334004d50 0x7fc334005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3480b4110 con 0x7fc334003fe0 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc348070770 con 0x7fc334004d50 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc33400bad0 con 0x7fc348143f20 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc334003fe0 0x7fc334004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34e04c640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc334003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34e84d640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334004d50 0x7fc334005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 -- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 msgr2=0x7fc334003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc348143f20 0x7fc334003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 -- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334004d50 msgr2=0x7fc334005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334004d50 0x7fc334005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 -- 172.21.15.83:0/1464308286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc334003530 con 0x7fc334003fe0 2024-05-17T22:05:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc334003fe0 0x7fc334004810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc338009ae0 tx=0x7fc33801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43428ec88084405 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380b2070 con 0x7fc334003fe0 2024-05-17T22:05:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3380a5550 con 0x7fc334003fe0 2024-05-17T22:05:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.644+0000 7fc34d04a640 1 -- 172.21.15.83:0/1464308286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380096b0 con 0x7fc334003fe0 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- 172.21.15.83:0/1464308286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc334003fe0 msgr2=0x7fc334004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1464308286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc334003fe0 0x7fc334004810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc338009ae0 tx=0x7fc33801ca70 comp rx=0 tx=0).stop 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 -- 172.21.15.83:0/1464308286 reap_dead start 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- 172.21.15.83:0/1464308286 shutdown_connections 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- 172.21.15.83:0/1464308286 >> 172.21.15.83:0/1464308286 conn(0x7fc34806cd90 msgr2=0x7fc3480b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- 172.21.15.83:0/1464308286 shutdown_connections 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- 172.21.15.83:0/1464308286 wait complete. 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 Processor -- start 2024-05-17T22:05:29.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- start start 2024-05-17T22:05:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 0x7fc3480b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3481ec3e0 0x7fc3480cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc348147600 con 0x7fc3480cbe80 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc34806fed0 con 0x7fc334003fe0 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc3502b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc348144330 con 0x7fc3481ec3e0 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55992/0 (socket says 172.21.15.83:55992) 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 -- 172.21.15.83:0/1556230339 learned_addr learned my addr 172.21.15.83:0/1556230339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e04c640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 0x7fc3480b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34d84b640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3481ec3e0 0x7fc3480cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 -- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3481ec3e0 msgr2=0x7fc3480cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3481ec3e0 0x7fc3480cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 -- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 msgr2=0x7fc3480b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 0x7fc3480b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e84d640 1 -- 172.21.15.83:0/1556230339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3480b4df0 con 0x7fc3480cbe80 2024-05-17T22:05:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.648+0000 7fc34e04c640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 0x7fc3480b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc34e84d640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc3481472f0 tx=0x7fc3400115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340011e10 con 0x7fc3480cbe80 2024-05-17T22:05:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34001b070 con 0x7fc3480cbe80 2024-05-17T22:05:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3480b50d0 con 0x7fc3480cbe80 2024-05-17T22:05:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3400039a0 con 0x7fc3480cbe80 2024-05-17T22:05:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3480b5690 con 0x7fc3480cbe80 2024-05-17T22:05:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc348070c00 con 0x7fc3480cbe80 2024-05-17T22:05:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc340003b40 con 0x7fc3480cbe80 2024-05-17T22:05:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 0x7fc32c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc34e04c640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 0x7fc32c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fc3400909f0 con 0x7fc3480cbe80 2024-05-17T22:05:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.652+0000 7fc34e04c640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 0x7fc32c071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc344001ec0 tx=0x7fc344001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.656+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc340025020 con 0x7fc3480cbe80 2024-05-17T22:05:29.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:29.848+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc3480b4b80 con 0x7fc3480cbe80 2024-05-17T22:05:30.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.460+0000 7fc33effd640 1 -- 172.21.15.83:0/1556230339 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7fc34005d8c0 con 0x7fc3480cbe80 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 msgr2=0x7fc32c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 0x7fc32c071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc344001ec0 tx=0x7fc344001690 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 msgr2=0x7fc3481f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fc3481472f0 tx=0x7fc3400115a0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 shutdown_connections 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3480cbe80 0x7fc3481f9000 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3481ec3e0 0x7fc3480cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc32c06f400 0x7fc32c071910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 --2- 172.21.15.83:0/1556230339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc334003fe0 0x7fc3480b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 >> 172.21.15.83:0/1556230339 conn(0x7fc34806cd90 msgr2=0x7fc348141060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:30.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 shutdown_connections 2024-05-17T22:05:30.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.464+0000 7fc3502b0640 1 -- 172.21.15.83:0/1556230339 wait complete. 2024-05-17T22:05:30.481 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T22:05:30.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:30.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 Processor -- start 2024-05-17T22:05:30.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 -- start start 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 0x7fbf0815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf081465d0 con 0x7fbf0815e6b0 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbf0806ef30 con 0x7fbf081461c0 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0fa1a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0806f210 con 0x7fbf080706e0 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0dfb7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0dfb7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:38400/0 (socket says 172.21.15.83:38400) 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0dfb7640 1 -- 172.21.15.83:0/2695175117 learned_addr learned my addr 172.21.15.83:0/2695175117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:30.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.636+0000 7fbf0d7b6640 1 --2- 172.21.15.83:0/2695175117 >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 0x7fbf0815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0dfb7640 1 -- 172.21.15.83:0/2695175117 >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 msgr2=0x7fbf0815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0dfb7640 1 --2- 172.21.15.83:0/2695175117 >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 0x7fbf0815e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0dfb7640 1 -- 172.21.15.83:0/2695175117 >> v1:172.21.15.155:6789/0 conn(0x7fbf0815e6b0 legacy=0x7fbf0806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0dfb7640 1 -- 172.21.15.83:0/2695175117 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0806f4f0 con 0x7fbf080706e0 2024-05-17T22:05:30.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0d7b6640 1 --2- 172.21.15.83:0/2695175117 >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 0x7fbf0815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:05:30.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0dfb7640 1 --2- 172.21.15.83:0/2695175117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009e50 tx=0x7fbf0401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33476d515f9705f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:30.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbefffff640 1 -- 172.21.15.83:0/2695175117 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a57e0 con 0x7fbf080706e0 2024-05-17T22:05:30.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbefffff640 1 -- 172.21.15.83:0/2695175117 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf040a5980 con 0x7fbf080706e0 2024-05-17T22:05:30.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbefffff640 1 -- 172.21.15.83:0/2695175117 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a5d50 con 0x7fbf080706e0 2024-05-17T22:05:30.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/2695175117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 msgr2=0x7fbf081605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/2695175117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009e50 tx=0x7fbf0401ca00 comp rx=0 tx=0).stop 2024-05-17T22:05:30.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/2695175117 shutdown_connections 2024-05-17T22:05:30.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/2695175117 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf080706e0 0x7fbf081605a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/2695175117 >> v2:172.21.15.155:3300/0 conn(0x7fbf081461c0 0x7fbf0815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/2695175117 >> 172.21.15.83:0/2695175117 conn(0x7fbf0806cd90 msgr2=0x7fbf080b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/2695175117 shutdown_connections 2024-05-17T22:05:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.640+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/2695175117 wait complete. 2024-05-17T22:05:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 Processor -- start 2024-05-17T22:05:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 -- start start 2024-05-17T22:05:30.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf081461c0 0x7fbf08145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf0815e6b0 0x7fbf080b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf08169940 con 0x7fbf080706e0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf080d12b0 con 0x7fbf0815e6b0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0fa1a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbf08169ac0 con 0x7fbf081461c0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0d7b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0d7b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55998/0 (socket says 172.21.15.83:55998) 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.644+0000 7fbf0d7b6640 1 -- 172.21.15.83:0/828940316 learned_addr learned my addr 172.21.15.83:0/828940316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0cfb5640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf081461c0 0x7fbf08145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0dfb7640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf0815e6b0 0x7fbf080b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 -- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf081461c0 msgr2=0x7fbf08145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf081461c0 0x7fbf08145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 -- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf0815e6b0 msgr2=0x7fbf080b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf0815e6b0 0x7fbf080b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 -- 172.21.15.83:0/828940316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf08145ac0 con 0x7fbf080706e0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0d7b6640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e930 tx=0x7fbf0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00017070 con 0x7fbf080706e0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0000ca00 con 0x7fbf080706e0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf081413f0 con 0x7fbf080706e0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000cdd0 con 0x7fbf080706e0 2024-05-17T22:05:30.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf081419b0 con 0x7fbf080706e0 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf081605a0 con 0x7fbf080706e0 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbf0001a070 con 0x7fbf080706e0 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 0x7fbeec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0cfb5640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 0x7fbeec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fbf0008f080 con 0x7fbf080706e0 2024-05-17T22:05:30.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.648+0000 7fbf0cfb5640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 0x7fbeec071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b08e0 tx=0x7fbef8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:30.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.652+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbf08145ac0 con 0x7fbf080706e0 2024-05-17T22:05:30.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.844+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf080b41a0 con 0x7fbf080706e0 2024-05-17T22:05:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.844+0000 7fbefdffb640 1 -- 172.21.15.83:0/828940316 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5189 (secure 0 0 0) 0x7fbf0005c090 con 0x7fbf080706e0 2024-05-17T22:05:30.837 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:30.838 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:05:29.852155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7407":{"gid":7407,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1996242614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1996242614},{"type":"v1","addr":"172.21.15.83:6836","nonce":1996242614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7407,"qdb_cluster":[7407]},"id":7}]} 2024-05-17T22:05:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 81 2024-05-17T22:05:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 msgr2=0x7fbeec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 0x7fbeec071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf080b08e0 tx=0x7fbef8009040 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 msgr2=0x7fbf0815ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e930 tx=0x7fbf0000edf0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 shutdown_connections 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbeec06f430 0x7fbeec071940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf0815e6b0 0x7fbf080b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf081461c0 0x7fbf08145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 --2- 172.21.15.83:0/828940316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf080706e0 0x7fbf0815ba30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 >> 172.21.15.83:0/828940316 conn(0x7fbf0806cd90 msgr2=0x7fbf08075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 shutdown_connections 2024-05-17T22:05:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:30.848+0000 7fbf0fa1a640 1 -- 172.21.15.83:0/828940316 wait complete. 2024-05-17T22:05:30.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:05:30.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 Processor -- start 2024-05-17T22:05:30.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 -- start start 2024-05-17T22:05:30.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f51f015d2f0 0x7f51f0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f0073540 0x7f51f0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:30.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f015c830 con 0x7f51f015fe20 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f51f01608a0 con 0x7f51f015d2f0 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f7809640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51f0160b80 con 0x7f51f0073540 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f55a5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f51f015d2f0 0x7f51f0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f4da4640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51f015fe20 0x7f51f0072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56136/0 (socket says 172.21.15.83:56136) 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f4da4640 1 -- 172.21.15.83:0/912113934 learned_addr learned my addr 172.21.15.83:0/912113934 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f5da6640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f0073540 0x7f51f0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f55a5640 1 -- 172.21.15.83:0/912113934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f0073540 msgr2=0x7f51f0160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f55a5640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f0073540 0x7f51f0160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f55a5640 1 -- 172.21.15.83:0/912113934 >> v1:172.21.15.83:6789/0 conn(0x7f51f015fe20 legacy=0x7f51f0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51f55a5640 1 -- 172.21.15.83:0/912113934 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f015caf0 con 0x7f51f015d2f0 2024-05-17T22:05:30.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.000+0000 7f51e7fff640 1 -- 172.21.15.83:0/912113934 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935944180 0 0) 0x7f51f015c830 con 0x7f51f015fe20 2024-05-17T22:05:30.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f55a5640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f0159c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00a960 tx=0x7f51ec01d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e139f6dbc5db40 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51e7fff640 1 -- 172.21.15.83:0/912113934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0b3070 con 0x7f51f015d2f0 2024-05-17T22:05:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51e7fff640 1 -- 172.21.15.83:0/912113934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec0a6740 con 0x7f51f015d2f0 2024-05-17T22:05:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51e7fff640 1 -- 172.21.15.83:0/912113934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00b040 con 0x7f51f015d2f0 2024-05-17T22:05:30.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 -- 172.21.15.83:0/912113934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 msgr2=0x7f51f0159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f0159c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00a960 tx=0x7f51ec01d7e0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 -- 172.21.15.83:0/912113934 shutdown_connections 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f0073540 0x7f51f0160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 --2- 172.21.15.83:0/912113934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f0159c20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 -- 172.21.15.83:0/912113934 >> 172.21.15.83:0/912113934 conn(0x7f51f006cd90 msgr2=0x7f51f00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 -- 172.21.15.83:0/912113934 shutdown_connections 2024-05-17T22:05:30.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.004+0000 7f51f7809640 1 -- 172.21.15.83:0/912113934 wait complete. 2024-05-17T22:05:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 Processor -- start 2024-05-17T22:05:30.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- start start 2024-05-17T22:05:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 0x7f51f014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51f014c620 0x7f51f01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00b04f0 con 0x7f51f015d2f0 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00b0370 con 0x7f51f014c620 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51f00b0670 con 0x7f51f015fe20 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56018/0 (socket says 172.21.15.83:56018) 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 -- 172.21.15.83:0/955667460 learned_addr learned my addr 172.21.15.83:0/955667460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f4da4640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 0x7f51f014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f5da6640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51f014c620 0x7f51f01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 -- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 msgr2=0x7f51f014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 0x7f51f014c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 -- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51f014c620 msgr2=0x7f51f01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51f014c620 0x7f51f01501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 -- 172.21.15.83:0/955667460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f014ca30 con 0x7f51f015d2f0 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f4da4640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 0x7f51f014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:31.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f55a5640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0b1040 tx=0x7f51ec01a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0b3040 con 0x7f51f015d2f0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec0bd070 con 0x7f51f015d2f0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f01538d0 con 0x7f51f015d2f0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0ae6a0 con 0x7f51f015d2f0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f0152da0 con 0x7f51f015d2f0 2024-05-17T22:05:31.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.008+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f0160230 con 0x7f51f015d2f0 2024-05-17T22:05:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.012+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51ec0ae840 con 0x7f51f015d2f0 2024-05-17T22:05:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.012+0000 7f51e5ffb640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 0x7f51d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.012+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f51ec129490 con 0x7f51f015d2f0 2024-05-17T22:05:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.012+0000 7f51f4da4640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 0x7f51d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.012+0000 7f51f4da4640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 0x7f51d4071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51f01535d0 tx=0x7f51e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.016+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f51f014ca30 con 0x7f51f015d2f0 2024-05-17T22:05:31.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.164+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51f00b68b0 con 0x7f51f015d2f0 2024-05-17T22:05:31.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.164+0000 7f51e5ffb640 1 -- 172.21.15.83:0/955667460 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16584 (secure 0 0 0) 0x7f51ec00a030 con 0x7f51f015d2f0 2024-05-17T22:05:31.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:31.158 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":96,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:05:28.395630+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:05:24.506575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-17T22:05:25.749261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 msgr2=0x7f51d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 0x7f51d4071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51f01535d0 tx=0x7f51e0009040 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 msgr2=0x7f51f00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0b1040 tx=0x7f51ec01a580 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 shutdown_connections 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51d406f400 0x7f51d4071910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51f014c620 0x7f51f01501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51f015fe20 0x7f51f014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 --2- 172.21.15.83:0/955667460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51f015d2f0 0x7f51f00cd780 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 >> 172.21.15.83:0/955667460 conn(0x7f51f006cd90 msgr2=0x7f51f00b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 shutdown_connections 2024-05-17T22:05:31.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.168+0000 7f51f7809640 1 -- 172.21.15.83:0/955667460 wait complete. 2024-05-17T22:05:31.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:05:31.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:05:31.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:05:31.226 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:05:31.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 Processor -- start 2024-05-17T22:05:31.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 -- start start 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9c141605a0 0x7f9c141609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c141465d0 con 0x7f9c141461c0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c14160ef0 con 0x7f9c1406ffd0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c1b214640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1406ef30 con 0x7f9c141605a0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c18fb0640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9c141461c0 0x7f9c1415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56138/0 (socket says 172.21.15.83:56138) 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c18fb0640 1 -- 172.21.15.83:0/2414694751 learned_addr learned my addr 172.21.15.83:0/2414694751 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c197b1640 1 --2- 172.21.15.83:0/2414694751 >> v2:172.21.15.83:3300/0 conn(0x7f9c141605a0 0x7f9c141609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c0b7fe640 1 -- 172.21.15.83:0/2414694751 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133378832 0 0) 0x7f9c141465d0 con 0x7f9c141461c0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c0b7fe640 1 -- 172.21.15.83:0/2414694751 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bfc003530 con 0x7f9c141461c0 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c197b1640 1 -- 172.21.15.83:0/2414694751 >> v1:172.21.15.155:6789/0 conn(0x7f9c1406ffd0 legacy=0x7f9c140703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c197b1640 1 -- 172.21.15.83:0/2414694751 >> v1:172.21.15.83:6789/0 conn(0x7f9c141461c0 legacy=0x7f9c1415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.336+0000 7f9c197b1640 1 -- 172.21.15.83:0/2414694751 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1406f210 con 0x7f9c141605a0 2024-05-17T22:05:31.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c197b1640 1 --2- 172.21.15.83:0/2414694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c141609b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009d90 tx=0x7f9c0c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc62cccb035de21b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c0b7fe640 1 -- 172.21.15.83:0/2414694751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0b2070 con 0x7f9c141605a0 2024-05-17T22:05:31.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c0b7fe640 1 -- 172.21.15.83:0/2414694751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0a5820 con 0x7f9c141605a0 2024-05-17T22:05:31.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c0b7fe640 1 -- 172.21.15.83:0/2414694751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0a5bf0 con 0x7f9c141605a0 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- 172.21.15.83:0/2414694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 msgr2=0x7f9c141609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 --2- 172.21.15.83:0/2414694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c141609b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009d90 tx=0x7f9c0c01c760 comp rx=0 tx=0).stop 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- 172.21.15.83:0/2414694751 shutdown_connections 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 --2- 172.21.15.83:0/2414694751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c141609b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- 172.21.15.83:0/2414694751 >> 172.21.15.83:0/2414694751 conn(0x7f9c1406cd90 msgr2=0x7f9c140b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:31.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- 172.21.15.83:0/2414694751 shutdown_connections 2024-05-17T22:05:31.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- 172.21.15.83:0/2414694751 wait complete. 2024-05-17T22:05:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 Processor -- start 2024-05-17T22:05:31.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.340+0000 7f9c1b214640 1 -- start start 2024-05-17T22:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c1406ffd0 0x7f9c1415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 0x7f9c1413b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c140d10a0 con 0x7f9c141605a0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1415e880 con 0x7f9c141461c0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140d1220 con 0x7f9c1406ffd0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56030/0 (socket says 172.21.15.83:56030) 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 -- 172.21.15.83:0/91854922 learned_addr learned my addr 172.21.15.83:0/91854922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c18fb0640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c1406ffd0 0x7f9c1415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c0bfff640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 0x7f9c1413b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 -- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c1406ffd0 msgr2=0x7f9c1415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c1406ffd0 0x7f9c1415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 -- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 msgr2=0x7f9c1413b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 0x7f9c1413b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 -- 172.21.15.83:0/91854922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1413e6b0 con 0x7f9c141605a0 2024-05-17T22:05:31.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c0bfff640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 0x7f9c1413b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c197b1640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c01ae10 tx=0x7f9c0c019500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0b2040 con 0x7f9c141605a0 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0bf070 con 0x7f9c141605a0 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1415b060 con 0x7f9c141605a0 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0b8670 con 0x7f9c141605a0 2024-05-17T22:05:31.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1415b620 con 0x7f9c141605a0 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c140b41a0 con 0x7f9c141605a0 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.344+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9c0c0b8810 con 0x7f9c141605a0 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.348+0000 7f9c097fa640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 0x7f9c00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.348+0000 7f9c18fb0640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 0x7f9c00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.348+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f9c0c12ad60 con 0x7f9c141605a0 2024-05-17T22:05:31.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.348+0000 7f9c18fb0640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 0x7f9c00071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c10007a70 tx=0x7f9c10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.352+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9c14160ef0 con 0x7f9c141605a0 2024-05-17T22:05:31.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.500+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c1413c440 con 0x7f9c0006f400 2024-05-17T22:05:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.512+0000 7f9c097fa640 1 -- 172.21.15.83:0/91854922 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f9c1413c440 con 0x7f9c0006f400 2024-05-17T22:05:31.505 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:31.506 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 msgr2=0x7f9c00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 0x7f9c00071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c10007a70 tx=0x7f9c10009040 comp rx=0 tx=0).stop 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 msgr2=0x7f9c1413dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c01ae10 tx=0x7f9c0c019500 comp rx=0 tx=0).stop 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 shutdown_connections 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c141605a0 0x7f9c1413dd60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9c0006f400 0x7f9c00071910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c141461c0 0x7f9c1413b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 --2- 172.21.15.83:0/91854922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c1406ffd0 0x7f9c1415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 >> 172.21.15.83:0/91854922 conn(0x7f9c1406cd90 msgr2=0x7f9c1415f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 shutdown_connections 2024-05-17T22:05:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.516+0000 7f9c1b214640 1 -- 172.21.15.83:0/91854922 wait complete. 2024-05-17T22:05:31.525 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-05-17T22:05:31.145764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259920,"kb_used_data":10444,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714800,"statfs":{"total":773094113280,"available":772827955200,"internally_reserved":0,"allocated":10694656,"data_stored":5247896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.788063"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510966+0000","last_change":"2024-05-17T22:05:27.437616+0000","last_active":"2024-05-17T22:05:28.510966+0000","last_peered":"2024-05-17T22:05:28.510966+0000","last_clean":"2024-05-17T22:05:28.510966+0000","last_became_active":"2024-05-17T22:05:27.435692+0000","last_became_peered":"2024-05-17T22:05:27.435692+0000","last_unstale":"2024-05-17T22:05:28.510966+0000","last_undegraded":"2024-05-17T22:05:28.510966+0000","last_fullsized":"2024-05-17T22:05:28.510966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:04:18.149802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465877+0000","last_change":"2024-05-17T22:05:27.455228+0000","last_active":"2024-05-17T22:05:28.465877+0000","last_peered":"2024-05-17T22:05:28.465877+0000","last_clean":"2024-05-17T22:05:28.465877+0000","last_became_active":"2024-05-17T22:05:27.452733+0000","last_became_peered":"2024-05-17T22:05:27.452733+0000","last_unstale":"2024-05-17T22:05:28.465877+0000","last_undegraded":"2024-05-17T22:05:28.465877+0000","last_fullsized":"2024-05-17T22:05:28.465877+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:24.428328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511603+0000","last_change":"2024-05-17T22:05:27.436096+0000","last_active":"2024-05-17T22:05:28.511603+0000","last_peered":"2024-05-17T22:05:28.511603+0000","last_clean":"2024-05-17T22:05:28.511603+0000","last_became_active":"2024-05-17T22:05:27.435215+0000","last_became_peered":"2024-05-17T22:05:27.435215+0000","last_unstale":"2024-05-17T22:05:28.511603+0000","last_undegraded":"2024-05-17T22:05:28.511603+0000","last_fullsized":"2024-05-17T22:05:28.511603+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:01.067972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512672+0000","last_change":"2024-05-17T22:05:27.449736+0000","last_active":"2024-05-17T22:05:28.512672+0000","last_peered":"2024-05-17T22:05:28.512672+0000","last_clean":"2024-05-17T22:05:28.512672+0000","last_became_active":"2024-05-17T22:05:27.445376+0000","last_became_peered":"2024-05-17T22:05:27.445376+0000","last_unstale":"2024-05-17T22:05:28.512672+0000","last_undegraded":"2024-05-17T22:05:28.512672+0000","last_fullsized":"2024-05-17T22:05:28.512672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512205+0000","last_change":"2024-05-17T22:05:27.449927+0000","last_active":"2024-05-17T22:05:28.512205+0000","last_peered":"2024-05-17T22:05:28.512205+0000","last_clean":"2024-05-17T22:05:28.512205+0000","last_became_active":"2024-05-17T22:05:27.449247+0000","last_became_peered":"2024-05-17T22:05:27.449247+0000","last_unstale":"2024-05-17T22:05:28.512205+0000","last_undegraded":"2024-05-17T22:05:28.512205+0000","last_fullsized":"2024-05-17T22:05:28.512205+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512451+0000","last_change":"2024-05-17T22:05:27.435224+0000","last_active":"2024-05-17T22:05:28.512451+0000","last_peered":"2024-05-17T22:05:28.512451+0000","last_clean":"2024-05-17T22:05:28.512451+0000","last_became_active":"2024-05-17T22:05:27.434335+0000","last_became_peered":"2024-05-17T22:05:27.434335+0000","last_unstale":"2024-05-17T22:05:28.512451+0000","last_undegraded":"2024-05-17T22:05:28.512451+0000","last_fullsized":"2024-05-17T22:05:28.512451+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:30.767567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510903+0000","last_change":"2024-05-17T22:05:27.435224+0000","last_active":"2024-05-17T22:05:28.510903+0000","last_peered":"2024-05-17T22:05:28.510903+0000","last_clean":"2024-05-17T22:05:28.510903+0000","last_became_active":"2024-05-17T22:05:27.434352+0000","last_became_peered":"2024-05-17T22:05:27.434352+0000","last_unstale":"2024-05-17T22:05:28.510903+0000","last_undegraded":"2024-05-17T22:05:28.510903+0000","last_fullsized":"2024-05-17T22:05:28.510903+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465270+0000","last_change":"2024-05-17T22:05:27.432637+0000","last_active":"2024-05-17T22:05:28.465270+0000","last_peered":"2024-05-17T22:05:28.465270+0000","last_clean":"2024-05-17T22:05:28.465270+0000","last_became_active":"2024-05-17T22:05:27.431831+0000","last_became_peered":"2024-05-17T22:05:27.431831+0000","last_unstale":"2024-05-17T22:05:28.465270+0000","last_undegraded":"2024-05-17T22:05:28.465270+0000","last_fullsized":"2024-05-17T22:05:28.465270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:40.459967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412923+0000","last_change":"2024-05-17T22:05:27.445862+0000","last_active":"2024-05-17T22:05:28.412923+0000","last_peered":"2024-05-17T22:05:28.412923+0000","last_clean":"2024-05-17T22:05:28.412923+0000","last_became_active":"2024-05-17T22:05:27.444739+0000","last_became_peered":"2024-05-17T22:05:27.444739+0000","last_unstale":"2024-05-17T22:05:28.412923+0000","last_undegraded":"2024-05-17T22:05:28.412923+0000","last_fullsized":"2024-05-17T22:05:28.412923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:32.068585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511531+0000","last_change":"2024-05-17T22:05:27.442733+0000","last_active":"2024-05-17T22:05:28.511531+0000","last_peered":"2024-05-17T22:05:28.511531+0000","last_clean":"2024-05-17T22:05:28.511531+0000","last_became_active":"2024-05-17T22:05:27.441844+0000","last_became_peered":"2024-05-17T22:05:27.441844+0000","last_unstale":"2024-05-17T22:05:28.511531+0000","last_undegraded":"2024-05-17T22:05:28.511531+0000","last_fullsized":"2024-05-17T22:05:28.511531+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511456+0000","last_change":"2024-05-17T22:05:27.432985+0000","last_active":"2024-05-17T22:05:28.511456+0000","last_peered":"2024-05-17T22:05:28.511456+0000","last_clean":"2024-05-17T22:05:28.511456+0000","last_became_active":"2024-05-17T22:05:27.432071+0000","last_became_peered":"2024-05-17T22:05:27.432071+0000","last_unstale":"2024-05-17T22:05:28.511456+0000","last_undegraded":"2024-05-17T22:05:28.511456+0000","last_fullsized":"2024-05-17T22:05:28.511456+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413841+0000","last_change":"2024-05-17T22:05:27.427483+0000","last_active":"2024-05-17T22:05:28.413841+0000","last_peered":"2024-05-17T22:05:28.413841+0000","last_clean":"2024-05-17T22:05:28.413841+0000","last_became_active":"2024-05-17T22:05:27.426574+0000","last_became_peered":"2024-05-17T22:05:27.426574+0000","last_unstale":"2024-05-17T22:05:28.413841+0000","last_undegraded":"2024-05-17T22:05:28.413841+0000","last_fullsized":"2024-05-17T22:05:28.413841+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512355+0000","last_change":"2024-05-17T22:05:27.439750+0000","last_active":"2024-05-17T22:05:28.512355+0000","last_peered":"2024-05-17T22:05:28.512355+0000","last_clean":"2024-05-17T22:05:28.512355+0000","last_became_active":"2024-05-17T22:05:27.437212+0000","last_became_peered":"2024-05-17T22:05:27.437212+0000","last_unstale":"2024-05-17T22:05:28.512355+0000","last_undegraded":"2024-05-17T22:05:28.512355+0000","last_fullsized":"2024-05-17T22:05:28.512355+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412116+0000","last_change":"2024-05-17T22:05:27.451726+0000","last_active":"2024-05-17T22:05:28.412116+0000","last_peered":"2024-05-17T22:05:28.412116+0000","last_clean":"2024-05-17T22:05:28.412116+0000","last_became_active":"2024-05-17T22:05:27.450823+0000","last_became_peered":"2024-05-17T22:05:27.450823+0000","last_unstale":"2024-05-17T22:05:28.412116+0000","last_undegraded":"2024-05-17T22:05:28.412116+0000","last_fullsized":"2024-05-17T22:05:28.412116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:16:00.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511271+0000","last_change":"2024-05-17T22:05:27.448190+0000","last_active":"2024-05-17T22:05:28.511271+0000","last_peered":"2024-05-17T22:05:28.511271+0000","last_clean":"2024-05-17T22:05:28.511271+0000","last_became_active":"2024-05-17T22:05:27.447608+0000","last_became_peered":"2024-05-17T22:05:27.447608+0000","last_unstale":"2024-05-17T22:05:28.511271+0000","last_undegraded":"2024-05-17T22:05:28.511271+0000","last_fullsized":"2024-05-17T22:05:28.511271+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510847+0000","last_change":"2024-05-17T22:05:27.447395+0000","last_active":"2024-05-17T22:05:28.510847+0000","last_peered":"2024-05-17T22:05:28.510847+0000","last_clean":"2024-05-17T22:05:28.510847+0000","last_became_active":"2024-05-17T22:05:27.446457+0000","last_became_peered":"2024-05-17T22:05:27.446457+0000","last_unstale":"2024-05-17T22:05:28.510847+0000","last_undegraded":"2024-05-17T22:05:28.510847+0000","last_fullsized":"2024-05-17T22:05:28.510847+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465962+0000","last_change":"2024-05-17T22:05:27.447783+0000","last_active":"2024-05-17T22:05:28.465962+0000","last_peered":"2024-05-17T22:05:28.465962+0000","last_clean":"2024-05-17T22:05:28.465962+0000","last_became_active":"2024-05-17T22:05:27.446586+0000","last_became_peered":"2024-05-17T22:05:27.446586+0000","last_unstale":"2024-05-17T22:05:28.465962+0000","last_undegraded":"2024-05-17T22:05:28.465962+0000","last_fullsized":"2024-05-17T22:05:28.465962+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:44.123521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512117+0000","last_change":"2024-05-17T22:05:27.446975+0000","last_active":"2024-05-17T22:05:28.512117+0000","last_peered":"2024-05-17T22:05:28.512117+0000","last_clean":"2024-05-17T22:05:28.512117+0000","last_became_active":"2024-05-17T22:05:27.446058+0000","last_became_peered":"2024-05-17T22:05:27.446058+0000","last_unstale":"2024-05-17T22:05:28.512117+0000","last_undegraded":"2024-05-17T22:05:28.512117+0000","last_fullsized":"2024-05-17T22:05:28.512117+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412800+0000","last_change":"2024-05-17T22:05:27.426973+0000","last_active":"2024-05-17T22:05:28.412800+0000","last_peered":"2024-05-17T22:05:28.412800+0000","last_clean":"2024-05-17T22:05:28.412800+0000","last_became_active":"2024-05-17T22:05:27.422435+0000","last_became_peered":"2024-05-17T22:05:27.422435+0000","last_unstale":"2024-05-17T22:05:28.412800+0000","last_undegraded":"2024-05-17T22:05:28.412800+0000","last_fullsized":"2024-05-17T22:05:28.412800+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:27:35.606607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414698+0000","last_change":"2024-05-17T22:05:27.446147+0000","last_active":"2024-05-17T22:05:28.414698+0000","last_peered":"2024-05-17T22:05:28.414698+0000","last_clean":"2024-05-17T22:05:28.414698+0000","last_became_active":"2024-05-17T22:05:27.445228+0000","last_became_peered":"2024-05-17T22:05:27.445228+0000","last_unstale":"2024-05-17T22:05:28.414698+0000","last_undegraded":"2024-05-17T22:05:28.414698+0000","last_fullsized":"2024-05-17T22:05:28.414698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412238+0000","last_change":"2024-05-17T22:05:27.411491+0000","last_active":"2024-05-17T22:05:28.412238+0000","last_peered":"2024-05-17T22:05:28.412238+0000","last_clean":"2024-05-17T22:05:28.412238+0000","last_became_active":"2024-05-17T22:05:27.410584+0000","last_became_peered":"2024-05-17T22:05:27.410584+0000","last_unstale":"2024-05-17T22:05:28.412238+0000","last_undegraded":"2024-05-17T22:05:28.412238+0000","last_fullsized":"2024-05-17T22:05:28.412238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:06.944584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465400+0000","last_change":"2024-05-17T22:05:27.436719+0000","last_active":"2024-05-17T22:05:28.465400+0000","last_peered":"2024-05-17T22:05:28.465400+0000","last_clean":"2024-05-17T22:05:28.465400+0000","last_became_active":"2024-05-17T22:05:27.435870+0000","last_became_peered":"2024-05-17T22:05:27.435870+0000","last_unstale":"2024-05-17T22:05:28.465400+0000","last_undegraded":"2024-05-17T22:05:28.465400+0000","last_fullsized":"2024-05-17T22:05:28.465400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:01:24.876664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465295+0000","last_change":"2024-05-17T22:05:27.435595+0000","last_active":"2024-05-17T22:05:28.465295+0000","last_peered":"2024-05-17T22:05:28.465295+0000","last_clean":"2024-05-17T22:05:28.465295+0000","last_became_active":"2024-05-17T22:05:27.434786+0000","last_became_peered":"2024-05-17T22:05:27.434786+0000","last_unstale":"2024-05-17T22:05:28.465295+0000","last_undegraded":"2024-05-17T22:05:28.465295+0000","last_fullsized":"2024-05-17T22:05:28.465295+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:02:32.030203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510968+0000","last_change":"2024-05-17T22:05:27.429067+0000","last_active":"2024-05-17T22:05:28.510968+0000","last_peered":"2024-05-17T22:05:28.510968+0000","last_clean":"2024-05-17T22:05:28.510968+0000","last_became_active":"2024-05-17T22:05:27.428004+0000","last_became_peered":"2024-05-17T22:05:27.428004+0000","last_unstale":"2024-05-17T22:05:28.510968+0000","last_undegraded":"2024-05-17T22:05:28.510968+0000","last_fullsized":"2024-05-17T22:05:28.510968+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512972+0000","last_change":"2024-05-17T22:05:27.450609+0000","last_active":"2024-05-17T22:05:28.512972+0000","last_peered":"2024-05-17T22:05:28.512972+0000","last_clean":"2024-05-17T22:05:28.512972+0000","last_became_active":"2024-05-17T22:05:27.449075+0000","last_became_peered":"2024-05-17T22:05:27.449075+0000","last_unstale":"2024-05-17T22:05:28.512972+0000","last_undegraded":"2024-05-17T22:05:28.512972+0000","last_fullsized":"2024-05-17T22:05:28.512972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465769+0000","last_change":"2024-05-17T22:05:27.454186+0000","last_active":"2024-05-17T22:05:28.465769+0000","last_peered":"2024-05-17T22:05:28.465769+0000","last_clean":"2024-05-17T22:05:28.465769+0000","last_became_active":"2024-05-17T22:05:27.448403+0000","last_became_peered":"2024-05-17T22:05:27.448403+0000","last_unstale":"2024-05-17T22:05:28.465769+0000","last_undegraded":"2024-05-17T22:05:28.465769+0000","last_fullsized":"2024-05-17T22:05:28.465769+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:29:54.556862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512026+0000","last_change":"2024-05-17T22:05:27.442102+0000","last_active":"2024-05-17T22:05:28.512026+0000","last_peered":"2024-05-17T22:05:28.512026+0000","last_clean":"2024-05-17T22:05:28.512026+0000","last_became_active":"2024-05-17T22:05:27.441268+0000","last_became_peered":"2024-05-17T22:05:27.441268+0000","last_unstale":"2024-05-17T22:05:28.512026+0000","last_undegraded":"2024-05-17T22:05:28.512026+0000","last_fullsized":"2024-05-17T22:05:28.512026+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512881+0000","last_change":"2024-05-17T22:05:27.450220+0000","last_active":"2024-05-17T22:05:28.512881+0000","last_peered":"2024-05-17T22:05:28.512881+0000","last_clean":"2024-05-17T22:05:28.512881+0000","last_became_active":"2024-05-17T22:05:27.449522+0000","last_became_peered":"2024-05-17T22:05:27.449522+0000","last_unstale":"2024-05-17T22:05:28.512881+0000","last_undegraded":"2024-05-17T22:05:28.512881+0000","last_fullsized":"2024-05-17T22:05:28.512881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465332+0000","last_change":"2024-05-17T22:05:27.445831+0000","last_active":"2024-05-17T22:05:28.465332+0000","last_peered":"2024-05-17T22:05:28.465332+0000","last_clean":"2024-05-17T22:05:28.465332+0000","last_became_active":"2024-05-17T22:05:27.445032+0000","last_became_peered":"2024-05-17T22:05:27.445032+0000","last_unstale":"2024-05-17T22:05:28.465332+0000","last_undegraded":"2024-05-17T22:05:28.465332+0000","last_fullsized":"2024-05-17T22:05:28.465332+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:01:10.338332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412160+0000","last_change":"2024-05-17T22:05:27.426891+0000","last_active":"2024-05-17T22:05:28.412160+0000","last_peered":"2024-05-17T22:05:28.412160+0000","last_clean":"2024-05-17T22:05:28.412160+0000","last_became_active":"2024-05-17T22:05:27.425979+0000","last_became_peered":"2024-05-17T22:05:27.425979+0000","last_unstale":"2024-05-17T22:05:28.412160+0000","last_undegraded":"2024-05-17T22:05:28.412160+0000","last_fullsized":"2024-05-17T22:05:28.412160+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:42:53.395537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511522+0000","last_change":"2024-05-17T22:05:27.457824+0000","last_active":"2024-05-17T22:05:28.511522+0000","last_peered":"2024-05-17T22:05:28.511522+0000","last_clean":"2024-05-17T22:05:28.511522+0000","last_became_active":"2024-05-17T22:05:27.457211+0000","last_became_peered":"2024-05-17T22:05:27.457211+0000","last_unstale":"2024-05-17T22:05:28.511522+0000","last_undegraded":"2024-05-17T22:05:28.511522+0000","last_fullsized":"2024-05-17T22:05:28.511522+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510468+0000","last_change":"2024-05-17T22:05:27.448441+0000","last_active":"2024-05-17T22:05:28.510468+0000","last_peered":"2024-05-17T22:05:28.510468+0000","last_clean":"2024-05-17T22:05:28.510468+0000","last_became_active":"2024-05-17T22:05:27.446781+0000","last_became_peered":"2024-05-17T22:05:27.446781+0000","last_unstale":"2024-05-17T22:05:28.510468+0000","last_undegraded":"2024-05-17T22:05:28.510468+0000","last_fullsized":"2024-05-17T22:05:28.510468+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:50:46.664220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412763+0000","last_change":"2024-05-17T22:05:27.433888+0000","last_active":"2024-05-17T22:05:28.412763+0000","last_peered":"2024-05-17T22:05:28.412763+0000","last_clean":"2024-05-17T22:05:28.412763+0000","last_became_active":"2024-05-17T22:05:27.431384+0000","last_became_peered":"2024-05-17T22:05:27.431384+0000","last_unstale":"2024-05-17T22:05:28.412763+0000","last_undegraded":"2024-05-17T22:05:28.412763+0000","last_fullsized":"2024-05-17T22:05:28.412763+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:37:49.769320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465848+0000","last_change":"2024-05-17T22:05:26.410331+0000","last_active":"2024-05-17T22:05:28.465848+0000","last_peered":"2024-05-17T22:05:28.465848+0000","last_clean":"2024-05-17T22:05:28.465848+0000","last_became_active":"2024-05-17T22:05:26.409867+0000","last_became_peered":"2024-05-17T22:05:26.409867+0000","last_unstale":"2024-05-17T22:05:28.465848+0000","last_undegraded":"2024-05-17T22:05:28.465848+0000","last_fullsized":"2024-05-17T22:05:28.465848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:03:42.413177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512635+0000","last_change":"2024-05-17T22:05:27.457988+0000","last_active":"2024-05-17T22:05:28.512635+0000","last_peered":"2024-05-17T22:05:28.512635+0000","last_clean":"2024-05-17T22:05:28.512635+0000","last_became_active":"2024-05-17T22:05:27.457287+0000","last_became_peered":"2024-05-17T22:05:27.457287+0000","last_unstale":"2024-05-17T22:05:28.512635+0000","last_undegraded":"2024-05-17T22:05:28.512635+0000","last_fullsized":"2024-05-17T22:05:28.512635+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511734+0000","last_change":"2024-05-17T22:05:26.400449+0000","last_active":"2024-05-17T22:05:28.511734+0000","last_peered":"2024-05-17T22:05:28.511734+0000","last_clean":"2024-05-17T22:05:28.511734+0000","last_became_active":"2024-05-17T22:05:26.399787+0000","last_became_peered":"2024-05-17T22:05:26.399787+0000","last_unstale":"2024-05-17T22:05:28.511734+0000","last_undegraded":"2024-05-17T22:05:28.511734+0000","last_fullsized":"2024-05-17T22:05:28.511734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511204+0000","last_change":"2024-05-17T22:05:27.434317+0000","last_active":"2024-05-17T22:05:28.511204+0000","last_peered":"2024-05-17T22:05:28.511204+0000","last_clean":"2024-05-17T22:05:28.511204+0000","last_became_active":"2024-05-17T22:05:27.432757+0000","last_became_peered":"2024-05-17T22:05:27.432757+0000","last_unstale":"2024-05-17T22:05:28.511204+0000","last_undegraded":"2024-05-17T22:05:28.511204+0000","last_fullsized":"2024-05-17T22:05:28.511204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:41.298302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465624+0000","last_change":"2024-05-17T22:05:26.380122+0000","last_active":"2024-05-17T22:05:28.465624+0000","last_peered":"2024-05-17T22:05:28.465624+0000","last_clean":"2024-05-17T22:05:28.465624+0000","last_became_active":"2024-05-17T22:05:26.379777+0000","last_became_peered":"2024-05-17T22:05:26.379777+0000","last_unstale":"2024-05-17T22:05:28.465624+0000","last_undegraded":"2024-05-17T22:05:28.465624+0000","last_fullsized":"2024-05-17T22:05:28.465624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:57.468540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412827+0000","last_change":"2024-05-17T22:05:27.414672+0000","last_active":"2024-05-17T22:05:28.412827+0000","last_peered":"2024-05-17T22:05:28.412827+0000","last_clean":"2024-05-17T22:05:28.412827+0000","last_became_active":"2024-05-17T22:05:27.413380+0000","last_became_peered":"2024-05-17T22:05:27.413380+0000","last_unstale":"2024-05-17T22:05:28.412827+0000","last_undegraded":"2024-05-17T22:05:28.412827+0000","last_fullsized":"2024-05-17T22:05:28.412827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:00:43.215036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465652+0000","last_change":"2024-05-17T22:05:26.394737+0000","last_active":"2024-05-17T22:05:28.465652+0000","last_peered":"2024-05-17T22:05:28.465652+0000","last_clean":"2024-05-17T22:05:28.465652+0000","last_became_active":"2024-05-17T22:05:26.394330+0000","last_became_peered":"2024-05-17T22:05:26.394330+0000","last_unstale":"2024-05-17T22:05:28.465652+0000","last_undegraded":"2024-05-17T22:05:28.465652+0000","last_fullsized":"2024-05-17T22:05:28.465652+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413501+0000","last_change":"2024-05-17T22:05:27.427588+0000","last_active":"2024-05-17T22:05:28.413501+0000","last_peered":"2024-05-17T22:05:28.413501+0000","last_clean":"2024-05-17T22:05:28.413501+0000","last_became_active":"2024-05-17T22:05:27.426388+0000","last_became_peered":"2024-05-17T22:05:27.426388+0000","last_unstale":"2024-05-17T22:05:28.413501+0000","last_undegraded":"2024-05-17T22:05:28.413501+0000","last_fullsized":"2024-05-17T22:05:28.413501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:11:40.445555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511137+0000","last_change":"2024-05-17T22:05:26.387580+0000","last_active":"2024-05-17T22:05:28.511137+0000","last_peered":"2024-05-17T22:05:28.511137+0000","last_clean":"2024-05-17T22:05:28.511137+0000","last_became_active":"2024-05-17T22:05:26.387218+0000","last_became_peered":"2024-05-17T22:05:26.387218+0000","last_unstale":"2024-05-17T22:05:28.511137+0000","last_undegraded":"2024-05-17T22:05:28.511137+0000","last_fullsized":"2024-05-17T22:05:28.511137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:44:00.680001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512541+0000","last_change":"2024-05-17T22:05:27.450229+0000","last_active":"2024-05-17T22:05:28.512541+0000","last_peered":"2024-05-17T22:05:28.512541+0000","last_clean":"2024-05-17T22:05:28.512541+0000","last_became_active":"2024-05-17T22:05:27.449394+0000","last_became_peered":"2024-05-17T22:05:27.449394+0000","last_unstale":"2024-05-17T22:05:28.512541+0000","last_undegraded":"2024-05-17T22:05:28.512541+0000","last_fullsized":"2024-05-17T22:05:28.512541+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:40.459967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465421+0000","last_change":"2024-05-17T22:05:26.381762+0000","last_active":"2024-05-17T22:05:28.465421+0000","last_peered":"2024-05-17T22:05:28.465421+0000","last_clean":"2024-05-17T22:05:28.465421+0000","last_became_active":"2024-05-17T22:05:26.379895+0000","last_became_peered":"2024-05-17T22:05:26.379895+0000","last_unstale":"2024-05-17T22:05:28.465421+0000","last_undegraded":"2024-05-17T22:05:28.465421+0000","last_fullsized":"2024-05-17T22:05:28.465421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:41:58.027527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511161+0000","last_change":"2024-05-17T22:05:27.436384+0000","last_active":"2024-05-17T22:05:28.511161+0000","last_peered":"2024-05-17T22:05:28.511161+0000","last_clean":"2024-05-17T22:05:28.511161+0000","last_became_active":"2024-05-17T22:05:27.435594+0000","last_became_peered":"2024-05-17T22:05:27.435594+0000","last_unstale":"2024-05-17T22:05:28.511161+0000","last_undegraded":"2024-05-17T22:05:28.511161+0000","last_fullsized":"2024-05-17T22:05:28.511161+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512221+0000","last_change":"2024-05-17T22:05:26.407612+0000","last_active":"2024-05-17T22:05:28.512221+0000","last_peered":"2024-05-17T22:05:28.512221+0000","last_clean":"2024-05-17T22:05:28.512221+0000","last_became_active":"2024-05-17T22:05:26.406766+0000","last_became_peered":"2024-05-17T22:05:26.406766+0000","last_unstale":"2024-05-17T22:05:28.512221+0000","last_undegraded":"2024-05-17T22:05:28.512221+0000","last_fullsized":"2024-05-17T22:05:28.512221+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:42:55.218241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411948+0000","last_change":"2024-05-17T22:05:27.414050+0000","last_active":"2024-05-17T22:05:28.411948+0000","last_peered":"2024-05-17T22:05:28.411948+0000","last_clean":"2024-05-17T22:05:28.411948+0000","last_became_active":"2024-05-17T22:05:27.413156+0000","last_became_peered":"2024-05-17T22:05:27.413156+0000","last_unstale":"2024-05-17T22:05:28.411948+0000","last_undegraded":"2024-05-17T22:05:28.411948+0000","last_fullsized":"2024-05-17T22:05:28.411948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:41:59.038541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510429+0000","last_change":"2024-05-17T22:05:26.389010+0000","last_active":"2024-05-17T22:05:28.510429+0000","last_peered":"2024-05-17T22:05:28.510429+0000","last_clean":"2024-05-17T22:05:28.510429+0000","last_became_active":"2024-05-17T22:05:26.387817+0000","last_became_peered":"2024-05-17T22:05:26.387817+0000","last_unstale":"2024-05-17T22:05:28.510429+0000","last_undegraded":"2024-05-17T22:05:28.510429+0000","last_fullsized":"2024-05-17T22:05:28.510429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:08:39.375715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465884+0000","last_change":"2024-05-17T22:05:27.453658+0000","last_active":"2024-05-17T22:05:28.465884+0000","last_peered":"2024-05-17T22:05:28.465884+0000","last_clean":"2024-05-17T22:05:28.465884+0000","last_became_active":"2024-05-17T22:05:27.447524+0000","last_became_peered":"2024-05-17T22:05:27.447524+0000","last_unstale":"2024-05-17T22:05:28.465884+0000","last_undegraded":"2024-05-17T22:05:28.465884+0000","last_fullsized":"2024-05-17T22:05:28.465884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:40.661750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511659+0000","last_change":"2024-05-17T22:05:26.376826+0000","last_active":"2024-05-17T22:05:28.511659+0000","last_peered":"2024-05-17T22:05:28.511659+0000","last_clean":"2024-05-17T22:05:28.511659+0000","last_became_active":"2024-05-17T22:05:26.376471+0000","last_became_peered":"2024-05-17T22:05:26.376471+0000","last_unstale":"2024-05-17T22:05:28.511659+0000","last_undegraded":"2024-05-17T22:05:28.511659+0000","last_fullsized":"2024-05-17T22:05:28.511659+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:06.705185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413318+0000","last_change":"2024-05-17T22:05:27.414509+0000","last_active":"2024-05-17T22:05:28.413318+0000","last_peered":"2024-05-17T22:05:28.413318+0000","last_clean":"2024-05-17T22:05:28.413318+0000","last_became_active":"2024-05-17T22:05:27.413575+0000","last_became_peered":"2024-05-17T22:05:27.413575+0000","last_unstale":"2024-05-17T22:05:28.413318+0000","last_undegraded":"2024-05-17T22:05:28.413318+0000","last_fullsized":"2024-05-17T22:05:28.413318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:20.803666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512414+0000","last_change":"2024-05-17T22:05:26.376395+0000","last_active":"2024-05-17T22:05:28.512414+0000","last_peered":"2024-05-17T22:05:28.512414+0000","last_clean":"2024-05-17T22:05:28.512414+0000","last_became_active":"2024-05-17T22:05:26.375768+0000","last_became_peered":"2024-05-17T22:05:26.375768+0000","last_unstale":"2024-05-17T22:05:28.512414+0000","last_undegraded":"2024-05-17T22:05:28.512414+0000","last_fullsized":"2024-05-17T22:05:28.512414+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513086+0000","last_change":"2024-05-17T22:05:27.449618+0000","last_active":"2024-05-17T22:05:28.513086+0000","last_peered":"2024-05-17T22:05:28.513086+0000","last_clean":"2024-05-17T22:05:28.513086+0000","last_became_active":"2024-05-17T22:05:27.448834+0000","last_became_peered":"2024-05-17T22:05:27.448834+0000","last_unstale":"2024-05-17T22:05:28.513086+0000","last_undegraded":"2024-05-17T22:05:28.513086+0000","last_fullsized":"2024-05-17T22:05:28.513086+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:30.767567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512300+0000","last_change":"2024-05-17T22:05:26.407057+0000","last_active":"2024-05-17T22:05:28.512300+0000","last_peered":"2024-05-17T22:05:28.512300+0000","last_clean":"2024-05-17T22:05:28.512300+0000","last_became_active":"2024-05-17T22:05:26.406287+0000","last_became_peered":"2024-05-17T22:05:26.406287+0000","last_unstale":"2024-05-17T22:05:28.512300+0000","last_undegraded":"2024-05-17T22:05:28.512300+0000","last_fullsized":"2024-05-17T22:05:28.512300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511917+0000","last_change":"2024-05-17T22:05:27.445181+0000","last_active":"2024-05-17T22:05:28.511917+0000","last_peered":"2024-05-17T22:05:28.511917+0000","last_clean":"2024-05-17T22:05:28.511917+0000","last_became_active":"2024-05-17T22:05:27.444401+0000","last_became_peered":"2024-05-17T22:05:27.444401+0000","last_unstale":"2024-05-17T22:05:28.511917+0000","last_undegraded":"2024-05-17T22:05:28.511917+0000","last_fullsized":"2024-05-17T22:05:28.511917+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414577+0000","last_change":"2024-05-17T22:05:26.392154+0000","last_active":"2024-05-17T22:05:28.414577+0000","last_peered":"2024-05-17T22:05:28.414577+0000","last_clean":"2024-05-17T22:05:28.414577+0000","last_became_active":"2024-05-17T22:05:26.390754+0000","last_became_peered":"2024-05-17T22:05:26.390754+0000","last_unstale":"2024-05-17T22:05:28.414577+0000","last_undegraded":"2024-05-17T22:05:28.414577+0000","last_fullsized":"2024-05-17T22:05:28.414577+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:14.911182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511478+0000","last_change":"2024-05-17T22:05:27.441163+0000","last_active":"2024-05-17T22:05:28.511478+0000","last_peered":"2024-05-17T22:05:28.511478+0000","last_clean":"2024-05-17T22:05:28.511478+0000","last_became_active":"2024-05-17T22:05:27.439993+0000","last_became_peered":"2024-05-17T22:05:27.439993+0000","last_unstale":"2024-05-17T22:05:28.511478+0000","last_undegraded":"2024-05-17T22:05:28.511478+0000","last_fullsized":"2024-05-17T22:05:28.511478+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412388+0000","last_change":"2024-05-17T22:05:26.397019+0000","last_active":"2024-05-17T22:05:28.412388+0000","last_peered":"2024-05-17T22:05:28.412388+0000","last_clean":"2024-05-17T22:05:28.412388+0000","last_became_active":"2024-05-17T22:05:26.396484+0000","last_became_peered":"2024-05-17T22:05:26.396484+0000","last_unstale":"2024-05-17T22:05:28.412388+0000","last_undegraded":"2024-05-17T22:05:28.412388+0000","last_fullsized":"2024-05-17T22:05:28.412388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:45:11.894103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510850+0000","last_change":"2024-05-17T22:05:27.439006+0000","last_active":"2024-05-17T22:05:28.510850+0000","last_peered":"2024-05-17T22:05:28.510850+0000","last_clean":"2024-05-17T22:05:28.510850+0000","last_became_active":"2024-05-17T22:05:27.434400+0000","last_became_peered":"2024-05-17T22:05:27.434400+0000","last_unstale":"2024-05-17T22:05:28.510850+0000","last_undegraded":"2024-05-17T22:05:28.510850+0000","last_fullsized":"2024-05-17T22:05:28.510850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:53:22.603437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412646+0000","last_change":"2024-05-17T22:05:26.381773+0000","last_active":"2024-05-17T22:05:28.412646+0000","last_peered":"2024-05-17T22:05:28.412646+0000","last_clean":"2024-05-17T22:05:28.412646+0000","last_became_active":"2024-05-17T22:05:26.380956+0000","last_became_peered":"2024-05-17T22:05:26.380956+0000","last_unstale":"2024-05-17T22:05:28.412646+0000","last_undegraded":"2024-05-17T22:05:28.412646+0000","last_fullsized":"2024-05-17T22:05:28.412646+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:22.035385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411707+0000","last_change":"2024-05-17T22:05:27.414342+0000","last_active":"2024-05-17T22:05:28.411707+0000","last_peered":"2024-05-17T22:05:28.411707+0000","last_clean":"2024-05-17T22:05:28.411707+0000","last_became_active":"2024-05-17T22:05:27.413415+0000","last_became_peered":"2024-05-17T22:05:27.413415+0000","last_unstale":"2024-05-17T22:05:28.411707+0000","last_undegraded":"2024-05-17T22:05:28.411707+0000","last_fullsized":"2024-05-17T22:05:28.411707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:03:43.424191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411840+0000","last_change":"2024-05-17T22:05:26.411036+0000","last_active":"2024-05-17T22:05:28.411840+0000","last_peered":"2024-05-17T22:05:28.411840+0000","last_clean":"2024-05-17T22:05:28.411840+0000","last_became_active":"2024-05-17T22:05:26.410159+0000","last_became_peered":"2024-05-17T22:05:26.410159+0000","last_unstale":"2024-05-17T22:05:28.411840+0000","last_undegraded":"2024-05-17T22:05:28.411840+0000","last_fullsized":"2024-05-17T22:05:28.411840+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:41.129609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412115+0000","last_change":"2024-05-17T22:05:27.414057+0000","last_active":"2024-05-17T22:05:28.412115+0000","last_peered":"2024-05-17T22:05:28.412115+0000","last_clean":"2024-05-17T22:05:28.412115+0000","last_became_active":"2024-05-17T22:05:27.413128+0000","last_became_peered":"2024-05-17T22:05:27.413128+0000","last_unstale":"2024-05-17T22:05:28.412115+0000","last_undegraded":"2024-05-17T22:05:28.412115+0000","last_fullsized":"2024-05-17T22:05:28.412115+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:02:57.723926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510741+0000","last_change":"2024-05-17T22:05:26.397262+0000","last_active":"2024-05-17T22:05:28.510741+0000","last_peered":"2024-05-17T22:05:28.510741+0000","last_clean":"2024-05-17T22:05:28.510741+0000","last_became_active":"2024-05-17T22:05:26.396653+0000","last_became_peered":"2024-05-17T22:05:26.396653+0000","last_unstale":"2024-05-17T22:05:28.510741+0000","last_undegraded":"2024-05-17T22:05:28.510741+0000","last_fullsized":"2024-05-17T22:05:28.510741+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:58:53.828477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513083+0000","last_change":"2024-05-17T22:05:27.450951+0000","last_active":"2024-05-17T22:05:28.513083+0000","last_peered":"2024-05-17T22:05:28.513083+0000","last_clean":"2024-05-17T22:05:28.513083+0000","last_became_active":"2024-05-17T22:05:27.450180+0000","last_became_peered":"2024-05-17T22:05:27.450180+0000","last_unstale":"2024-05-17T22:05:28.513083+0000","last_undegraded":"2024-05-17T22:05:28.513083+0000","last_fullsized":"2024-05-17T22:05:28.513083+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412745+0000","last_change":"2024-05-17T22:05:26.400196+0000","last_active":"2024-05-17T22:05:28.412745+0000","last_peered":"2024-05-17T22:05:28.412745+0000","last_clean":"2024-05-17T22:05:28.412745+0000","last_became_active":"2024-05-17T22:05:26.399378+0000","last_became_peered":"2024-05-17T22:05:26.399378+0000","last_unstale":"2024-05-17T22:05:28.412745+0000","last_undegraded":"2024-05-17T22:05:28.412745+0000","last_fullsized":"2024-05-17T22:05:28.412745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:12:20.023566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414178+0000","last_change":"2024-05-17T22:05:27.454581+0000","last_active":"2024-05-17T22:05:28.414178+0000","last_peered":"2024-05-17T22:05:28.414178+0000","last_clean":"2024-05-17T22:05:28.414178+0000","last_became_active":"2024-05-17T22:05:27.454044+0000","last_became_peered":"2024-05-17T22:05:27.454044+0000","last_unstale":"2024-05-17T22:05:28.414178+0000","last_undegraded":"2024-05-17T22:05:28.414178+0000","last_fullsized":"2024-05-17T22:05:28.414178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:42.140624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512337+0000","last_change":"2024-05-17T22:05:26.389173+0000","last_active":"2024-05-17T22:05:28.512337+0000","last_peered":"2024-05-17T22:05:28.512337+0000","last_clean":"2024-05-17T22:05:28.512337+0000","last_became_active":"2024-05-17T22:05:26.388309+0000","last_became_peered":"2024-05-17T22:05:26.388309+0000","last_unstale":"2024-05-17T22:05:28.512337+0000","last_undegraded":"2024-05-17T22:05:28.512337+0000","last_fullsized":"2024-05-17T22:05:28.512337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512379+0000","last_change":"2024-05-17T22:05:27.440184+0000","last_active":"2024-05-17T22:05:28.512379+0000","last_peered":"2024-05-17T22:05:28.512379+0000","last_clean":"2024-05-17T22:05:28.512379+0000","last_became_active":"2024-05-17T22:05:27.439330+0000","last_became_peered":"2024-05-17T22:05:27.439330+0000","last_unstale":"2024-05-17T22:05:28.512379+0000","last_undegraded":"2024-05-17T22:05:28.512379+0000","last_fullsized":"2024-05-17T22:05:28.512379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465663+0000","last_change":"2024-05-17T22:05:26.396565+0000","last_active":"2024-05-17T22:05:28.465663+0000","last_peered":"2024-05-17T22:05:28.465663+0000","last_clean":"2024-05-17T22:05:28.465663+0000","last_became_active":"2024-05-17T22:05:26.396040+0000","last_became_peered":"2024-05-17T22:05:26.396040+0000","last_unstale":"2024-05-17T22:05:28.465663+0000","last_undegraded":"2024-05-17T22:05:28.465663+0000","last_fullsized":"2024-05-17T22:05:28.465663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:33:32.933119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412834+0000","last_change":"2024-05-17T22:05:27.449626+0000","last_active":"2024-05-17T22:05:28.412834+0000","last_peered":"2024-05-17T22:05:28.412834+0000","last_clean":"2024-05-17T22:05:28.412834+0000","last_became_active":"2024-05-17T22:05:27.448722+0000","last_became_peered":"2024-05-17T22:05:27.448722+0000","last_unstale":"2024-05-17T22:05:28.412834+0000","last_undegraded":"2024-05-17T22:05:28.412834+0000","last_fullsized":"2024-05-17T22:05:28.412834+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:44:01.691016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465519+0000","last_change":"2024-05-17T22:05:26.386697+0000","last_active":"2024-05-17T22:05:28.465519+0000","last_peered":"2024-05-17T22:05:28.465519+0000","last_clean":"2024-05-17T22:05:28.465519+0000","last_became_active":"2024-05-17T22:05:26.386096+0000","last_became_peered":"2024-05-17T22:05:26.386096+0000","last_unstale":"2024-05-17T22:05:28.465519+0000","last_undegraded":"2024-05-17T22:05:28.465519+0000","last_fullsized":"2024-05-17T22:05:28.465519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:00:42.204022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412633+0000","last_change":"2024-05-17T22:05:27.452521+0000","last_active":"2024-05-17T22:05:28.412633+0000","last_peered":"2024-05-17T22:05:28.412633+0000","last_clean":"2024-05-17T22:05:28.412633+0000","last_became_active":"2024-05-17T22:05:27.451787+0000","last_became_peered":"2024-05-17T22:05:27.451787+0000","last_unstale":"2024-05-17T22:05:28.412633+0000","last_undegraded":"2024-05-17T22:05:28.412633+0000","last_fullsized":"2024-05-17T22:05:28.412633+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:50:09.041997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413795+0000","last_change":"2024-05-17T22:05:26.377842+0000","last_active":"2024-05-17T22:05:28.413795+0000","last_peered":"2024-05-17T22:05:28.413795+0000","last_clean":"2024-05-17T22:05:28.413795+0000","last_became_active":"2024-05-17T22:05:26.377041+0000","last_became_peered":"2024-05-17T22:05:26.377041+0000","last_unstale":"2024-05-17T22:05:28.413795+0000","last_undegraded":"2024-05-17T22:05:28.413795+0000","last_fullsized":"2024-05-17T22:05:28.413795+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:15.786127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411814+0000","last_change":"2024-05-17T22:05:27.435956+0000","last_active":"2024-05-17T22:05:28.411814+0000","last_peered":"2024-05-17T22:05:28.411814+0000","last_clean":"2024-05-17T22:05:28.411814+0000","last_became_active":"2024-05-17T22:05:27.435123+0000","last_became_peered":"2024-05-17T22:05:27.435123+0000","last_unstale":"2024-05-17T22:05:28.411814+0000","last_undegraded":"2024-05-17T22:05:28.411814+0000","last_fullsized":"2024-05-17T22:05:28.411814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:29.817682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413605+0000","last_change":"2024-05-17T22:05:26.388188+0000","last_active":"2024-05-17T22:05:28.413605+0000","last_peered":"2024-05-17T22:05:28.413605+0000","last_clean":"2024-05-17T22:05:28.413605+0000","last_became_active":"2024-05-17T22:05:26.387288+0000","last_became_peered":"2024-05-17T22:05:26.387288+0000","last_unstale":"2024-05-17T22:05:28.413605+0000","last_undegraded":"2024-05-17T22:05:28.413605+0000","last_fullsized":"2024-05-17T22:05:28.413605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:00.056958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511904+0000","last_change":"2024-05-17T22:05:27.441725+0000","last_active":"2024-05-17T22:05:28.511904+0000","last_peered":"2024-05-17T22:05:28.511904+0000","last_clean":"2024-05-17T22:05:28.511904+0000","last_became_active":"2024-05-17T22:05:27.440926+0000","last_became_peered":"2024-05-17T22:05:27.440926+0000","last_unstale":"2024-05-17T22:05:28.511904+0000","last_undegraded":"2024-05-17T22:05:28.511904+0000","last_fullsized":"2024-05-17T22:05:28.511904+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465327+0000","last_change":"2024-05-17T22:05:26.378034+0000","last_active":"2024-05-17T22:05:28.465327+0000","last_peered":"2024-05-17T22:05:28.465327+0000","last_clean":"2024-05-17T22:05:28.465327+0000","last_became_active":"2024-05-17T22:05:26.377700+0000","last_became_peered":"2024-05-17T22:05:26.377700+0000","last_unstale":"2024-05-17T22:05:28.465327+0000","last_undegraded":"2024-05-17T22:05:28.465327+0000","last_fullsized":"2024-05-17T22:05:28.465327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:27:14.293781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.466059+0000","last_change":"2024-05-17T22:05:27.438205+0000","last_active":"2024-05-17T22:05:28.466059+0000","last_peered":"2024-05-17T22:05:28.466059+0000","last_clean":"2024-05-17T22:05:28.466059+0000","last_became_active":"2024-05-17T22:05:27.436508+0000","last_became_peered":"2024-05-17T22:05:27.436508+0000","last_unstale":"2024-05-17T22:05:28.466059+0000","last_undegraded":"2024-05-17T22:05:28.466059+0000","last_fullsized":"2024-05-17T22:05:28.466059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:19.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414299+0000","last_change":"2024-05-17T22:05:26.384952+0000","last_active":"2024-05-17T22:05:28.414299+0000","last_peered":"2024-05-17T22:05:28.414299+0000","last_clean":"2024-05-17T22:05:28.414299+0000","last_became_active":"2024-05-17T22:05:26.383936+0000","last_became_peered":"2024-05-17T22:05:26.383936+0000","last_unstale":"2024-05-17T22:05:28.414299+0000","last_undegraded":"2024-05-17T22:05:28.414299+0000","last_fullsized":"2024-05-17T22:05:28.414299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:45:44.409039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511388+0000","last_change":"2024-05-17T22:05:27.434762+0000","last_active":"2024-05-17T22:05:28.511388+0000","last_peered":"2024-05-17T22:05:28.511388+0000","last_clean":"2024-05-17T22:05:28.511388+0000","last_became_active":"2024-05-17T22:05:27.433884+0000","last_became_peered":"2024-05-17T22:05:27.433884+0000","last_unstale":"2024-05-17T22:05:28.511388+0000","last_undegraded":"2024-05-17T22:05:28.511388+0000","last_fullsized":"2024-05-17T22:05:28.511388+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:15:25.227504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.467573+0000","last_change":"2024-05-17T22:05:26.384752+0000","last_active":"2024-05-17T22:05:28.467573+0000","last_peered":"2024-05-17T22:05:28.467573+0000","last_clean":"2024-05-17T22:05:28.467573+0000","last_became_active":"2024-05-17T22:05:26.382500+0000","last_became_peered":"2024-05-17T22:05:26.382500+0000","last_unstale":"2024-05-17T22:05:28.467573+0000","last_undegraded":"2024-05-17T22:05:28.467573+0000","last_fullsized":"2024-05-17T22:05:28.467573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:53.887504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511601+0000","last_change":"2024-05-17T22:05:27.455421+0000","last_active":"2024-05-17T22:05:28.511601+0000","last_peered":"2024-05-17T22:05:28.511601+0000","last_clean":"2024-05-17T22:05:28.511601+0000","last_became_active":"2024-05-17T22:05:27.454641+0000","last_became_peered":"2024-05-17T22:05:27.454641+0000","last_unstale":"2024-05-17T22:05:28.511601+0000","last_undegraded":"2024-05-17T22:05:28.511601+0000","last_fullsized":"2024-05-17T22:05:28.511601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:15:23.998957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465893+0000","last_change":"2024-05-17T22:05:26.393401+0000","last_active":"2024-05-17T22:05:28.465893+0000","last_peered":"2024-05-17T22:05:28.465893+0000","last_clean":"2024-05-17T22:05:28.465893+0000","last_became_active":"2024-05-17T22:05:26.392987+0000","last_became_peered":"2024-05-17T22:05:26.392987+0000","last_unstale":"2024-05-17T22:05:28.465893+0000","last_undegraded":"2024-05-17T22:05:28.465893+0000","last_fullsized":"2024-05-17T22:05:28.465893+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:58:32.953243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512007+0000","last_change":"2024-05-17T22:05:27.444076+0000","last_active":"2024-05-17T22:05:28.512007+0000","last_peered":"2024-05-17T22:05:28.512007+0000","last_clean":"2024-05-17T22:05:28.512007+0000","last_became_active":"2024-05-17T22:05:27.443260+0000","last_became_peered":"2024-05-17T22:05:27.443260+0000","last_unstale":"2024-05-17T22:05:28.512007+0000","last_undegraded":"2024-05-17T22:05:28.512007+0000","last_fullsized":"2024-05-17T22:05:28.512007+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513059+0000","last_change":"2024-05-17T22:05:26.406964+0000","last_active":"2024-05-17T22:05:28.513059+0000","last_peered":"2024-05-17T22:05:28.513059+0000","last_clean":"2024-05-17T22:05:28.513059+0000","last_became_active":"2024-05-17T22:05:26.406023+0000","last_became_peered":"2024-05-17T22:05:26.406023+0000","last_unstale":"2024-05-17T22:05:28.513059+0000","last_undegraded":"2024-05-17T22:05:28.513059+0000","last_fullsized":"2024-05-17T22:05:28.513059+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412515+0000","last_change":"2024-05-17T22:05:27.416376+0000","last_active":"2024-05-17T22:05:28.412515+0000","last_peered":"2024-05-17T22:05:28.412515+0000","last_clean":"2024-05-17T22:05:28.412515+0000","last_became_active":"2024-05-17T22:05:27.415143+0000","last_became_peered":"2024-05-17T22:05:27.415143+0000","last_unstale":"2024-05-17T22:05:28.412515+0000","last_undegraded":"2024-05-17T22:05:28.412515+0000","last_fullsized":"2024-05-17T22:05:28.412515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:40.444170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465548+0000","last_change":"2024-05-17T22:05:26.389298+0000","last_active":"2024-05-17T22:05:28.465548+0000","last_peered":"2024-05-17T22:05:28.465548+0000","last_clean":"2024-05-17T22:05:28.465548+0000","last_became_active":"2024-05-17T22:05:26.388581+0000","last_became_peered":"2024-05-17T22:05:26.388581+0000","last_unstale":"2024-05-17T22:05:28.465548+0000","last_undegraded":"2024-05-17T22:05:28.465548+0000","last_fullsized":"2024-05-17T22:05:28.465548+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:31.057570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413881+0000","last_change":"2024-05-17T22:05:27.454673+0000","last_active":"2024-05-17T22:05:28.413881+0000","last_peered":"2024-05-17T22:05:28.413881+0000","last_clean":"2024-05-17T22:05:28.413881+0000","last_became_active":"2024-05-17T22:05:27.453764+0000","last_became_peered":"2024-05-17T22:05:27.453764+0000","last_unstale":"2024-05-17T22:05:28.413881+0000","last_undegraded":"2024-05-17T22:05:28.413881+0000","last_fullsized":"2024-05-17T22:05:28.413881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512020+0000","last_change":"2024-05-17T22:05:26.402412+0000","last_active":"2024-05-17T22:05:28.512020+0000","last_peered":"2024-05-17T22:05:28.512020+0000","last_clean":"2024-05-17T22:05:28.512020+0000","last_became_active":"2024-05-17T22:05:26.401498+0000","last_became_peered":"2024-05-17T22:05:26.401498+0000","last_unstale":"2024-05-17T22:05:28.512020+0000","last_undegraded":"2024-05-17T22:05:28.512020+0000","last_fullsized":"2024-05-17T22:05:28.512020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:57.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411800+0000","last_change":"2024-05-17T22:05:27.430574+0000","last_active":"2024-05-17T22:05:28.411800+0000","last_peered":"2024-05-17T22:05:28.411800+0000","last_clean":"2024-05-17T22:05:28.411800+0000","last_became_active":"2024-05-17T22:05:27.429640+0000","last_became_peered":"2024-05-17T22:05:27.429640+0000","last_unstale":"2024-05-17T22:05:28.411800+0000","last_undegraded":"2024-05-17T22:05:28.411800+0000","last_fullsized":"2024-05-17T22:05:28.411800+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:07:20.330652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511632+0000","last_change":"2024-05-17T22:05:26.386246+0000","last_active":"2024-05-17T22:05:28.511632+0000","last_peered":"2024-05-17T22:05:28.511632+0000","last_clean":"2024-05-17T22:05:28.511632+0000","last_became_active":"2024-05-17T22:05:26.385885+0000","last_became_peered":"2024-05-17T22:05:26.385885+0000","last_unstale":"2024-05-17T22:05:28.511632+0000","last_undegraded":"2024-05-17T22:05:28.511632+0000","last_fullsized":"2024-05-17T22:05:28.511632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511976+0000","last_change":"2024-05-17T22:05:27.452434+0000","last_active":"2024-05-17T22:05:28.511976+0000","last_peered":"2024-05-17T22:05:28.511976+0000","last_clean":"2024-05-17T22:05:28.511976+0000","last_became_active":"2024-05-17T22:05:27.451097+0000","last_became_peered":"2024-05-17T22:05:27.451097+0000","last_unstale":"2024-05-17T22:05:28.511976+0000","last_undegraded":"2024-05-17T22:05:28.511976+0000","last_fullsized":"2024-05-17T22:05:28.511976+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411927+0000","last_change":"2024-05-17T22:05:26.376780+0000","last_active":"2024-05-17T22:05:28.411927+0000","last_peered":"2024-05-17T22:05:28.411927+0000","last_clean":"2024-05-17T22:05:28.411927+0000","last_became_active":"2024-05-17T22:05:26.376444+0000","last_became_peered":"2024-05-17T22:05:26.376444+0000","last_unstale":"2024-05-17T22:05:28.411927+0000","last_undegraded":"2024-05-17T22:05:28.411927+0000","last_fullsized":"2024-05-17T22:05:28.411927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.044320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412939+0000","last_change":"2024-05-17T22:05:27.442004+0000","last_active":"2024-05-17T22:05:28.412939+0000","last_peered":"2024-05-17T22:05:28.412939+0000","last_clean":"2024-05-17T22:05:28.412939+0000","last_became_active":"2024-05-17T22:05:27.441304+0000","last_became_peered":"2024-05-17T22:05:27.441304+0000","last_unstale":"2024-05-17T22:05:28.412939+0000","last_undegraded":"2024-05-17T22:05:28.412939+0000","last_fullsized":"2024-05-17T22:05:28.412939+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:08:40.386729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511381+0000","last_change":"2024-05-17T22:05:26.400374+0000","last_active":"2024-05-17T22:05:28.511381+0000","last_peered":"2024-05-17T22:05:28.511381+0000","last_clean":"2024-05-17T22:05:28.511381+0000","last_became_active":"2024-05-17T22:05:26.399106+0000","last_became_peered":"2024-05-17T22:05:26.399106+0000","last_unstale":"2024-05-17T22:05:28.511381+0000","last_undegraded":"2024-05-17T22:05:28.511381+0000","last_fullsized":"2024-05-17T22:05:28.511381+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:57.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":249,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511111+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:05:28.511111+0000","last_peered":"2024-05-17T22:05:28.511111+0000","last_clean":"2024-05-17T22:05:28.511111+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:05:28.511111+0000","last_undegraded":"2024-05-17T22:05:28.511111+0000","last_fullsized":"2024-05-17T22:05:28.511111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1777664,"data_stored":1095910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":512382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":505500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":507064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":511764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1187840,"data_stored":511730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1773568,"data_stored":1096380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:05:31.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:05:31.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.716+0000 7ff1d6fbf640 1 Processor -- start 2024-05-17T22:05:31.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d6fbf640 1 -- start start 2024-05-17T22:05:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d6fbf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff1d0073540 0x7ff1d0168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d6fbf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d015e790 con 0x7ff1d0144080 2024-05-17T22:05:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d6fbf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d015ea40 con 0x7ff1d00d1040 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d6fbf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d015ed20 con 0x7ff1d0073540 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff1d0144080 0x7ff1d0072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56142/0 (socket says 172.21.15.83:56142) 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 -- 172.21.15.83:0/3524663792 learned_addr learned my addr 172.21.15.83:0/3524663792 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d555c640 1 --2- 172.21.15.83:0/3524663792 >> v2:172.21.15.155:3300/0 conn(0x7ff1d0073540 0x7ff1d0168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034705403 0 0) 0x7ff1d015e790 con 0x7ff1d0144080 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1b8003530 con 0x7ff1d0144080 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4173076471 0 0) 0x7ff1b8003530 con 0x7ff1d0144080 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 >> v2:172.21.15.155:3300/0 conn(0x7ff1d0073540 msgr2=0x7ff1d0168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 --2- 172.21.15.83:0/3524663792 >> v2:172.21.15.155:3300/0 conn(0x7ff1d0073540 0x7ff1d0168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 >> v1:172.21.15.155:6789/0 conn(0x7ff1d00d1040 legacy=0x7ff1d015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d015f000 con 0x7ff1d0144080 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ff1c4001700 con 0x7ff1d0144080 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 >> v1:172.21.15.83:6789/0 conn(0x7ff1d0144080 legacy=0x7ff1d0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8003a60 0x7ff1b8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 0x7ff1b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8005260 0x7ff1b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d015ed20 con 0x7ff1b8004810 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800c1f0 con 0x7ff1b8003a60 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b800c3f0 con 0x7ff1b8005260 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 0x7ff1b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 -- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8005260 msgr2=0x7ff1b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1b8005260 0x7ff1b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 -- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8003a60 msgr2=0x7ff1b8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8003a60 0x7ff1b8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1cffff640 1 -- 172.21.15.83:0/3524663792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b8003530 con 0x7ff1b8004810 2024-05-17T22:05:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.720+0000 7ff1d555c640 1 -- 172.21.15.83:0/3524663792 reap_dead start 2024-05-17T22:05:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1cffff640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 0x7ff1b8004c20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0073ed0 tx=0x7ff1c401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df559260d8a7689d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4007cf0 con 0x7ff1b8004810 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4005ae0 con 0x7ff1b8004810 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1cf7fe640 1 -- 172.21.15.83:0/3524663792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4007070 con 0x7ff1b8004810 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 msgr2=0x7ff1b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 0x7ff1b8004c20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0073ed0 tx=0x7ff1c401f2d0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/3524663792 shutdown_connections 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/3524663792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1b8004810 0x7ff1b8004c20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/3524663792 >> 172.21.15.83:0/3524663792 conn(0x7ff1d006cd90 msgr2=0x7ff1d00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/3524663792 shutdown_connections 2024-05-17T22:05:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/3524663792 wait complete. 2024-05-17T22:05:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 Processor -- start 2024-05-17T22:05:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.724+0000 7ff1d6fbf640 1 -- start start 2024-05-17T22:05:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 0x7ff1d0140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 0x7ff1d01410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d0144490 con 0x7ff1d00ae720 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d006f210 con 0x7ff1b8004810 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d015e080 con 0x7ff1d015c740 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56044/0 (socket says 172.21.15.83:56044) 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 -- 172.21.15.83:0/2493027283 learned_addr learned my addr 172.21.15.83:0/2493027283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cffff640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 0x7ff1d01410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d4d5b640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 0x7ff1d0140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 -- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 msgr2=0x7ff1d01410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 0x7ff1d01410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 -- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 msgr2=0x7ff1d0140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 0x7ff1d0140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 -- 172.21.15.83:0/2493027283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d01616e0 con 0x7ff1d00ae720 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d4d5b640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 0x7ff1d0140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cffff640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 0x7ff1d01410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d555c640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00ce920 tx=0x7ff1c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800bab0 con 0x7ff1d00ae720 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c801a070 con 0x7ff1d00ae720 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8014400 con 0x7ff1d00ae720 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d01619c0 con 0x7ff1d00ae720 2024-05-17T22:05:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d0161f80 con 0x7ff1d00ae720 2024-05-17T22:05:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d015e3b0 con 0x7ff1d00ae720 2024-05-17T22:05:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff1c8013070 con 0x7ff1d00ae720 2024-05-17T22:05:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.728+0000 7ff1cd7fa640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 0x7ff1bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.732+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7ff1c808f290 con 0x7ff1d00ae720 2024-05-17T22:05:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.732+0000 7ff1d4d5b640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 0x7ff1bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.732+0000 7ff1d4d5b640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 0x7ff1bc071940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0001170 tx=0x7ff1c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:31.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.736+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff1c805c180 con 0x7ff1d00ae720 2024-05-17T22:05:31.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.892+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1d0142e90 con 0x7ff1bc06f430 2024-05-17T22:05:31.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.904+0000 7ff1cd7fa640 1 -- 172.21.15.83:0/2493027283 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7ff1d0142e90 con 0x7ff1bc06f430 2024-05-17T22:05:31.897 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:31.899 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:05:31.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 msgr2=0x7ff1bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 0x7ff1bc071940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0001170 tx=0x7ff1c0009040 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 msgr2=0x7ff1d00aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00ce920 tx=0x7ff1c800b480 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 shutdown_connections 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d00ae720 0x7ff1d00aeb30 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff1d015c740 0x7ff1d01410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1bc06f430 0x7ff1bc071940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 --2- 172.21.15.83:0/2493027283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1b8004810 0x7ff1d0140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 >> 172.21.15.83:0/2493027283 conn(0x7ff1d006cd90 msgr2=0x7ff1d015d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 shutdown_connections 2024-05-17T22:05:31.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:31.908+0000 7ff1d6fbf640 1 -- 172.21.15.83:0/2493027283 wait complete. 2024-05-17T22:05:31.920 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-05-17T22:05:31.145764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259920,"kb_used_data":10444,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714800,"statfs":{"total":773094113280,"available":772827955200,"internally_reserved":0,"allocated":10694656,"data_stored":5247896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.788063"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510966+0000","last_change":"2024-05-17T22:05:27.437616+0000","last_active":"2024-05-17T22:05:28.510966+0000","last_peered":"2024-05-17T22:05:28.510966+0000","last_clean":"2024-05-17T22:05:28.510966+0000","last_became_active":"2024-05-17T22:05:27.435692+0000","last_became_peered":"2024-05-17T22:05:27.435692+0000","last_unstale":"2024-05-17T22:05:28.510966+0000","last_undegraded":"2024-05-17T22:05:28.510966+0000","last_fullsized":"2024-05-17T22:05:28.510966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:04:18.149802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465877+0000","last_change":"2024-05-17T22:05:27.455228+0000","last_active":"2024-05-17T22:05:28.465877+0000","last_peered":"2024-05-17T22:05:28.465877+0000","last_clean":"2024-05-17T22:05:28.465877+0000","last_became_active":"2024-05-17T22:05:27.452733+0000","last_became_peered":"2024-05-17T22:05:27.452733+0000","last_unstale":"2024-05-17T22:05:28.465877+0000","last_undegraded":"2024-05-17T22:05:28.465877+0000","last_fullsized":"2024-05-17T22:05:28.465877+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:24.428328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511603+0000","last_change":"2024-05-17T22:05:27.436096+0000","last_active":"2024-05-17T22:05:28.511603+0000","last_peered":"2024-05-17T22:05:28.511603+0000","last_clean":"2024-05-17T22:05:28.511603+0000","last_became_active":"2024-05-17T22:05:27.435215+0000","last_became_peered":"2024-05-17T22:05:27.435215+0000","last_unstale":"2024-05-17T22:05:28.511603+0000","last_undegraded":"2024-05-17T22:05:28.511603+0000","last_fullsized":"2024-05-17T22:05:28.511603+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:01.067972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512672+0000","last_change":"2024-05-17T22:05:27.449736+0000","last_active":"2024-05-17T22:05:28.512672+0000","last_peered":"2024-05-17T22:05:28.512672+0000","last_clean":"2024-05-17T22:05:28.512672+0000","last_became_active":"2024-05-17T22:05:27.445376+0000","last_became_peered":"2024-05-17T22:05:27.445376+0000","last_unstale":"2024-05-17T22:05:28.512672+0000","last_undegraded":"2024-05-17T22:05:28.512672+0000","last_fullsized":"2024-05-17T22:05:28.512672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512205+0000","last_change":"2024-05-17T22:05:27.449927+0000","last_active":"2024-05-17T22:05:28.512205+0000","last_peered":"2024-05-17T22:05:28.512205+0000","last_clean":"2024-05-17T22:05:28.512205+0000","last_became_active":"2024-05-17T22:05:27.449247+0000","last_became_peered":"2024-05-17T22:05:27.449247+0000","last_unstale":"2024-05-17T22:05:28.512205+0000","last_undegraded":"2024-05-17T22:05:28.512205+0000","last_fullsized":"2024-05-17T22:05:28.512205+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512451+0000","last_change":"2024-05-17T22:05:27.435224+0000","last_active":"2024-05-17T22:05:28.512451+0000","last_peered":"2024-05-17T22:05:28.512451+0000","last_clean":"2024-05-17T22:05:28.512451+0000","last_became_active":"2024-05-17T22:05:27.434335+0000","last_became_peered":"2024-05-17T22:05:27.434335+0000","last_unstale":"2024-05-17T22:05:28.512451+0000","last_undegraded":"2024-05-17T22:05:28.512451+0000","last_fullsized":"2024-05-17T22:05:28.512451+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:30.767567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510903+0000","last_change":"2024-05-17T22:05:27.435224+0000","last_active":"2024-05-17T22:05:28.510903+0000","last_peered":"2024-05-17T22:05:28.510903+0000","last_clean":"2024-05-17T22:05:28.510903+0000","last_became_active":"2024-05-17T22:05:27.434352+0000","last_became_peered":"2024-05-17T22:05:27.434352+0000","last_unstale":"2024-05-17T22:05:28.510903+0000","last_undegraded":"2024-05-17T22:05:28.510903+0000","last_fullsized":"2024-05-17T22:05:28.510903+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465270+0000","last_change":"2024-05-17T22:05:27.432637+0000","last_active":"2024-05-17T22:05:28.465270+0000","last_peered":"2024-05-17T22:05:28.465270+0000","last_clean":"2024-05-17T22:05:28.465270+0000","last_became_active":"2024-05-17T22:05:27.431831+0000","last_became_peered":"2024-05-17T22:05:27.431831+0000","last_unstale":"2024-05-17T22:05:28.465270+0000","last_undegraded":"2024-05-17T22:05:28.465270+0000","last_fullsized":"2024-05-17T22:05:28.465270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:40.459967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412923+0000","last_change":"2024-05-17T22:05:27.445862+0000","last_active":"2024-05-17T22:05:28.412923+0000","last_peered":"2024-05-17T22:05:28.412923+0000","last_clean":"2024-05-17T22:05:28.412923+0000","last_became_active":"2024-05-17T22:05:27.444739+0000","last_became_peered":"2024-05-17T22:05:27.444739+0000","last_unstale":"2024-05-17T22:05:28.412923+0000","last_undegraded":"2024-05-17T22:05:28.412923+0000","last_fullsized":"2024-05-17T22:05:28.412923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:32.068585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511531+0000","last_change":"2024-05-17T22:05:27.442733+0000","last_active":"2024-05-17T22:05:28.511531+0000","last_peered":"2024-05-17T22:05:28.511531+0000","last_clean":"2024-05-17T22:05:28.511531+0000","last_became_active":"2024-05-17T22:05:27.441844+0000","last_became_peered":"2024-05-17T22:05:27.441844+0000","last_unstale":"2024-05-17T22:05:28.511531+0000","last_undegraded":"2024-05-17T22:05:28.511531+0000","last_fullsized":"2024-05-17T22:05:28.511531+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511456+0000","last_change":"2024-05-17T22:05:27.432985+0000","last_active":"2024-05-17T22:05:28.511456+0000","last_peered":"2024-05-17T22:05:28.511456+0000","last_clean":"2024-05-17T22:05:28.511456+0000","last_became_active":"2024-05-17T22:05:27.432071+0000","last_became_peered":"2024-05-17T22:05:27.432071+0000","last_unstale":"2024-05-17T22:05:28.511456+0000","last_undegraded":"2024-05-17T22:05:28.511456+0000","last_fullsized":"2024-05-17T22:05:28.511456+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413841+0000","last_change":"2024-05-17T22:05:27.427483+0000","last_active":"2024-05-17T22:05:28.413841+0000","last_peered":"2024-05-17T22:05:28.413841+0000","last_clean":"2024-05-17T22:05:28.413841+0000","last_became_active":"2024-05-17T22:05:27.426574+0000","last_became_peered":"2024-05-17T22:05:27.426574+0000","last_unstale":"2024-05-17T22:05:28.413841+0000","last_undegraded":"2024-05-17T22:05:28.413841+0000","last_fullsized":"2024-05-17T22:05:28.413841+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512355+0000","last_change":"2024-05-17T22:05:27.439750+0000","last_active":"2024-05-17T22:05:28.512355+0000","last_peered":"2024-05-17T22:05:28.512355+0000","last_clean":"2024-05-17T22:05:28.512355+0000","last_became_active":"2024-05-17T22:05:27.437212+0000","last_became_peered":"2024-05-17T22:05:27.437212+0000","last_unstale":"2024-05-17T22:05:28.512355+0000","last_undegraded":"2024-05-17T22:05:28.512355+0000","last_fullsized":"2024-05-17T22:05:28.512355+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412116+0000","last_change":"2024-05-17T22:05:27.451726+0000","last_active":"2024-05-17T22:05:28.412116+0000","last_peered":"2024-05-17T22:05:28.412116+0000","last_clean":"2024-05-17T22:05:28.412116+0000","last_became_active":"2024-05-17T22:05:27.450823+0000","last_became_peered":"2024-05-17T22:05:27.450823+0000","last_unstale":"2024-05-17T22:05:28.412116+0000","last_undegraded":"2024-05-17T22:05:28.412116+0000","last_fullsized":"2024-05-17T22:05:28.412116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:16:00.874790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511271+0000","last_change":"2024-05-17T22:05:27.448190+0000","last_active":"2024-05-17T22:05:28.511271+0000","last_peered":"2024-05-17T22:05:28.511271+0000","last_clean":"2024-05-17T22:05:28.511271+0000","last_became_active":"2024-05-17T22:05:27.447608+0000","last_became_peered":"2024-05-17T22:05:27.447608+0000","last_unstale":"2024-05-17T22:05:28.511271+0000","last_undegraded":"2024-05-17T22:05:28.511271+0000","last_fullsized":"2024-05-17T22:05:28.511271+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510847+0000","last_change":"2024-05-17T22:05:27.447395+0000","last_active":"2024-05-17T22:05:28.510847+0000","last_peered":"2024-05-17T22:05:28.510847+0000","last_clean":"2024-05-17T22:05:28.510847+0000","last_became_active":"2024-05-17T22:05:27.446457+0000","last_became_peered":"2024-05-17T22:05:27.446457+0000","last_unstale":"2024-05-17T22:05:28.510847+0000","last_undegraded":"2024-05-17T22:05:28.510847+0000","last_fullsized":"2024-05-17T22:05:28.510847+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465962+0000","last_change":"2024-05-17T22:05:27.447783+0000","last_active":"2024-05-17T22:05:28.465962+0000","last_peered":"2024-05-17T22:05:28.465962+0000","last_clean":"2024-05-17T22:05:28.465962+0000","last_became_active":"2024-05-17T22:05:27.446586+0000","last_became_peered":"2024-05-17T22:05:27.446586+0000","last_unstale":"2024-05-17T22:05:28.465962+0000","last_undegraded":"2024-05-17T22:05:28.465962+0000","last_fullsized":"2024-05-17T22:05:28.465962+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:34:44.123521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512117+0000","last_change":"2024-05-17T22:05:27.446975+0000","last_active":"2024-05-17T22:05:28.512117+0000","last_peered":"2024-05-17T22:05:28.512117+0000","last_clean":"2024-05-17T22:05:28.512117+0000","last_became_active":"2024-05-17T22:05:27.446058+0000","last_became_peered":"2024-05-17T22:05:27.446058+0000","last_unstale":"2024-05-17T22:05:28.512117+0000","last_undegraded":"2024-05-17T22:05:28.512117+0000","last_fullsized":"2024-05-17T22:05:28.512117+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:12:15.322248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412800+0000","last_change":"2024-05-17T22:05:27.426973+0000","last_active":"2024-05-17T22:05:28.412800+0000","last_peered":"2024-05-17T22:05:28.412800+0000","last_clean":"2024-05-17T22:05:28.412800+0000","last_became_active":"2024-05-17T22:05:27.422435+0000","last_became_peered":"2024-05-17T22:05:27.422435+0000","last_unstale":"2024-05-17T22:05:28.412800+0000","last_undegraded":"2024-05-17T22:05:28.412800+0000","last_fullsized":"2024-05-17T22:05:28.412800+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:27:35.606607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414698+0000","last_change":"2024-05-17T22:05:27.446147+0000","last_active":"2024-05-17T22:05:28.414698+0000","last_peered":"2024-05-17T22:05:28.414698+0000","last_clean":"2024-05-17T22:05:28.414698+0000","last_became_active":"2024-05-17T22:05:27.445228+0000","last_became_peered":"2024-05-17T22:05:27.445228+0000","last_unstale":"2024-05-17T22:05:28.414698+0000","last_undegraded":"2024-05-17T22:05:28.414698+0000","last_fullsized":"2024-05-17T22:05:28.414698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412238+0000","last_change":"2024-05-17T22:05:27.411491+0000","last_active":"2024-05-17T22:05:28.412238+0000","last_peered":"2024-05-17T22:05:28.412238+0000","last_clean":"2024-05-17T22:05:28.412238+0000","last_became_active":"2024-05-17T22:05:27.410584+0000","last_became_peered":"2024-05-17T22:05:27.410584+0000","last_unstale":"2024-05-17T22:05:28.412238+0000","last_undegraded":"2024-05-17T22:05:28.412238+0000","last_fullsized":"2024-05-17T22:05:28.412238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:06:06.944584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465400+0000","last_change":"2024-05-17T22:05:27.436719+0000","last_active":"2024-05-17T22:05:28.465400+0000","last_peered":"2024-05-17T22:05:28.465400+0000","last_clean":"2024-05-17T22:05:28.465400+0000","last_became_active":"2024-05-17T22:05:27.435870+0000","last_became_peered":"2024-05-17T22:05:27.435870+0000","last_unstale":"2024-05-17T22:05:28.465400+0000","last_undegraded":"2024-05-17T22:05:28.465400+0000","last_fullsized":"2024-05-17T22:05:28.465400+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:01:24.876664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465295+0000","last_change":"2024-05-17T22:05:27.435595+0000","last_active":"2024-05-17T22:05:28.465295+0000","last_peered":"2024-05-17T22:05:28.465295+0000","last_clean":"2024-05-17T22:05:28.465295+0000","last_became_active":"2024-05-17T22:05:27.434786+0000","last_became_peered":"2024-05-17T22:05:27.434786+0000","last_unstale":"2024-05-17T22:05:28.465295+0000","last_undegraded":"2024-05-17T22:05:28.465295+0000","last_fullsized":"2024-05-17T22:05:28.465295+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:02:32.030203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510968+0000","last_change":"2024-05-17T22:05:27.429067+0000","last_active":"2024-05-17T22:05:28.510968+0000","last_peered":"2024-05-17T22:05:28.510968+0000","last_clean":"2024-05-17T22:05:28.510968+0000","last_became_active":"2024-05-17T22:05:27.428004+0000","last_became_peered":"2024-05-17T22:05:27.428004+0000","last_unstale":"2024-05-17T22:05:28.510968+0000","last_undegraded":"2024-05-17T22:05:28.510968+0000","last_fullsized":"2024-05-17T22:05:28.510968+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512972+0000","last_change":"2024-05-17T22:05:27.450609+0000","last_active":"2024-05-17T22:05:28.512972+0000","last_peered":"2024-05-17T22:05:28.512972+0000","last_clean":"2024-05-17T22:05:28.512972+0000","last_became_active":"2024-05-17T22:05:27.449075+0000","last_became_peered":"2024-05-17T22:05:27.449075+0000","last_unstale":"2024-05-17T22:05:28.512972+0000","last_undegraded":"2024-05-17T22:05:28.512972+0000","last_fullsized":"2024-05-17T22:05:28.512972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465769+0000","last_change":"2024-05-17T22:05:27.454186+0000","last_active":"2024-05-17T22:05:28.465769+0000","last_peered":"2024-05-17T22:05:28.465769+0000","last_clean":"2024-05-17T22:05:28.465769+0000","last_became_active":"2024-05-17T22:05:27.448403+0000","last_became_peered":"2024-05-17T22:05:27.448403+0000","last_unstale":"2024-05-17T22:05:28.465769+0000","last_undegraded":"2024-05-17T22:05:28.465769+0000","last_fullsized":"2024-05-17T22:05:28.465769+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:29:54.556862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512026+0000","last_change":"2024-05-17T22:05:27.442102+0000","last_active":"2024-05-17T22:05:28.512026+0000","last_peered":"2024-05-17T22:05:28.512026+0000","last_clean":"2024-05-17T22:05:28.512026+0000","last_became_active":"2024-05-17T22:05:27.441268+0000","last_became_peered":"2024-05-17T22:05:27.441268+0000","last_unstale":"2024-05-17T22:05:28.512026+0000","last_undegraded":"2024-05-17T22:05:28.512026+0000","last_fullsized":"2024-05-17T22:05:28.512026+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512881+0000","last_change":"2024-05-17T22:05:27.450220+0000","last_active":"2024-05-17T22:05:28.512881+0000","last_peered":"2024-05-17T22:05:28.512881+0000","last_clean":"2024-05-17T22:05:28.512881+0000","last_became_active":"2024-05-17T22:05:27.449522+0000","last_became_peered":"2024-05-17T22:05:27.449522+0000","last_unstale":"2024-05-17T22:05:28.512881+0000","last_undegraded":"2024-05-17T22:05:28.512881+0000","last_fullsized":"2024-05-17T22:05:28.512881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465332+0000","last_change":"2024-05-17T22:05:27.445831+0000","last_active":"2024-05-17T22:05:28.465332+0000","last_peered":"2024-05-17T22:05:28.465332+0000","last_clean":"2024-05-17T22:05:28.465332+0000","last_became_active":"2024-05-17T22:05:27.445032+0000","last_became_peered":"2024-05-17T22:05:27.445032+0000","last_unstale":"2024-05-17T22:05:28.465332+0000","last_undegraded":"2024-05-17T22:05:28.465332+0000","last_fullsized":"2024-05-17T22:05:28.465332+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:01:10.338332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412160+0000","last_change":"2024-05-17T22:05:27.426891+0000","last_active":"2024-05-17T22:05:28.412160+0000","last_peered":"2024-05-17T22:05:28.412160+0000","last_clean":"2024-05-17T22:05:28.412160+0000","last_became_active":"2024-05-17T22:05:27.425979+0000","last_became_peered":"2024-05-17T22:05:27.425979+0000","last_unstale":"2024-05-17T22:05:28.412160+0000","last_undegraded":"2024-05-17T22:05:28.412160+0000","last_fullsized":"2024-05-17T22:05:28.412160+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:42:53.395537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511522+0000","last_change":"2024-05-17T22:05:27.457824+0000","last_active":"2024-05-17T22:05:28.511522+0000","last_peered":"2024-05-17T22:05:28.511522+0000","last_clean":"2024-05-17T22:05:28.511522+0000","last_became_active":"2024-05-17T22:05:27.457211+0000","last_became_peered":"2024-05-17T22:05:27.457211+0000","last_unstale":"2024-05-17T22:05:28.511522+0000","last_undegraded":"2024-05-17T22:05:28.511522+0000","last_fullsized":"2024-05-17T22:05:28.511522+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510468+0000","last_change":"2024-05-17T22:05:27.448441+0000","last_active":"2024-05-17T22:05:28.510468+0000","last_peered":"2024-05-17T22:05:28.510468+0000","last_clean":"2024-05-17T22:05:28.510468+0000","last_became_active":"2024-05-17T22:05:27.446781+0000","last_became_peered":"2024-05-17T22:05:27.446781+0000","last_unstale":"2024-05-17T22:05:28.510468+0000","last_undegraded":"2024-05-17T22:05:28.510468+0000","last_fullsized":"2024-05-17T22:05:28.510468+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:50:46.664220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412763+0000","last_change":"2024-05-17T22:05:27.433888+0000","last_active":"2024-05-17T22:05:28.412763+0000","last_peered":"2024-05-17T22:05:28.412763+0000","last_clean":"2024-05-17T22:05:28.412763+0000","last_became_active":"2024-05-17T22:05:27.431384+0000","last_became_peered":"2024-05-17T22:05:27.431384+0000","last_unstale":"2024-05-17T22:05:28.412763+0000","last_undegraded":"2024-05-17T22:05:28.412763+0000","last_fullsized":"2024-05-17T22:05:28.412763+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:37:49.769320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465848+0000","last_change":"2024-05-17T22:05:26.410331+0000","last_active":"2024-05-17T22:05:28.465848+0000","last_peered":"2024-05-17T22:05:28.465848+0000","last_clean":"2024-05-17T22:05:28.465848+0000","last_became_active":"2024-05-17T22:05:26.409867+0000","last_became_peered":"2024-05-17T22:05:26.409867+0000","last_unstale":"2024-05-17T22:05:28.465848+0000","last_undegraded":"2024-05-17T22:05:28.465848+0000","last_fullsized":"2024-05-17T22:05:28.465848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:03:42.413177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512635+0000","last_change":"2024-05-17T22:05:27.457988+0000","last_active":"2024-05-17T22:05:28.512635+0000","last_peered":"2024-05-17T22:05:28.512635+0000","last_clean":"2024-05-17T22:05:28.512635+0000","last_became_active":"2024-05-17T22:05:27.457287+0000","last_became_peered":"2024-05-17T22:05:27.457287+0000","last_unstale":"2024-05-17T22:05:28.512635+0000","last_undegraded":"2024-05-17T22:05:28.512635+0000","last_fullsized":"2024-05-17T22:05:28.512635+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:21:50.018383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511734+0000","last_change":"2024-05-17T22:05:26.400449+0000","last_active":"2024-05-17T22:05:28.511734+0000","last_peered":"2024-05-17T22:05:28.511734+0000","last_clean":"2024-05-17T22:05:28.511734+0000","last_became_active":"2024-05-17T22:05:26.399787+0000","last_became_peered":"2024-05-17T22:05:26.399787+0000","last_unstale":"2024-05-17T22:05:28.511734+0000","last_undegraded":"2024-05-17T22:05:28.511734+0000","last_fullsized":"2024-05-17T22:05:28.511734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511204+0000","last_change":"2024-05-17T22:05:27.434317+0000","last_active":"2024-05-17T22:05:28.511204+0000","last_peered":"2024-05-17T22:05:28.511204+0000","last_clean":"2024-05-17T22:05:28.511204+0000","last_became_active":"2024-05-17T22:05:27.432757+0000","last_became_peered":"2024-05-17T22:05:27.432757+0000","last_unstale":"2024-05-17T22:05:28.511204+0000","last_undegraded":"2024-05-17T22:05:28.511204+0000","last_fullsized":"2024-05-17T22:05:28.511204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:04:41.298302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465624+0000","last_change":"2024-05-17T22:05:26.380122+0000","last_active":"2024-05-17T22:05:28.465624+0000","last_peered":"2024-05-17T22:05:28.465624+0000","last_clean":"2024-05-17T22:05:28.465624+0000","last_became_active":"2024-05-17T22:05:26.379777+0000","last_became_peered":"2024-05-17T22:05:26.379777+0000","last_unstale":"2024-05-17T22:05:28.465624+0000","last_undegraded":"2024-05-17T22:05:28.465624+0000","last_fullsized":"2024-05-17T22:05:28.465624+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:57.468540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412827+0000","last_change":"2024-05-17T22:05:27.414672+0000","last_active":"2024-05-17T22:05:28.412827+0000","last_peered":"2024-05-17T22:05:28.412827+0000","last_clean":"2024-05-17T22:05:28.412827+0000","last_became_active":"2024-05-17T22:05:27.413380+0000","last_became_peered":"2024-05-17T22:05:27.413380+0000","last_unstale":"2024-05-17T22:05:28.412827+0000","last_undegraded":"2024-05-17T22:05:28.412827+0000","last_fullsized":"2024-05-17T22:05:28.412827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:00:43.215036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465652+0000","last_change":"2024-05-17T22:05:26.394737+0000","last_active":"2024-05-17T22:05:28.465652+0000","last_peered":"2024-05-17T22:05:28.465652+0000","last_clean":"2024-05-17T22:05:28.465652+0000","last_became_active":"2024-05-17T22:05:26.394330+0000","last_became_peered":"2024-05-17T22:05:26.394330+0000","last_unstale":"2024-05-17T22:05:28.465652+0000","last_undegraded":"2024-05-17T22:05:28.465652+0000","last_fullsized":"2024-05-17T22:05:28.465652+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413501+0000","last_change":"2024-05-17T22:05:27.427588+0000","last_active":"2024-05-17T22:05:28.413501+0000","last_peered":"2024-05-17T22:05:28.413501+0000","last_clean":"2024-05-17T22:05:28.413501+0000","last_became_active":"2024-05-17T22:05:27.426388+0000","last_became_peered":"2024-05-17T22:05:27.426388+0000","last_unstale":"2024-05-17T22:05:28.413501+0000","last_undegraded":"2024-05-17T22:05:28.413501+0000","last_fullsized":"2024-05-17T22:05:28.413501+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:11:40.445555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511137+0000","last_change":"2024-05-17T22:05:26.387580+0000","last_active":"2024-05-17T22:05:28.511137+0000","last_peered":"2024-05-17T22:05:28.511137+0000","last_clean":"2024-05-17T22:05:28.511137+0000","last_became_active":"2024-05-17T22:05:26.387218+0000","last_became_peered":"2024-05-17T22:05:26.387218+0000","last_unstale":"2024-05-17T22:05:28.511137+0000","last_undegraded":"2024-05-17T22:05:28.511137+0000","last_fullsized":"2024-05-17T22:05:28.511137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:44:00.680001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512541+0000","last_change":"2024-05-17T22:05:27.450229+0000","last_active":"2024-05-17T22:05:28.512541+0000","last_peered":"2024-05-17T22:05:28.512541+0000","last_clean":"2024-05-17T22:05:28.512541+0000","last_became_active":"2024-05-17T22:05:27.449394+0000","last_became_peered":"2024-05-17T22:05:27.449394+0000","last_unstale":"2024-05-17T22:05:28.512541+0000","last_undegraded":"2024-05-17T22:05:28.512541+0000","last_fullsized":"2024-05-17T22:05:28.512541+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:10:40.459967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465421+0000","last_change":"2024-05-17T22:05:26.381762+0000","last_active":"2024-05-17T22:05:28.465421+0000","last_peered":"2024-05-17T22:05:28.465421+0000","last_clean":"2024-05-17T22:05:28.465421+0000","last_became_active":"2024-05-17T22:05:26.379895+0000","last_became_peered":"2024-05-17T22:05:26.379895+0000","last_unstale":"2024-05-17T22:05:28.465421+0000","last_undegraded":"2024-05-17T22:05:28.465421+0000","last_fullsized":"2024-05-17T22:05:28.465421+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:41:58.027527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511161+0000","last_change":"2024-05-17T22:05:27.436384+0000","last_active":"2024-05-17T22:05:28.511161+0000","last_peered":"2024-05-17T22:05:28.511161+0000","last_clean":"2024-05-17T22:05:28.511161+0000","last_became_active":"2024-05-17T22:05:27.435594+0000","last_became_peered":"2024-05-17T22:05:27.435594+0000","last_unstale":"2024-05-17T22:05:28.511161+0000","last_undegraded":"2024-05-17T22:05:28.511161+0000","last_fullsized":"2024-05-17T22:05:28.511161+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:54.898518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512221+0000","last_change":"2024-05-17T22:05:26.407612+0000","last_active":"2024-05-17T22:05:28.512221+0000","last_peered":"2024-05-17T22:05:28.512221+0000","last_clean":"2024-05-17T22:05:28.512221+0000","last_became_active":"2024-05-17T22:05:26.406766+0000","last_became_peered":"2024-05-17T22:05:26.406766+0000","last_unstale":"2024-05-17T22:05:28.512221+0000","last_undegraded":"2024-05-17T22:05:28.512221+0000","last_fullsized":"2024-05-17T22:05:28.512221+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:42:55.218241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411948+0000","last_change":"2024-05-17T22:05:27.414050+0000","last_active":"2024-05-17T22:05:28.411948+0000","last_peered":"2024-05-17T22:05:28.411948+0000","last_clean":"2024-05-17T22:05:28.411948+0000","last_became_active":"2024-05-17T22:05:27.413156+0000","last_became_peered":"2024-05-17T22:05:27.413156+0000","last_unstale":"2024-05-17T22:05:28.411948+0000","last_undegraded":"2024-05-17T22:05:28.411948+0000","last_fullsized":"2024-05-17T22:05:28.411948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:41:59.038541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510429+0000","last_change":"2024-05-17T22:05:26.389010+0000","last_active":"2024-05-17T22:05:28.510429+0000","last_peered":"2024-05-17T22:05:28.510429+0000","last_clean":"2024-05-17T22:05:28.510429+0000","last_became_active":"2024-05-17T22:05:26.387817+0000","last_became_peered":"2024-05-17T22:05:26.387817+0000","last_unstale":"2024-05-17T22:05:28.510429+0000","last_undegraded":"2024-05-17T22:05:28.510429+0000","last_fullsized":"2024-05-17T22:05:28.510429+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:08:39.375715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465884+0000","last_change":"2024-05-17T22:05:27.453658+0000","last_active":"2024-05-17T22:05:28.465884+0000","last_peered":"2024-05-17T22:05:28.465884+0000","last_clean":"2024-05-17T22:05:28.465884+0000","last_became_active":"2024-05-17T22:05:27.447524+0000","last_became_peered":"2024-05-17T22:05:27.447524+0000","last_unstale":"2024-05-17T22:05:28.465884+0000","last_undegraded":"2024-05-17T22:05:28.465884+0000","last_fullsized":"2024-05-17T22:05:28.465884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:49:40.661750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511659+0000","last_change":"2024-05-17T22:05:26.376826+0000","last_active":"2024-05-17T22:05:28.511659+0000","last_peered":"2024-05-17T22:05:28.511659+0000","last_clean":"2024-05-17T22:05:28.511659+0000","last_became_active":"2024-05-17T22:05:26.376471+0000","last_became_peered":"2024-05-17T22:05:26.376471+0000","last_unstale":"2024-05-17T22:05:28.511659+0000","last_undegraded":"2024-05-17T22:05:28.511659+0000","last_fullsized":"2024-05-17T22:05:28.511659+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:59:06.705185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413318+0000","last_change":"2024-05-17T22:05:27.414509+0000","last_active":"2024-05-17T22:05:28.413318+0000","last_peered":"2024-05-17T22:05:28.413318+0000","last_clean":"2024-05-17T22:05:28.413318+0000","last_became_active":"2024-05-17T22:05:27.413575+0000","last_became_peered":"2024-05-17T22:05:27.413575+0000","last_unstale":"2024-05-17T22:05:28.413318+0000","last_undegraded":"2024-05-17T22:05:28.413318+0000","last_fullsized":"2024-05-17T22:05:28.413318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:35:20.803666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512414+0000","last_change":"2024-05-17T22:05:26.376395+0000","last_active":"2024-05-17T22:05:28.512414+0000","last_peered":"2024-05-17T22:05:28.512414+0000","last_clean":"2024-05-17T22:05:28.512414+0000","last_became_active":"2024-05-17T22:05:26.375768+0000","last_became_peered":"2024-05-17T22:05:26.375768+0000","last_unstale":"2024-05-17T22:05:28.512414+0000","last_undegraded":"2024-05-17T22:05:28.512414+0000","last_fullsized":"2024-05-17T22:05:28.512414+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513086+0000","last_change":"2024-05-17T22:05:27.449618+0000","last_active":"2024-05-17T22:05:28.513086+0000","last_peered":"2024-05-17T22:05:28.513086+0000","last_clean":"2024-05-17T22:05:28.513086+0000","last_became_active":"2024-05-17T22:05:27.448834+0000","last_became_peered":"2024-05-17T22:05:27.448834+0000","last_unstale":"2024-05-17T22:05:28.513086+0000","last_undegraded":"2024-05-17T22:05:28.513086+0000","last_fullsized":"2024-05-17T22:05:28.513086+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:05:30.767567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512300+0000","last_change":"2024-05-17T22:05:26.407057+0000","last_active":"2024-05-17T22:05:28.512300+0000","last_peered":"2024-05-17T22:05:28.512300+0000","last_clean":"2024-05-17T22:05:28.512300+0000","last_became_active":"2024-05-17T22:05:26.406287+0000","last_became_peered":"2024-05-17T22:05:26.406287+0000","last_unstale":"2024-05-17T22:05:28.512300+0000","last_undegraded":"2024-05-17T22:05:28.512300+0000","last_fullsized":"2024-05-17T22:05:28.512300+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511917+0000","last_change":"2024-05-17T22:05:27.445181+0000","last_active":"2024-05-17T22:05:28.511917+0000","last_peered":"2024-05-17T22:05:28.511917+0000","last_clean":"2024-05-17T22:05:28.511917+0000","last_became_active":"2024-05-17T22:05:27.444401+0000","last_became_peered":"2024-05-17T22:05:27.444401+0000","last_unstale":"2024-05-17T22:05:28.511917+0000","last_undegraded":"2024-05-17T22:05:28.511917+0000","last_fullsized":"2024-05-17T22:05:28.511917+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:19:56.055133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414577+0000","last_change":"2024-05-17T22:05:26.392154+0000","last_active":"2024-05-17T22:05:28.414577+0000","last_peered":"2024-05-17T22:05:28.414577+0000","last_clean":"2024-05-17T22:05:28.414577+0000","last_became_active":"2024-05-17T22:05:26.390754+0000","last_became_peered":"2024-05-17T22:05:26.390754+0000","last_unstale":"2024-05-17T22:05:28.414577+0000","last_undegraded":"2024-05-17T22:05:28.414577+0000","last_fullsized":"2024-05-17T22:05:28.414577+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:10:14.911182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511478+0000","last_change":"2024-05-17T22:05:27.441163+0000","last_active":"2024-05-17T22:05:28.511478+0000","last_peered":"2024-05-17T22:05:28.511478+0000","last_clean":"2024-05-17T22:05:28.511478+0000","last_became_active":"2024-05-17T22:05:27.439993+0000","last_became_peered":"2024-05-17T22:05:27.439993+0000","last_unstale":"2024-05-17T22:05:28.511478+0000","last_undegraded":"2024-05-17T22:05:28.511478+0000","last_fullsized":"2024-05-17T22:05:28.511478+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412388+0000","last_change":"2024-05-17T22:05:26.397019+0000","last_active":"2024-05-17T22:05:28.412388+0000","last_peered":"2024-05-17T22:05:28.412388+0000","last_clean":"2024-05-17T22:05:28.412388+0000","last_became_active":"2024-05-17T22:05:26.396484+0000","last_became_peered":"2024-05-17T22:05:26.396484+0000","last_unstale":"2024-05-17T22:05:28.412388+0000","last_undegraded":"2024-05-17T22:05:28.412388+0000","last_fullsized":"2024-05-17T22:05:28.412388+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:45:11.894103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510850+0000","last_change":"2024-05-17T22:05:27.439006+0000","last_active":"2024-05-17T22:05:28.510850+0000","last_peered":"2024-05-17T22:05:28.510850+0000","last_clean":"2024-05-17T22:05:28.510850+0000","last_became_active":"2024-05-17T22:05:27.434400+0000","last_became_peered":"2024-05-17T22:05:27.434400+0000","last_unstale":"2024-05-17T22:05:28.510850+0000","last_undegraded":"2024-05-17T22:05:28.510850+0000","last_fullsized":"2024-05-17T22:05:28.510850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:53:22.603437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412646+0000","last_change":"2024-05-17T22:05:26.381773+0000","last_active":"2024-05-17T22:05:28.412646+0000","last_peered":"2024-05-17T22:05:28.412646+0000","last_clean":"2024-05-17T22:05:28.412646+0000","last_became_active":"2024-05-17T22:05:26.380956+0000","last_became_peered":"2024-05-17T22:05:26.380956+0000","last_unstale":"2024-05-17T22:05:28.412646+0000","last_undegraded":"2024-05-17T22:05:28.412646+0000","last_fullsized":"2024-05-17T22:05:28.412646+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:22.035385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411707+0000","last_change":"2024-05-17T22:05:27.414342+0000","last_active":"2024-05-17T22:05:28.411707+0000","last_peered":"2024-05-17T22:05:28.411707+0000","last_clean":"2024-05-17T22:05:28.411707+0000","last_became_active":"2024-05-17T22:05:27.413415+0000","last_became_peered":"2024-05-17T22:05:27.413415+0000","last_unstale":"2024-05-17T22:05:28.411707+0000","last_undegraded":"2024-05-17T22:05:28.411707+0000","last_fullsized":"2024-05-17T22:05:28.411707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T10:03:43.424191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411840+0000","last_change":"2024-05-17T22:05:26.411036+0000","last_active":"2024-05-17T22:05:28.411840+0000","last_peered":"2024-05-17T22:05:28.411840+0000","last_clean":"2024-05-17T22:05:28.411840+0000","last_became_active":"2024-05-17T22:05:26.410159+0000","last_became_peered":"2024-05-17T22:05:26.410159+0000","last_unstale":"2024-05-17T22:05:28.411840+0000","last_undegraded":"2024-05-17T22:05:28.411840+0000","last_fullsized":"2024-05-17T22:05:28.411840+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:41.129609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412115+0000","last_change":"2024-05-17T22:05:27.414057+0000","last_active":"2024-05-17T22:05:28.412115+0000","last_peered":"2024-05-17T22:05:28.412115+0000","last_clean":"2024-05-17T22:05:28.412115+0000","last_became_active":"2024-05-17T22:05:27.413128+0000","last_became_peered":"2024-05-17T22:05:27.413128+0000","last_unstale":"2024-05-17T22:05:28.412115+0000","last_undegraded":"2024-05-17T22:05:28.412115+0000","last_fullsized":"2024-05-17T22:05:28.412115+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:02:57.723926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.510741+0000","last_change":"2024-05-17T22:05:26.397262+0000","last_active":"2024-05-17T22:05:28.510741+0000","last_peered":"2024-05-17T22:05:28.510741+0000","last_clean":"2024-05-17T22:05:28.510741+0000","last_became_active":"2024-05-17T22:05:26.396653+0000","last_became_peered":"2024-05-17T22:05:26.396653+0000","last_unstale":"2024-05-17T22:05:28.510741+0000","last_undegraded":"2024-05-17T22:05:28.510741+0000","last_fullsized":"2024-05-17T22:05:28.510741+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:58:53.828477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513083+0000","last_change":"2024-05-17T22:05:27.450951+0000","last_active":"2024-05-17T22:05:28.513083+0000","last_peered":"2024-05-17T22:05:28.513083+0000","last_clean":"2024-05-17T22:05:28.513083+0000","last_became_active":"2024-05-17T22:05:27.450180+0000","last_became_peered":"2024-05-17T22:05:27.450180+0000","last_unstale":"2024-05-17T22:05:28.513083+0000","last_undegraded":"2024-05-17T22:05:28.513083+0000","last_fullsized":"2024-05-17T22:05:28.513083+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412745+0000","last_change":"2024-05-17T22:05:26.400196+0000","last_active":"2024-05-17T22:05:28.412745+0000","last_peered":"2024-05-17T22:05:28.412745+0000","last_clean":"2024-05-17T22:05:28.412745+0000","last_became_active":"2024-05-17T22:05:26.399378+0000","last_became_peered":"2024-05-17T22:05:26.399378+0000","last_unstale":"2024-05-17T22:05:28.412745+0000","last_undegraded":"2024-05-17T22:05:28.412745+0000","last_fullsized":"2024-05-17T22:05:28.412745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:12:20.023566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414178+0000","last_change":"2024-05-17T22:05:27.454581+0000","last_active":"2024-05-17T22:05:28.414178+0000","last_peered":"2024-05-17T22:05:28.414178+0000","last_clean":"2024-05-17T22:05:28.414178+0000","last_became_active":"2024-05-17T22:05:27.454044+0000","last_became_peered":"2024-05-17T22:05:27.454044+0000","last_unstale":"2024-05-17T22:05:28.414178+0000","last_undegraded":"2024-05-17T22:05:28.414178+0000","last_fullsized":"2024-05-17T22:05:28.414178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:53:42.140624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512337+0000","last_change":"2024-05-17T22:05:26.389173+0000","last_active":"2024-05-17T22:05:28.512337+0000","last_peered":"2024-05-17T22:05:28.512337+0000","last_clean":"2024-05-17T22:05:28.512337+0000","last_became_active":"2024-05-17T22:05:26.388309+0000","last_became_peered":"2024-05-17T22:05:26.388309+0000","last_unstale":"2024-05-17T22:05:28.512337+0000","last_undegraded":"2024-05-17T22:05:28.512337+0000","last_fullsized":"2024-05-17T22:05:28.512337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512379+0000","last_change":"2024-05-17T22:05:27.440184+0000","last_active":"2024-05-17T22:05:28.512379+0000","last_peered":"2024-05-17T22:05:28.512379+0000","last_clean":"2024-05-17T22:05:28.512379+0000","last_became_active":"2024-05-17T22:05:27.439330+0000","last_became_peered":"2024-05-17T22:05:27.439330+0000","last_unstale":"2024-05-17T22:05:28.512379+0000","last_undegraded":"2024-05-17T22:05:28.512379+0000","last_fullsized":"2024-05-17T22:05:28.512379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:22:12.965667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465663+0000","last_change":"2024-05-17T22:05:26.396565+0000","last_active":"2024-05-17T22:05:28.465663+0000","last_peered":"2024-05-17T22:05:28.465663+0000","last_clean":"2024-05-17T22:05:28.465663+0000","last_became_active":"2024-05-17T22:05:26.396040+0000","last_became_peered":"2024-05-17T22:05:26.396040+0000","last_unstale":"2024-05-17T22:05:28.465663+0000","last_undegraded":"2024-05-17T22:05:28.465663+0000","last_fullsized":"2024-05-17T22:05:28.465663+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:33:32.933119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412834+0000","last_change":"2024-05-17T22:05:27.449626+0000","last_active":"2024-05-17T22:05:28.412834+0000","last_peered":"2024-05-17T22:05:28.412834+0000","last_clean":"2024-05-17T22:05:28.412834+0000","last_became_active":"2024-05-17T22:05:27.448722+0000","last_became_peered":"2024-05-17T22:05:27.448722+0000","last_unstale":"2024-05-17T22:05:28.412834+0000","last_undegraded":"2024-05-17T22:05:28.412834+0000","last_fullsized":"2024-05-17T22:05:28.412834+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:44:01.691016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465519+0000","last_change":"2024-05-17T22:05:26.386697+0000","last_active":"2024-05-17T22:05:28.465519+0000","last_peered":"2024-05-17T22:05:28.465519+0000","last_clean":"2024-05-17T22:05:28.465519+0000","last_became_active":"2024-05-17T22:05:26.386096+0000","last_became_peered":"2024-05-17T22:05:26.386096+0000","last_unstale":"2024-05-17T22:05:28.465519+0000","last_undegraded":"2024-05-17T22:05:28.465519+0000","last_fullsized":"2024-05-17T22:05:28.465519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:00:42.204022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412633+0000","last_change":"2024-05-17T22:05:27.452521+0000","last_active":"2024-05-17T22:05:28.412633+0000","last_peered":"2024-05-17T22:05:28.412633+0000","last_clean":"2024-05-17T22:05:28.412633+0000","last_became_active":"2024-05-17T22:05:27.451787+0000","last_became_peered":"2024-05-17T22:05:27.451787+0000","last_unstale":"2024-05-17T22:05:28.412633+0000","last_undegraded":"2024-05-17T22:05:28.412633+0000","last_fullsized":"2024-05-17T22:05:28.412633+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:50:09.041997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413795+0000","last_change":"2024-05-17T22:05:26.377842+0000","last_active":"2024-05-17T22:05:28.413795+0000","last_peered":"2024-05-17T22:05:28.413795+0000","last_clean":"2024-05-17T22:05:28.413795+0000","last_became_active":"2024-05-17T22:05:26.377041+0000","last_became_peered":"2024-05-17T22:05:26.377041+0000","last_unstale":"2024-05-17T22:05:28.413795+0000","last_undegraded":"2024-05-17T22:05:28.413795+0000","last_fullsized":"2024-05-17T22:05:28.413795+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:54:15.786127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411814+0000","last_change":"2024-05-17T22:05:27.435956+0000","last_active":"2024-05-17T22:05:28.411814+0000","last_peered":"2024-05-17T22:05:28.411814+0000","last_clean":"2024-05-17T22:05:28.411814+0000","last_became_active":"2024-05-17T22:05:27.435123+0000","last_became_peered":"2024-05-17T22:05:27.435123+0000","last_unstale":"2024-05-17T22:05:28.411814+0000","last_undegraded":"2024-05-17T22:05:28.411814+0000","last_fullsized":"2024-05-17T22:05:28.411814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:29.817682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413605+0000","last_change":"2024-05-17T22:05:26.388188+0000","last_active":"2024-05-17T22:05:28.413605+0000","last_peered":"2024-05-17T22:05:28.413605+0000","last_clean":"2024-05-17T22:05:28.413605+0000","last_became_active":"2024-05-17T22:05:26.387288+0000","last_became_peered":"2024-05-17T22:05:26.387288+0000","last_unstale":"2024-05-17T22:05:28.413605+0000","last_undegraded":"2024-05-17T22:05:28.413605+0000","last_fullsized":"2024-05-17T22:05:28.413605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:00.056958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511904+0000","last_change":"2024-05-17T22:05:27.441725+0000","last_active":"2024-05-17T22:05:28.511904+0000","last_peered":"2024-05-17T22:05:28.511904+0000","last_clean":"2024-05-17T22:05:28.511904+0000","last_became_active":"2024-05-17T22:05:27.440926+0000","last_became_peered":"2024-05-17T22:05:27.440926+0000","last_unstale":"2024-05-17T22:05:28.511904+0000","last_undegraded":"2024-05-17T22:05:28.511904+0000","last_fullsized":"2024-05-17T22:05:28.511904+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465327+0000","last_change":"2024-05-17T22:05:26.378034+0000","last_active":"2024-05-17T22:05:28.465327+0000","last_peered":"2024-05-17T22:05:28.465327+0000","last_clean":"2024-05-17T22:05:28.465327+0000","last_became_active":"2024-05-17T22:05:26.377700+0000","last_became_peered":"2024-05-17T22:05:26.377700+0000","last_unstale":"2024-05-17T22:05:28.465327+0000","last_undegraded":"2024-05-17T22:05:28.465327+0000","last_fullsized":"2024-05-17T22:05:28.465327+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:27:14.293781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.466059+0000","last_change":"2024-05-17T22:05:27.438205+0000","last_active":"2024-05-17T22:05:28.466059+0000","last_peered":"2024-05-17T22:05:28.466059+0000","last_clean":"2024-05-17T22:05:28.466059+0000","last_became_active":"2024-05-17T22:05:27.436508+0000","last_became_peered":"2024-05-17T22:05:27.436508+0000","last_unstale":"2024-05-17T22:05:28.466059+0000","last_undegraded":"2024-05-17T22:05:28.466059+0000","last_fullsized":"2024-05-17T22:05:28.466059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:14:19.107391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.414299+0000","last_change":"2024-05-17T22:05:26.384952+0000","last_active":"2024-05-17T22:05:28.414299+0000","last_peered":"2024-05-17T22:05:28.414299+0000","last_clean":"2024-05-17T22:05:28.414299+0000","last_became_active":"2024-05-17T22:05:26.383936+0000","last_became_peered":"2024-05-17T22:05:26.383936+0000","last_unstale":"2024-05-17T22:05:28.414299+0000","last_undegraded":"2024-05-17T22:05:28.414299+0000","last_fullsized":"2024-05-17T22:05:28.414299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:45:44.409039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511388+0000","last_change":"2024-05-17T22:05:27.434762+0000","last_active":"2024-05-17T22:05:28.511388+0000","last_peered":"2024-05-17T22:05:28.511388+0000","last_clean":"2024-05-17T22:05:28.511388+0000","last_became_active":"2024-05-17T22:05:27.433884+0000","last_became_peered":"2024-05-17T22:05:27.433884+0000","last_unstale":"2024-05-17T22:05:28.511388+0000","last_undegraded":"2024-05-17T22:05:28.511388+0000","last_fullsized":"2024-05-17T22:05:28.511388+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:15:25.227504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.467573+0000","last_change":"2024-05-17T22:05:26.384752+0000","last_active":"2024-05-17T22:05:28.467573+0000","last_peered":"2024-05-17T22:05:28.467573+0000","last_clean":"2024-05-17T22:05:28.467573+0000","last_became_active":"2024-05-17T22:05:26.382500+0000","last_became_peered":"2024-05-17T22:05:26.382500+0000","last_unstale":"2024-05-17T22:05:28.467573+0000","last_undegraded":"2024-05-17T22:05:28.467573+0000","last_fullsized":"2024-05-17T22:05:28.467573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:23:53.887504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511601+0000","last_change":"2024-05-17T22:05:27.455421+0000","last_active":"2024-05-17T22:05:28.511601+0000","last_peered":"2024-05-17T22:05:28.511601+0000","last_clean":"2024-05-17T22:05:28.511601+0000","last_became_active":"2024-05-17T22:05:27.454641+0000","last_became_peered":"2024-05-17T22:05:27.454641+0000","last_unstale":"2024-05-17T22:05:28.511601+0000","last_undegraded":"2024-05-17T22:05:28.511601+0000","last_fullsized":"2024-05-17T22:05:28.511601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:15:23.998957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465893+0000","last_change":"2024-05-17T22:05:26.393401+0000","last_active":"2024-05-17T22:05:28.465893+0000","last_peered":"2024-05-17T22:05:28.465893+0000","last_clean":"2024-05-17T22:05:28.465893+0000","last_became_active":"2024-05-17T22:05:26.392987+0000","last_became_peered":"2024-05-17T22:05:26.392987+0000","last_unstale":"2024-05-17T22:05:28.465893+0000","last_undegraded":"2024-05-17T22:05:28.465893+0000","last_fullsized":"2024-05-17T22:05:28.465893+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:58:32.953243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512007+0000","last_change":"2024-05-17T22:05:27.444076+0000","last_active":"2024-05-17T22:05:28.512007+0000","last_peered":"2024-05-17T22:05:28.512007+0000","last_clean":"2024-05-17T22:05:28.512007+0000","last_became_active":"2024-05-17T22:05:27.443260+0000","last_became_peered":"2024-05-17T22:05:27.443260+0000","last_unstale":"2024-05-17T22:05:28.512007+0000","last_undegraded":"2024-05-17T22:05:28.512007+0000","last_fullsized":"2024-05-17T22:05:28.512007+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:11:51.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.513059+0000","last_change":"2024-05-17T22:05:26.406964+0000","last_active":"2024-05-17T22:05:28.513059+0000","last_peered":"2024-05-17T22:05:28.513059+0000","last_clean":"2024-05-17T22:05:28.513059+0000","last_became_active":"2024-05-17T22:05:26.406023+0000","last_became_peered":"2024-05-17T22:05:26.406023+0000","last_unstale":"2024-05-17T22:05:28.513059+0000","last_undegraded":"2024-05-17T22:05:28.513059+0000","last_fullsized":"2024-05-17T22:05:28.513059+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:44:43.437348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412515+0000","last_change":"2024-05-17T22:05:27.416376+0000","last_active":"2024-05-17T22:05:28.412515+0000","last_peered":"2024-05-17T22:05:28.412515+0000","last_clean":"2024-05-17T22:05:28.412515+0000","last_became_active":"2024-05-17T22:05:27.415143+0000","last_became_peered":"2024-05-17T22:05:27.415143+0000","last_unstale":"2024-05-17T22:05:28.412515+0000","last_undegraded":"2024-05-17T22:05:28.412515+0000","last_fullsized":"2024-05-17T22:05:28.412515+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:54:40.444170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.465548+0000","last_change":"2024-05-17T22:05:26.389298+0000","last_active":"2024-05-17T22:05:28.465548+0000","last_peered":"2024-05-17T22:05:28.465548+0000","last_clean":"2024-05-17T22:05:28.465548+0000","last_became_active":"2024-05-17T22:05:26.388581+0000","last_became_peered":"2024-05-17T22:05:26.388581+0000","last_unstale":"2024-05-17T22:05:28.465548+0000","last_undegraded":"2024-05-17T22:05:28.465548+0000","last_fullsized":"2024-05-17T22:05:28.465548+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:21:31.057570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.413881+0000","last_change":"2024-05-17T22:05:27.454673+0000","last_active":"2024-05-17T22:05:28.413881+0000","last_peered":"2024-05-17T22:05:28.413881+0000","last_clean":"2024-05-17T22:05:28.413881+0000","last_became_active":"2024-05-17T22:05:27.453764+0000","last_became_peered":"2024-05-17T22:05:27.453764+0000","last_unstale":"2024-05-17T22:05:28.413881+0000","last_undegraded":"2024-05-17T22:05:28.413881+0000","last_fullsized":"2024-05-17T22:05:28.413881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:13:23.527983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.512020+0000","last_change":"2024-05-17T22:05:26.402412+0000","last_active":"2024-05-17T22:05:28.512020+0000","last_peered":"2024-05-17T22:05:28.512020+0000","last_clean":"2024-05-17T22:05:28.512020+0000","last_became_active":"2024-05-17T22:05:26.401498+0000","last_became_peered":"2024-05-17T22:05:26.401498+0000","last_unstale":"2024-05-17T22:05:28.512020+0000","last_undegraded":"2024-05-17T22:05:28.512020+0000","last_fullsized":"2024-05-17T22:05:28.512020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:57.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411800+0000","last_change":"2024-05-17T22:05:27.430574+0000","last_active":"2024-05-17T22:05:28.411800+0000","last_peered":"2024-05-17T22:05:28.411800+0000","last_clean":"2024-05-17T22:05:28.411800+0000","last_became_active":"2024-05-17T22:05:27.429640+0000","last_became_peered":"2024-05-17T22:05:27.429640+0000","last_unstale":"2024-05-17T22:05:28.411800+0000","last_undegraded":"2024-05-17T22:05:28.411800+0000","last_fullsized":"2024-05-17T22:05:28.411800+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:07:20.330652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511632+0000","last_change":"2024-05-17T22:05:26.386246+0000","last_active":"2024-05-17T22:05:28.511632+0000","last_peered":"2024-05-17T22:05:28.511632+0000","last_clean":"2024-05-17T22:05:28.511632+0000","last_became_active":"2024-05-17T22:05:26.385885+0000","last_became_peered":"2024-05-17T22:05:26.385885+0000","last_unstale":"2024-05-17T22:05:28.511632+0000","last_undegraded":"2024-05-17T22:05:28.511632+0000","last_fullsized":"2024-05-17T22:05:28.511632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:05:04.469310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511976+0000","last_change":"2024-05-17T22:05:27.452434+0000","last_active":"2024-05-17T22:05:28.511976+0000","last_peered":"2024-05-17T22:05:28.511976+0000","last_clean":"2024-05-17T22:05:28.511976+0000","last_became_active":"2024-05-17T22:05:27.451097+0000","last_became_peered":"2024-05-17T22:05:27.451097+0000","last_unstale":"2024-05-17T22:05:28.511976+0000","last_undegraded":"2024-05-17T22:05:28.511976+0000","last_fullsized":"2024-05-17T22:05:28.511976+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:20:41.112348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.411927+0000","last_change":"2024-05-17T22:05:26.376780+0000","last_active":"2024-05-17T22:05:28.411927+0000","last_peered":"2024-05-17T22:05:28.411927+0000","last_clean":"2024-05-17T22:05:28.411927+0000","last_became_active":"2024-05-17T22:05:26.376444+0000","last_became_peered":"2024-05-17T22:05:26.376444+0000","last_unstale":"2024-05-17T22:05:28.411927+0000","last_undegraded":"2024-05-17T22:05:28.411927+0000","last_fullsized":"2024-05-17T22:05:28.411927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:07:29.044320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.412939+0000","last_change":"2024-05-17T22:05:27.442004+0000","last_active":"2024-05-17T22:05:28.412939+0000","last_peered":"2024-05-17T22:05:28.412939+0000","last_clean":"2024-05-17T22:05:28.412939+0000","last_became_active":"2024-05-17T22:05:27.441304+0000","last_became_peered":"2024-05-17T22:05:27.441304+0000","last_unstale":"2024-05-17T22:05:28.412939+0000","last_undegraded":"2024-05-17T22:05:28.412939+0000","last_fullsized":"2024-05-17T22:05:28.412939+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:26.355538+0000","last_clean_scrub_stamp":"2024-05-17T22:05:26.355538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:08:40.386729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511381+0000","last_change":"2024-05-17T22:05:26.400374+0000","last_active":"2024-05-17T22:05:28.511381+0000","last_peered":"2024-05-17T22:05:28.511381+0000","last_clean":"2024-05-17T22:05:28.511381+0000","last_became_active":"2024-05-17T22:05:26.399106+0000","last_became_peered":"2024-05-17T22:05:26.399106+0000","last_unstale":"2024-05-17T22:05:28.511381+0000","last_undegraded":"2024-05-17T22:05:28.511381+0000","last_fullsized":"2024-05-17T22:05:28.511381+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:05:25.344524+0000","last_clean_scrub_stamp":"2024-05-17T22:05:25.344524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:54:57.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":249,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-05-17T22:05:28.511111+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:05:28.511111+0000","last_peered":"2024-05-17T22:05:28.511111+0000","last_clean":"2024-05-17T22:05:28.511111+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:05:28.511111+0000","last_undegraded":"2024-05-17T22:05:28.511111+0000","last_fullsized":"2024-05-17T22:05:28.511111+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1777664,"data_stored":1095910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":512382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":505500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":507166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":507064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":511764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1187840,"data_stored":511730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1773568,"data_stored":1096380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:05:31.924 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:05:31.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T22:05:32.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 Processor -- start 2024-05-17T22:05:32.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 -- start start 2024-05-17T22:05:32.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85700b5020 0x7f85701431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 0x7f85700b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8570143b30 con 0x7f85700b4c10 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f85700ce4d0 con 0x7f85700b5020 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f8575d85640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f857015e040 con 0x7f8570143720 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856effd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85700b5020 0x7f85701431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85700b4c10 0x7f85700cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56156/0 (socket says 172.21.15.83:56156) 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856f7fe640 1 -- 172.21.15.83:0/3899334 learned_addr learned my addr 172.21.15.83:0/3899334 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856ffff640 1 --2- 172.21.15.83:0/3899334 >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 0x7f85700b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856e7fc640 1 -- 172.21.15.83:0/3899334 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144935810 0 0) 0x7f8560001120 con 0x7f85700b4c10 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856e7fc640 1 -- 172.21.15.83:0/3899334 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8558003530 con 0x7f85700b4c10 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856effd640 1 -- 172.21.15.83:0/3899334 >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 msgr2=0x7f85700b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856effd640 1 --2- 172.21.15.83:0/3899334 >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 0x7f85700b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856effd640 1 -- 172.21.15.83:0/3899334 >> v1:172.21.15.83:6789/0 conn(0x7f85700b4c10 legacy=0x7f85700cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856effd640 1 -- 172.21.15.83:0/3899334 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857015e300 con 0x7f85700b5020 2024-05-17T22:05:32.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.132+0000 7f856ffff640 1 --2- 172.21.15.83:0/3899334 >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 0x7f85700b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:32.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f856effd640 1 --2- 172.21.15.83:0/3899334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b5020 0x7f85701431e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8570155100 tx=0x7f856401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769c6f425628f2a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f856e7fc640 1 -- 172.21.15.83:0/3899334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640b0070 con 0x7f85700b5020 2024-05-17T22:05:32.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f856e7fc640 1 -- 172.21.15.83:0/3899334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85640ab600 con 0x7f85700b5020 2024-05-17T22:05:32.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f856e7fc640 1 -- 172.21.15.83:0/3899334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640ab9d0 con 0x7f85700b5020 2024-05-17T22:05:32.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 -- 172.21.15.83:0/3899334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b5020 msgr2=0x7f85701431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 --2- 172.21.15.83:0/3899334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b5020 0x7f85701431e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8570155100 tx=0x7f856401c3f0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 -- 172.21.15.83:0/3899334 shutdown_connections 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 --2- 172.21.15.83:0/3899334 >> v2:172.21.15.155:3300/0 conn(0x7f8570143720 0x7f85700b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 --2- 172.21.15.83:0/3899334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b5020 0x7f85701431e0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 -- 172.21.15.83:0/3899334 >> 172.21.15.83:0/3899334 conn(0x7f85700ce6e0 msgr2=0x7f85700ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 -- 172.21.15.83:0/3899334 shutdown_connections 2024-05-17T22:05:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.136+0000 7f8575d85640 1 -- 172.21.15.83:0/3899334 wait complete. 2024-05-17T22:05:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 Processor -- start 2024-05-17T22:05:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- start start 2024-05-17T22:05:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 0x7f857013c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 0x7f85701fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85700b1560 con 0x7f85700b4c10 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85700689d0 con 0x7f85700b5020 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f85700b16e0 con 0x7f8570143720 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56062/0 (socket says 172.21.15.83:56062) 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 -- 172.21.15.83:0/4273028850 learned_addr learned my addr 172.21.15.83:0/4273028850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856effd640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 0x7f857013c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856ffff640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 0x7f85701fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 -- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 msgr2=0x7f85701fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 0x7f85701fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 -- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 msgr2=0x7f857013c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 0x7f857013c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 -- 172.21.15.83:0/4273028850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857013a070 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856effd640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 0x7f857013c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856ffff640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 0x7f85701fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f856f7fe640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f856000c730 tx=0x7f856000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560017070 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8560001d40 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857013cd40 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560005cc0 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.140+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857013d300 con 0x7f85700b4c10 2024-05-17T22:05:32.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8570160fb0 con 0x7f85700b4c10 2024-05-17T22:05:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f856001a070 con 0x7f85700b4c10 2024-05-17T22:05:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f8557fff640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f856008e4d0 con 0x7f85700b4c10 2024-05-17T22:05:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f856effd640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.144+0000 7f856effd640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 0x7f855c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85640ad000 tx=0x7f8564008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.148+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8570143b30 con 0x7f85700b4c10 2024-05-17T22:05:32.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.352+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f85700ce1d0 con 0x7f85700b4c10 2024-05-17T22:05:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.356+0000 7f8557fff640 1 -- 172.21.15.83:0/4273028850 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f856008ce20 con 0x7f85700b4c10 2024-05-17T22:05:32.347 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T22:05:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T22:05:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T22:05:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T22:05:32.348 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T22:05:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 msgr2=0x7f855c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 0x7f855c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85640ad000 tx=0x7f8564008040 comp rx=0 tx=0).stop 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 msgr2=0x7f8570138fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f856000c730 tx=0x7f856000cbf0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 shutdown_connections 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8570143720 0x7f85701fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85700b5020 0x7f857013c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 --2- 172.21.15.83:0/4273028850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85700b4c10 0x7f8570138fe0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 >> 172.21.15.83:0/4273028850 conn(0x7f85700ce6e0 msgr2=0x7f85700b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 shutdown_connections 2024-05-17T22:05:32.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.360+0000 7f8575d85640 1 -- 172.21.15.83:0/4273028850 wait complete. 2024-05-17T22:05:32.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 Processor -- start 2024-05-17T22:05:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 -- start start 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51d80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f51d806e3d0 0x7f51d80d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d80d34e0 con 0x7f51d80d1150 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f51d80d3780 con 0x7f51d806e3d0 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51ded66640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51d80d3a60 con 0x7f51d815ffa0 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d7fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51d80d1150 0x7f51d806dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56172/0 (socket says 172.21.15.83:56172) 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d7fff640 1 -- 172.21.15.83:0/2517249626 learned_addr learned my addr 172.21.15.83:0/2517249626 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51dcb02640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51d80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51dd303640 1 --2- 172.21.15.83:0/2517249626 >> v2:172.21.15.155:3300/0 conn(0x7f51d806e3d0 0x7f51d80d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422247478 0 0) 0x7f51d80d34e0 con 0x7f51d80d1150 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c0003530 con 0x7f51d80d1150 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1884673371 0 0) 0x7f51c0003530 con 0x7f51d80d1150 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 msgr2=0x7f51d80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51d80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 >> v2:172.21.15.155:3300/0 conn(0x7f51d806e3d0 msgr2=0x7f51d80d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 --2- 172.21.15.83:0/2517249626 >> v2:172.21.15.155:3300/0 conn(0x7f51d806e3d0 0x7f51d80d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d80d3d40 con 0x7f51d80d1150 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f51cc001700 con 0x7f51d80d1150 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 >> v1:172.21.15.83:6789/0 conn(0x7f51d80d1150 legacy=0x7f51d806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.524+0000 7f51d77fe640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51c0003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0003fe0 0x7f51c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80d3780 con 0x7f51c0003fe0 2024-05-17T22:05:32.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80d3a60 con 0x7f51c0004d50 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51c000bad0 con 0x7f51d815ffa0 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0003fe0 0x7f51c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51dcb02640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51c0003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51dd303640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 -- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 msgr2=0x7f51c0003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d815ffa0 0x7f51c0003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 -- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0004d50 msgr2=0x7f51c0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 -- 172.21.15.83:0/2517249626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c0003530 con 0x7f51c0003fe0 2024-05-17T22:05:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51dd303640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d7fff640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0003fe0 0x7f51c0004810 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c240 tx=0x7f51cc01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67aac721db4566c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007d60 con 0x7f51c0003fe0 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc005ae0 con 0x7f51c0003fe0 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51d77fe640 1 -- 172.21.15.83:0/2517249626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007070 con 0x7f51c0003fe0 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 -- 172.21.15.83:0/2517249626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0003fe0 msgr2=0x7f51c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 --2- 172.21.15.83:0/2517249626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c0003fe0 0x7f51c0004810 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c240 tx=0x7f51cc01f350 comp rx=0 tx=0).stop 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51dd303640 1 -- 172.21.15.83:0/2517249626 reap_dead start 2024-05-17T22:05:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 -- 172.21.15.83:0/2517249626 shutdown_connections 2024-05-17T22:05:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 -- 172.21.15.83:0/2517249626 >> 172.21.15.83:0/2517249626 conn(0x7f51d80d28f0 msgr2=0x7f51d80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 -- 172.21.15.83:0/2517249626 shutdown_connections 2024-05-17T22:05:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.528+0000 7f51ded66640 1 -- 172.21.15.83:0/2517249626 wait complete. 2024-05-17T22:05:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 Processor -- start 2024-05-17T22:05:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- start start 2024-05-17T22:05:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0003fe0 0x7f51d8157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d8161f30 0x7f51d8151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80d17e0 con 0x7f51d8150480 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80d1660 con 0x7f51c0003fe0 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f51d80d0c10 con 0x7f51d8161f30 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56080/0 (socket says 172.21.15.83:56080) 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 -- 172.21.15.83:0/783782273 learned_addr learned my addr 172.21.15.83:0/783782273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51dcb02640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0003fe0 0x7f51d8157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51dd303640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d8161f30 0x7f51d8151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 -- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d8161f30 msgr2=0x7f51d8151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d8161f30 0x7f51d8151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 -- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0003fe0 msgr2=0x7f51d8157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0003fe0 0x7f51d8157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 -- 172.21.15.83:0/783782273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d8162340 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d7fff640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000f70 tx=0x7f51c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0047f0 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc004990 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d8155720 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004d60 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.532+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d8152aa0 con 0x7f51d8150480 2024-05-17T22:05:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d8157a30 con 0x7f51d8150480 2024-05-17T22:05:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51cc007070 con 0x7f51d8150480 2024-05-17T22:05:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51d57fa640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 0x7f51c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51dcb02640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 0x7f51c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f51cc12c7d0 con 0x7f51d8150480 2024-05-17T22:05:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.536+0000 7f51dcb02640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 0x7f51c4071940 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009160 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.540+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f51d8162340 con 0x7f51d8150480 2024-05-17T22:05:32.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.724+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d806dcc0 con 0x7f51d8150480 2024-05-17T22:05:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.724+0000 7f51d57fa640 1 -- 172.21.15.83:0/783782273 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5189 (secure 0 0 0) 0x7f51cc0b7070 con 0x7f51d8150480 2024-05-17T22:05:32.715 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:32.715 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:05:29.852155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7407":{"gid":7407,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1996242614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1996242614},{"type":"v1","addr":"172.21.15.83:6836","nonce":1996242614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7407,"qdb_cluster":[7407]},"id":7}]} 2024-05-17T22:05:32.716 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 81 2024-05-17T22:05:32.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 msgr2=0x7f51c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 0x7f51c4071940 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009160 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 msgr2=0x7f51d81619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000f70 tx=0x7f51c0005840 comp rx=0 tx=0).stop 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 shutdown_connections 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51d8161f30 0x7f51d8151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51d8150480 0x7f51d81619f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f51c406f430 0x7f51c4071940 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 --2- 172.21.15.83:0/783782273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51c0003fe0 0x7f51d8157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 >> 172.21.15.83:0/783782273 conn(0x7f51d80d28f0 msgr2=0x7f51d80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 shutdown_connections 2024-05-17T22:05:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:32.728+0000 7f51ded66640 1 -- 172.21.15.83:0/783782273 wait complete. 2024-05-17T22:05:33.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:05:26.778012+0000', 'modified': '2024-05-17T22:05:29.852155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7407': {'gid': 7407, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/1996242614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 1996242614}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 1996242614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7407, 'qdb_cluster': [7407]} 2024-05-17T22:05:33.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:05:33.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:05:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 Processor -- start 2024-05-17T22:05:33.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 -- start start 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f639806c790 0x7f639806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63980b5670 0x7f63980b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63980b5fc0 con 0x7f639815fcc0 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f63980711a0 con 0x7f63980b5670 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639c8d2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f639806cc90 con 0x7f639806c790 2024-05-17T22:05:33.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f6396d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63980b5670 0x7f63980b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f6396d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63980b5670 0x7f63980b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56092/0 (socket says 172.21.15.83:56092) 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f6396d9d640 1 -- 172.21.15.83:0/2802210707 learned_addr learned my addr 172.21.15.83:0/2802210707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.304+0000 7f639659c640 1 --2- 172.21.15.83:0/2802210707 >> v2:172.21.15.155:3300/0 conn(0x7f639806c790 0x7f639806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f6396d9d640 1 -- 172.21.15.83:0/2802210707 >> v2:172.21.15.155:3300/0 conn(0x7f639806c790 msgr2=0x7f639806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f6396d9d640 1 --2- 172.21.15.83:0/2802210707 >> v2:172.21.15.155:3300/0 conn(0x7f639806c790 0x7f639806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f6396d9d640 1 -- 172.21.15.83:0/2802210707 >> v1:172.21.15.155:6789/0 conn(0x7f639815fcc0 legacy=0x7f6398070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f6396d9d640 1 -- 172.21.15.83:0/2802210707 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639806cf70 con 0x7f63980b5670 2024-05-17T22:05:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f6396d9d640 1 --2- 172.21.15.83:0/2802210707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63980b5670 0x7f63980b5a80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f639806f900 tx=0x7f639001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb57c6b89219d66 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639559a640 1 -- 172.21.15.83:0/2802210707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900a5880 con 0x7f63980b5670 2024-05-17T22:05:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639559a640 1 -- 172.21.15.83:0/2802210707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900a5a20 con 0x7f63980b5670 2024-05-17T22:05:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639559a640 1 -- 172.21.15.83:0/2802210707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900a5df0 con 0x7f63980b5670 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 -- 172.21.15.83:0/2802210707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63980b5670 msgr2=0x7f63980b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2802210707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63980b5670 0x7f63980b5a80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f639806f900 tx=0x7f639001c9c0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 -- 172.21.15.83:0/2802210707 shutdown_connections 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2802210707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63980b5670 0x7f63980b5a80 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2802210707 >> v2:172.21.15.155:3300/0 conn(0x7f639806c790 0x7f639806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 -- 172.21.15.83:0/2802210707 >> 172.21.15.83:0/2802210707 conn(0x7f63980d20b0 msgr2=0x7f63980d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 -- 172.21.15.83:0/2802210707 shutdown_connections 2024-05-17T22:05:33.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.308+0000 7f639c8d2640 1 -- 172.21.15.83:0/2802210707 wait complete. 2024-05-17T22:05:33.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 Processor -- start 2024-05-17T22:05:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- start start 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63980b5670 0x7f639815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f639815fcc0 0x7f639815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63980af910 con 0x7f639806c790 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63980af790 con 0x7f63980b5670 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f63980afa90 con 0x7f639815fcc0 2024-05-17T22:05:33.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56098/0 (socket says 172.21.15.83:56098) 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 -- 172.21.15.83:0/2479810933 learned_addr learned my addr 172.21.15.83:0/2479810933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f6395d9b640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63980b5670 0x7f639815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 -- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f639815fcc0 msgr2=0x7f639815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f639815fcc0 0x7f639815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 -- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63980b5670 msgr2=0x7f639815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63980b5670 0x7f639815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 -- 172.21.15.83:0/2479810933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63980b2270 con 0x7f639806c790 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639659c640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d5f0 tx=0x7f638c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f6386ffd640 1 -- 172.21.15.83:0/2479810933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0125f0 con 0x7f639806c790 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f6386ffd640 1 -- 172.21.15.83:0/2479810933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c019070 con 0x7f639806c790 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639814a700 con 0x7f639806c790 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f6386ffd640 1 -- 172.21.15.83:0/2479810933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0129b0 con 0x7f639806c790 2024-05-17T22:05:33.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.312+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639814acc0 con 0x7f639806c790 2024-05-17T22:05:33.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f6386ffd640 1 -- 172.21.15.83:0/2479810933 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f638c012b50 con 0x7f639806c790 2024-05-17T22:05:33.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f6386ffd640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f6386ffd640 1 -- 172.21.15.83:0/2479810933 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f638c08f750 con 0x7f639806c790 2024-05-17T22:05:33.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f6395d9b640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f6395d9b640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6388007fd0 tx=0x7f6388009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f63a3b4c000 1 Processor -- start 2024-05-17T22:05:33.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.316+0000 7f63a3b4c000 1 -- start start 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cab435eca0 con 0x55cab4359d30 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cab42d1110 con 0x55cab437afa0 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cab42c0cb0 con 0x55cab4370600 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56110/0 (socket says 172.21.15.83:56110) 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 -- 172.21.15.83:0/2789133708 learned_addr learned my addr 172.21.15.83:0/2789133708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f639659c640 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6395d9b640 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 -- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 msgr2=0x55cab44c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 -- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 msgr2=0x55cab44c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 -- 172.21.15.83:0/2789133708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab4285370 con 0x55cab4359d30 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6396d9d640 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f63900195d0 tx=0x7f6390004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2139ffaf489b1af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6377fff640 1 -- 172.21.15.83:0/2789133708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390009590 con 0x55cab4359d30 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6377fff640 1 -- 172.21.15.83:0/2789133708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900090d0 con 0x55cab4359d30 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f6377fff640 1 -- 172.21.15.83:0/2789133708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900a5ac0 con 0x55cab4359d30 2024-05-17T22:05:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- 172.21.15.83:0/2789133708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 msgr2=0x55cab44c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f63900195d0 tx=0x7f6390004a50 comp rx=0 tx=0).stop 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- 172.21.15.83:0/2789133708 shutdown_connections 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab44c1330 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/2789133708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- 172.21.15.83:0/2789133708 >> 172.21.15.83:0/2789133708 conn(0x55cab3f82460 msgr2=0x55cab4475e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- 172.21.15.83:0/2789133708 shutdown_connections 2024-05-17T22:05:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- 172.21.15.83:0/2789133708 wait complete. 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 -- start start 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.320+0000 7f63a3b4c000 1 Processor -- start 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4359d30 0x55cab44c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cab427cd40 con 0x55cab437afa0 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cab42c4cf0 con 0x55cab4359d30 2024-05-17T22:05:33.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cab4269e90 con 0x55cab4370600 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 10 client.0 _ll_register_callbacks cb 0x55cab429efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56120/0 (socket says 172.21.15.83:56120) 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 -- 172.21.15.83:0/843555848 learned_addr learned my addr 172.21.15.83:0/843555848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f639659c640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4359d30 0x55cab44c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6395d9b640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 -- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 msgr2=0x55cab44c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 -- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4359d30 msgr2=0x55cab44c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4359d30 0x55cab44c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 -- 172.21.15.83:0/843555848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab428f6e0 con 0x55cab437afa0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f6396d9d640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f63900ba040 tx=0x7f639001b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7218ee2fbea7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390009590 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f63900bd070 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63900090d0 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f63900cc050 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 10 client.7539 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55cab427f690 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900090d0 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f63900cf020 con 0x55cab437afa0 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 10 client.7539 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 10 client.7539 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 20 client.7539 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f677d0 0x55cab44d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 4 client.7539 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:05:33.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cab425ca00 con 0x55cab3f677d0 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f639659c640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f677d0 0x55cab44d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 10 client.7539 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f639659c640 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f677d0 0x55cab44d1f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feacd6850fc02a76 server_cookie=62b73cc2cd10c539 in_seq=0 out_seq=0 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 1 -- 172.21.15.83:0/843555848 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cab425ca00 con 0x55cab3f677d0 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 10 client.7539 handle_command_reply: tid=0 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 client.7539 shutdown 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 msgr2=0x55cab44c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f63900ba040 tx=0x7f639001b200 comp rx=0 tx=0).stop 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 shutdown_connections 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f677d0 0x55cab44d1f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab437afa0 0x55cab44c85b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4370600 0x55cab44c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/843555848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4359d30 0x55cab44c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63757fa640 0 client.7539 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 >> 172.21.15.83:0/843555848 conn(0x55cab3f82460 msgr2=0x55cab44765b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 shutdown_connections 2024-05-17T22:05:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 1 -- 172.21.15.83:0/843555848 wait complete. 2024-05-17T22:05:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.324+0000 7f63a3b4c000 20 client.7539 trim_cache size 0 max 16384 2024-05-17T22:05:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 Processor -- start 2024-05-17T22:05:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- start start 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4370600 0x55cab452f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab437afa0 0x55cab452fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b1f60 con 0x55cab4359d30 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63981fc950 con 0x55cab4370600 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cab426b900 con 0x55cab437afa0 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56130/0 (socket says 172.21.15.83:56130) 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 -- 172.21.15.83:0/1731588718 learned_addr learned my addr 172.21.15.83:0/1731588718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 -- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab437afa0 msgr2=0x55cab452fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab437afa0 0x55cab452fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 -- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4370600 msgr2=0x55cab452f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4370600 0x55cab452f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 -- 172.21.15.83:0/1731588718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cab4347cb0 con 0x55cab4359d30 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f639659c640 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f638c058530 tx=0x7f638c007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bf6ce89c12dcee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63777fe640 1 -- 172.21.15.83:0/1731588718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c05bf40 con 0x55cab4359d30 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63777fe640 1 -- 172.21.15.83:0/1731588718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c0602f0 con 0x55cab4359d30 2024-05-17T22:05:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63777fe640 1 -- 172.21.15.83:0/1731588718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c064da0 con 0x55cab4359d30 2024-05-17T22:05:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1731588718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 msgr2=0x55cab452f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f638c058530 tx=0x7f638c007440 comp rx=0 tx=0).stop 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1731588718 shutdown_connections 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab437afa0 0x55cab452fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab4370600 0x55cab452f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1731588718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4359d30 0x55cab452f040 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1731588718 >> 172.21.15.83:0/1731588718 conn(0x55cab3f82460 msgr2=0x55cab44c00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1731588718 shutdown_connections 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.344+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1731588718 wait complete. 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- start start 2024-05-17T22:05:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 Processor -- start 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4359d30 0x55cab4479700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cab42c4000 con 0x55cab4370600 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cab427cd40 con 0x55cab437afa0 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cab42cfee0 con 0x55cab4359d30 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 10 client.0 _ll_register_callbacks cb 0x7f6390009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56144/0 (socket says 172.21.15.83:56144) 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 -- 172.21.15.83:0/1422630371 learned_addr learned my addr 172.21.15.83:0/1422630371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6396d9d640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f639659c640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4359d30 0x55cab4479700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 -- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4359d30 msgr2=0x55cab4479700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4359d30 0x55cab4479700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 -- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 msgr2=0x55cab44eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 -- 172.21.15.83:0/1422630371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cab42804f0 con 0x55cab4370600 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f6395d9b640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f63880078a0 tx=0x7f6388000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849706de9c2e2b5d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388015b00 con 0x55cab4370600 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6388015ca0 con 0x55cab4370600 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638c0602f0 con 0x55cab4370600 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f6388022430 con 0x55cab4370600 2024-05-17T22:05:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 10 client.7545 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55cab428a4f0 con 0x55cab4370600 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c0602f0 con 0x55cab4370600 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f63880225d0 con 0x55cab4370600 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 10 client.7545 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 10 client.7545 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 20 client.7545 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f55830 0x55cab45332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 4 client.7545 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "status"}) -- 0x55cab42d3060 con 0x55cab3f55830 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f639659c640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f55830 0x55cab45332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 10 client.7545 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f639659c640 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f55830 0x55cab45332d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2790a5ae7498dbd3 server_cookie=a590f3d288ade95e in_seq=0 out_seq=0 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 1 -- 172.21.15.83:0/1422630371 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cab42d3060 con 0x55cab3f55830 2024-05-17T22:05:33.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 10 client.7545 handle_command_reply: tid=0 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 client.7545 shutdown 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 msgr2=0x55cab4479c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f63880078a0 tx=0x7f6388000ee0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 shutdown_connections 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55cab3f55830 0x55cab45332d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cab437afa0 0x55cab44eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cab4370600 0x55cab4479c40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63757fa640 0 client.7545 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 --2- 172.21.15.83:0/1422630371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cab4359d30 0x55cab4479700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.348+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 >> 172.21.15.83:0/1422630371 conn(0x55cab3f82460 msgr2=0x55cab4475460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 shutdown_connections 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f63a3b4c000 1 -- 172.21.15.83:0/1422630371 wait complete. 2024-05-17T22:05:33.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f63a3b4c000 20 client.7545 trim_cache size 0 max 16384 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7407, 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8953945570000004, 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 81, 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 96, 2024-05-17T22:05:33.347 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 96, 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 10.915513388999999 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 msgr2=0x7f6378071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6388007fd0 tx=0x7f6388009040 comp rx=0 tx=0).stop 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 msgr2=0x7f639814bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f638c00d5f0 tx=0x7f638c00dab0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 shutdown_connections 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f639815fcc0 0x7f639815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63980b5670 0x7f639815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 --2- 172.21.15.83:0/2479810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f639806c790 0x7f639814bea0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 >> 172.21.15.83:0/2479810933 conn(0x7f63980d20b0 msgr2=0x7f63980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.349 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:05:33.352+0000 7fee20eba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:05:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 shutdown_connections 2024-05-17T22:05:33.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.352+0000 7f639c8d2640 1 -- 172.21.15.83:0/2479810933 wait complete. 2024-05-17T22:05:33.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:05:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.500+0000 7f71d8cd7640 1 Processor -- start 2024-05-17T22:05:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.500+0000 7f71d8cd7640 1 -- start start 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d8cd7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71d41461c0 0x7f71d415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d8cd7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71d40706e0 0x7f71d41605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d8cd7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d41465d0 con 0x7f71d415e6b0 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d8cd7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f71d406ef30 con 0x7f71d41461c0 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d8cd7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f71d406f210 con 0x7f71d40706e0 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d1d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f71d415e6b0 0x7f71d406ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56188/0 (socket says 172.21.15.83:56188) 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71d41461c0 0x7f71d415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d1d9b640 1 -- 172.21.15.83:0/2326211194 learned_addr learned my addr 172.21.15.83:0/2326211194 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d159a640 1 -- 172.21.15.83:0/2326211194 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666653402 0 0) 0x7f71d41465d0 con 0x7f71d415e6b0 2024-05-17T22:05:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d159a640 1 -- 172.21.15.83:0/2326211194 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71bc003530 con 0x7f71d415e6b0 2024-05-17T22:05:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d259c640 1 -- 172.21.15.83:0/2326211194 >> v2:172.21.15.155:3300/0 conn(0x7f71d40706e0 msgr2=0x7f71d41605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d2d9d640 1 --2- 172.21.15.83:0/2326211194 >> v2:172.21.15.155:3300/0 conn(0x7f71d40706e0 0x7f71d41605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d259c640 1 --2- 172.21.15.83:0/2326211194 >> v2:172.21.15.155:3300/0 conn(0x7f71d40706e0 0x7f71d41605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d259c640 1 -- 172.21.15.83:0/2326211194 >> v1:172.21.15.83:6789/0 conn(0x7f71d415e6b0 legacy=0x7f71d406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.504+0000 7f71d259c640 1 -- 172.21.15.83:0/2326211194 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d406f4f0 con 0x7f71d41461c0 2024-05-17T22:05:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d259c640 1 --2- 172.21.15.83:0/2326211194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d41461c0 0x7f71d415e170 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f71d4072c00 tx=0x7f71c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eb434c0f740e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d159a640 1 -- 172.21.15.83:0/2326211194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b2070 con 0x7f71d41461c0 2024-05-17T22:05:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d159a640 1 -- 172.21.15.83:0/2326211194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c40a9070 con 0x7f71d41461c0 2024-05-17T22:05:33.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d159a640 1 -- 172.21.15.83:0/2326211194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40ad860 con 0x7f71d41461c0 2024-05-17T22:05:33.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- 172.21.15.83:0/2326211194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d41461c0 msgr2=0x7f71d415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/2326211194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d41461c0 0x7f71d415e170 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f71d4072c00 tx=0x7f71c401c450 comp rx=0 tx=0).stop 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- 172.21.15.83:0/2326211194 shutdown_connections 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/2326211194 >> v2:172.21.15.155:3300/0 conn(0x7f71d40706e0 0x7f71d41605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/2326211194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d41461c0 0x7f71d415e170 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- 172.21.15.83:0/2326211194 >> 172.21.15.83:0/2326211194 conn(0x7f71d406cd90 msgr2=0x7f71d40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- 172.21.15.83:0/2326211194 shutdown_connections 2024-05-17T22:05:33.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- 172.21.15.83:0/2326211194 wait complete. 2024-05-17T22:05:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 Processor -- start 2024-05-17T22:05:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.508+0000 7f71d8cd7640 1 -- start start 2024-05-17T22:05:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40706e0 0x7f71d413c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d41461c0 0x7f71d40b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4169940 con 0x7f71d415e6b0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71d40d12b0 con 0x7f71d41461c0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71d4169ac0 con 0x7f71d40706e0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56170/0 (socket says 172.21.15.83:56170) 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 -- 172.21.15.83:0/1007700214 learned_addr learned my addr 172.21.15.83:0/1007700214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d1d9b640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d41461c0 0x7f71d40b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d259c640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40706e0 0x7f71d413c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 -- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40706e0 msgr2=0x7f71d413c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40706e0 0x7f71d413c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 -- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d41461c0 msgr2=0x7f71d40b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d41461c0 0x7f71d40b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 -- 172.21.15.83:0/1007700214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d415c540 con 0x7f71d415e6b0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d2d9d640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cc60 tx=0x7f71cc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0039a0 con 0x7f71d415e6b0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc003b40 con 0x7f71d415e6b0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc004020 con 0x7f71d415e6b0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d415c000 con 0x7f71d415e6b0 2024-05-17T22:05:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d40b14a0 con 0x7f71d415e6b0 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d40b1ea0 con 0x7f71d415e6b0 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f71cc0041c0 con 0x7f71d415e6b0 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 0x7f71b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71d259c640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 0x7f71b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.512+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f71cc08fad0 con 0x7f71d415e6b0 2024-05-17T22:05:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.516+0000 7f71d259c640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 0x7f71b4071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f71c40af000 tx=0x7f71c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:33.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.516+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f71cc08fe20 con 0x7f71d415e6b0 2024-05-17T22:05:33.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71d40b0f50 con 0x7f71d415e6b0 2024-05-17T22:05:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71c2ffd640 1 -- 172.21.15.83:0/1007700214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f71d415c540 con 0x7f71d415e6b0 2024-05-17T22:05:33.691 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 msgr2=0x7f71b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 0x7f71b4071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f71c40af000 tx=0x7f71c4008040 comp rx=0 tx=0).stop 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 msgr2=0x7f71d40b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cc60 tx=0x7f71cc00a5c0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 shutdown_connections 2024-05-17T22:05:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d415e6b0 0x7f71d40b27a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f71b406f400 0x7f71b4071910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71d41461c0 0x7f71d40b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 --2- 172.21.15.83:0/1007700214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71d40706e0 0x7f71d413c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 >> 172.21.15.83:0/1007700214 conn(0x7f71d406cd90 msgr2=0x7f71d415fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 shutdown_connections 2024-05-17T22:05:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33.700+0000 7f71d8cd7640 1 -- 172.21.15.83:0/1007700214 wait complete. 2024-05-17T22:05:33.711 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:05:33.711 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:05:33.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:33.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:33.712 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:05:33.718 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:05:33.718 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.719 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42010sec preferred_lft 42010sec 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:05:33.720 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:33.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:33.721 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:05:33.772 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:05:33.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:33.774 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:05:33.826 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:05:33.827 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:05:33.827 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:33.827 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:33.827 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:33.827 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:05:33.827 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:05:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:05:33.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:33.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:33.946 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:05:33.999 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:05:33.999 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:05:33.999 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:33.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:33.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:05:34.054 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:05:34.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:05:34.054 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:34.102 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:34.102 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:05:34.103 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:34.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:34.207 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:34.256 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:34.257 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:34.258 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:34.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:34.259 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:34.306 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:05:34.306 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:05:34.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:34.366 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:34.417 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:34.418 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:34.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:34.418 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:34.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:05:34.468+0000 7f4c6e529580 -1 init, newargv = 0x55a0130bcb10 newargc=15 2024-05-17T22:05:34.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46358]: starting ceph client 2024-05-17T22:05:34.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46358]: starting fuse 2024-05-17T22:05:35.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:35.578 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:35.629 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:35.629 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:35.629 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:35.629 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:35.630 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:35.631 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:35.632 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:35.632 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:35.632 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:35.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:35.633 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:35.678 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:05:35.678 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:05:35.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:05:35.678 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:35.679 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:05:35.680 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:35.800 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.46358.asok 2024-05-17T22:05:35.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:35.812 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46358.asok 2024-05-17T22:05:35.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:35.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46358.asok status 2024-05-17T22:05:35.995 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:05:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:05:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:05:35.995 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "46358", 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7560, 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7560 2024-05-17T22:05:35.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 548416731 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 548416731 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.7560 192.168.0.1:0/548416731", 2024-05-17T22:05:35.997 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/548416731", 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 81, 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 96, 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:05:35.998 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:36.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:36.012 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:05:36.012 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:36.018 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:05:36.018 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:36.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:36.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:36.074 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:36.074 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:36.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:36.127 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-17T22:05:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records in 2024-05-17T22:05:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records out 2024-05-17T22:05:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0717641 s, 87.7 MB/s 2024-05-17T22:05:36.258 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:36.258 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:36.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:05:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.472+0000 7f692607d640 1 Processor -- start 2024-05-17T22:05:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.472+0000 7f692607d640 1 -- start start 2024-05-17T22:05:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f692607d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f69200b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f692607d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200b5640 con 0x7f69200cdf40 2024-05-17T22:05:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f692607d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200b58e0 con 0x7f692015d700 2024-05-17T22:05:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f692607d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6920070870 con 0x7f69200cf830 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f69200cdf40 0x7f692015cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56202/0 (socket says 172.21.15.83:56202) 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691effd640 1 -- 172.21.15.83:0/4056988867 learned_addr learned my addr 172.21.15.83:0/4056988867 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f69200b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794686431 0 0) 0x7f69200b5640 con 0x7f69200cdf40 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6908003530 con 0x7f69200cdf40 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1705791192 0 0) 0x7f6908003530 con 0x7f69200cdf40 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 msgr2=0x7f69200b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f69200b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 >> v1:172.21.15.155:6789/0 conn(0x7f692015d700 legacy=0x7f69201663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692006f060 con 0x7f69200cdf40 2024-05-17T22:05:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f6914001700 con 0x7f69200cdf40 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 >> v1:172.21.15.83:6789/0 conn(0x7f69200cdf40 legacy=0x7f692015cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f6908003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6908004010 0x7f6908004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908004d50 0x7f6908005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f690800bbe0 con 0x7f6908004d50 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f690800bd90 con 0x7f6908004010 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f690800bf90 con 0x7f69200cf830 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908004d50 0x7f6908005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691effd640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6908004010 0x7f6908004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f6908003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 -- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 msgr2=0x7f6908003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f6908003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 -- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6908004010 msgr2=0x7f6908004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6908004010 0x7f6908004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691ffff640 1 -- 172.21.15.83:0/4056988867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6908003530 con 0x7f6908004d50 2024-05-17T22:05:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.476+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69200cf830 0x7f6908003a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f691ffff640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908004d50 0x7f6908005200 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6920074050 tx=0x7f691801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb09271fbfad7c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918001ae0 con 0x7f6908004d50 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6918003b80 con 0x7f6908004d50 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f691e7fc640 1 -- 172.21.15.83:0/4056988867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180050d0 con 0x7f6908004d50 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- 172.21.15.83:0/4056988867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908004d50 msgr2=0x7f6908005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 --2- 172.21.15.83:0/4056988867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6908004d50 0x7f6908005200 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6920074050 tx=0x7f691801c750 comp rx=0 tx=0).stop 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f691ffff640 1 -- 172.21.15.83:0/4056988867 reap_dead start 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- 172.21.15.83:0/4056988867 shutdown_connections 2024-05-17T22:05:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- 172.21.15.83:0/4056988867 >> 172.21.15.83:0/4056988867 conn(0x7f692006cd90 msgr2=0x7f69200b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- 172.21.15.83:0/4056988867 shutdown_connections 2024-05-17T22:05:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- 172.21.15.83:0/4056988867 wait complete. 2024-05-17T22:05:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 Processor -- start 2024-05-17T22:05:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.480+0000 7f692607d640 1 -- start start 2024-05-17T22:05:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6908004d50 0x7f6920148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69200731a0 0x7f6920158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f692015dc90 con 0x7f69200735b0 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f692015db10 con 0x7f69200731a0 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69200b3aa0 con 0x7f6908004d50 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56186/0 (socket says 172.21.15.83:56186) 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 -- 172.21.15.83:0/4083467328 learned_addr learned my addr 172.21.15.83:0/4083467328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691effd640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69200731a0 0x7f6920158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6908004d50 0x7f6920148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 -- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6908004d50 msgr2=0x7f6920148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6908004d50 0x7f6920148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 -- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69200731a0 msgr2=0x7f6920158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69200731a0 0x7f6920158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 -- 172.21.15.83:0/4083467328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6920157760 con 0x7f69200735b0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f691ffff640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f691801ae50 tx=0x7f6908005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918001ac0 con 0x7f69200735b0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6918003980 con 0x7f69200735b0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6920157a40 con 0x7f69200735b0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180050d0 con 0x7f69200735b0 2024-05-17T22:05:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6920158000 con 0x7f69200735b0 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69200b54a0 con 0x7f69200735b0 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.484+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6918005270 con 0x7f69200735b0 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.488+0000 7f6907fff640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 0x7f690c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.488+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f6918129010 con 0x7f69200735b0 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.488+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 0x7f690c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.488+0000 7f691f7fe640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 0x7f690c071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6910009ac0 tx=0x7f6910008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.492+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f69180c7080 con 0x7f69200735b0 2024-05-17T22:05:36.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69200d0e20 con 0x7f69200735b0 2024-05-17T22:05:36.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f6907fff640 1 -- 172.21.15.83:0/4083467328 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f69180164b0 con 0x7f69200735b0 2024-05-17T22:05:36.655 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:05:36.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 msgr2=0x7f690c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 0x7f690c071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6910009ac0 tx=0x7f6910008040 comp rx=0 tx=0).stop 2024-05-17T22:05:36.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 msgr2=0x7f69201591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f691801ae50 tx=0x7f6908005840 comp rx=0 tx=0).stop 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 shutdown_connections 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69200735b0 0x7f69201591d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f690c06f400 0x7f690c071910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69200731a0 0x7f6920158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 --2- 172.21.15.83:0/4083467328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6908004d50 0x7f6920148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 >> 172.21.15.83:0/4083467328 conn(0x7f692006cd90 msgr2=0x7f69200d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 shutdown_connections 2024-05-17T22:05:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.664+0000 7f692607d640 1 -- 172.21.15.83:0/4083467328 wait complete. 2024-05-17T22:05:36.676 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:36.676 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:36.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:05:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 Processor -- start 2024-05-17T22:05:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 -- start start 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f19500b4810 0x7f19500d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500d4170 con 0x7f19500d2000 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500d4450 con 0x7f19500d0290 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f1955c7a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f19500d4730 con 0x7f19500b4810 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f194ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f19500d2000 0x7f19500d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56206/0 (socket says 172.21.15.83:56206) 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f194ffff640 1 -- 172.21.15.83:0/659488660 learned_addr learned my addr 172.21.15.83:0/659488660 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f194f7fe640 1 --2- 172.21.15.83:0/659488660 >> v2:172.21.15.155:3300/0 conn(0x7f19500b4810 0x7f19500d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093673647 0 0) 0x7f19500d4170 con 0x7f19500d2000 2024-05-17T22:05:36.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.888+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1938003530 con 0x7f19500d2000 2024-05-17T22:05:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2907157712 0 0) 0x7f1938003530 con 0x7f19500d2000 2024-05-17T22:05:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 >> v2:172.21.15.155:3300/0 conn(0x7f19500b4810 msgr2=0x7f19500d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 --2- 172.21.15.83:0/659488660 >> v2:172.21.15.155:3300/0 conn(0x7f19500b4810 0x7f19500d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 >> v1:172.21.15.155:6789/0 conn(0x7f19500d0290 legacy=0x7f195015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500d4a10 con 0x7f19500d2000 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1948001a80 con 0x7f19500d2000 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 >> v1:172.21.15.83:6789/0 conn(0x7f19500d2000 legacy=0x7f19500d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1938003a60 0x7f1938003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f1938004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1938005260 0x7f193800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d4730 con 0x7f1938004810 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f193800c1f0 con 0x7f1938003a60 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f193800c3f0 con 0x7f1938005260 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f1938004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 -- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1938005260 msgr2=0x7f193800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194f7fe640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1938003a60 0x7f1938003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194ffff640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1938005260 0x7f193800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1938005260 0x7f193800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 -- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1938003a60 msgr2=0x7f1938003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1938003a60 0x7f1938003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 -- 172.21.15.83:0/659488660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1938003530 con 0x7f1938004810 2024-05-17T22:05:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194ffff640 1 -- 172.21.15.83:0/659488660 reap_dead start 2024-05-17T22:05:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194f7fe640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1938003a60 0x7f1938003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194effd640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f1938004c20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1944001790 tx=0x7f194401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de86a5adb9b7630b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19440013e0 con 0x7f1938004810 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19440052d0 con 0x7f1938004810 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f194e7fc640 1 -- 172.21.15.83:0/659488660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944003840 con 0x7f1938004810 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 -- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 msgr2=0x7f1938004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f1938004c20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1944001790 tx=0x7f194401c3d0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 -- 172.21.15.83:0/659488660 shutdown_connections 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 --2- 172.21.15.83:0/659488660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f1938004c20 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 -- 172.21.15.83:0/659488660 >> 172.21.15.83:0/659488660 conn(0x7f195006e010 msgr2=0x7f195006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 -- 172.21.15.83:0/659488660 shutdown_connections 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.892+0000 7f1955c7a640 1 -- 172.21.15.83:0/659488660 wait complete. 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 Processor -- start 2024-05-17T22:05:36.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 -- start start 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 0x7f1950158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19500cfb50 0x7f19501585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d5210 con 0x7f1938004810 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d06a0 con 0x7f19500cf740 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f1955c7a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19500d5390 con 0x7f19500cfb50 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56206/0 (socket says 172.21.15.83:56206) 2024-05-17T22:05:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 -- 172.21.15.83:0/1433800532 learned_addr learned my addr 172.21.15.83:0/1433800532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194effd640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 0x7f1950158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 -- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19500cfb50 msgr2=0x7f19501585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194ffff640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19500cfb50 0x7f19501585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19500cfb50 0x7f19501585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 -- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 msgr2=0x7f1950158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 0x7f1950158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 -- 172.21.15.83:0/1433800532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195015ab70 con 0x7f1938004810 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194effd640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 0x7f1950158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.896+0000 7f194f7fe640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f19400107c0 tx=0x7f1940010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000f040 con 0x7f1938004810 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940004220 con 0x7f1938004810 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195015ae50 con 0x7f1938004810 2024-05-17T22:05:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940010e70 con 0x7f1938004810 2024-05-17T22:05:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1950160da0 con 0x7f1938004810 2024-05-17T22:05:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19500d2410 con 0x7f1938004810 2024-05-17T22:05:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f194000e900 con 0x7f1938004810 2024-05-17T22:05:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 0x7f193c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f194effd640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 0x7f193c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f194008f450 con 0x7f1938004810 2024-05-17T22:05:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.900+0000 7f194effd640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 0x7f193c071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f19440ac000 tx=0x7f1944009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:36.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:36.904+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1940018030 con 0x7f1938004810 2024-05-17T22:05:37.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.076+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f195015b370 con 0x7f1938004810 2024-05-17T22:05:37.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.076+0000 7f1937fff640 1 -- 172.21.15.83:0/1433800532 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f194005c3d0 con 0x7f1938004810 2024-05-17T22:05:37.069 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 msgr2=0x7f193c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 0x7f193c071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f19440ac000 tx=0x7f1944009040 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 msgr2=0x7f195015a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f19400107c0 tx=0x7f1940010c80 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 shutdown_connections 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19500cfb50 0x7f19501585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f193c06f400 0x7f193c071910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19500cf740 0x7f1950158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 --2- 172.21.15.83:0/1433800532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1938004810 0x7f195015a0b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 >> 172.21.15.83:0/1433800532 conn(0x7f195006e010 msgr2=0x7f19500b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 shutdown_connections 2024-05-17T22:05:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:37.080+0000 7f1955c7a640 1 -- 172.21.15.83:0/1433800532 wait complete. 2024-05-17T22:05:37.090 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:05:37.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:37.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:37.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46358]: fuse finished with error 0 and tester_r 0 2024-05-17T22:05:37.104 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:43.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:05:43.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:43.206 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:05:43.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:43.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 Processor -- start 2024-05-17T22:05:43.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 -- start start 2024-05-17T22:05:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c0a0 0x7f1bd8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bd809c570 0x7f1bd80722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd80d1950 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd80d1670 con 0x7f1bd809c570 2024-05-17T22:05:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdfb32640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd8071280 con 0x7f1bd809c0a0 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bd809c570 0x7f1bd80722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1bd809c570 0x7f1bd80722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45104/0 (socket says 172.21.15.83:45104) 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 -- 172.21.15.83:0/520699408 learned_addr learned my addr 172.21.15.83:0/520699408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bdd0cd640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c0a0 0x7f1bd8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 -- 172.21.15.83:0/520699408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c0a0 msgr2=0x7f1bd8071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c0a0 0x7f1bd8071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 -- 172.21.15.83:0/520699408 >> v1:172.21.15.155:6789/0 conn(0x7f1bd80cf2e0 legacy=0x7f1bd809b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.424+0000 7f1bde0cf640 1 -- 172.21.15.83:0/520699408 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd8071560 con 0x7f1bd809c570 2024-05-17T22:05:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bde0cf640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd809c570 0x7f1bd80722a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009e50 tx=0x7f1bd001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f7eb0a2af15f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdc8cc640 1 -- 172.21.15.83:0/520699408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00b4070 con 0x7f1bd809c570 2024-05-17T22:05:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdc8cc640 1 -- 172.21.15.83:0/520699408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd00af400 con 0x7f1bd809c570 2024-05-17T22:05:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdc8cc640 1 -- 172.21.15.83:0/520699408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00af7d0 con 0x7f1bd809c570 2024-05-17T22:05:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 -- 172.21.15.83:0/520699408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd809c570 msgr2=0x7f1bd80722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd809c570 0x7f1bd80722a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009e50 tx=0x7f1bd001f210 comp rx=0 tx=0).stop 2024-05-17T22:05:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 -- 172.21.15.83:0/520699408 shutdown_connections 2024-05-17T22:05:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd809c570 0x7f1bd80722a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/520699408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c0a0 0x7f1bd8071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 -- 172.21.15.83:0/520699408 >> 172.21.15.83:0/520699408 conn(0x7f1bd80b5120 msgr2=0x7f1bd80b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:43.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 -- 172.21.15.83:0/520699408 shutdown_connections 2024-05-17T22:05:43.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.428+0000 7f1bdfb32640 1 -- 172.21.15.83:0/520699408 wait complete. 2024-05-17T22:05:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 Processor -- start 2024-05-17T22:05:43.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 -- start start 2024-05-17T22:05:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c570 0x7f1bd8160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bd80c8d40 0x7f1bd80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd80d1380 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd80d1200 con 0x7f1bd80c8d40 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdfb32640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd80b8de0 con 0x7f1bd809c570 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45114/0 (socket says 172.21.15.83:45114) 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 -- 172.21.15.83:0/891143723 learned_addr learned my addr 172.21.15.83:0/891143723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd8ce640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c570 0x7f1bd8160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bde0cf640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bd80c8d40 0x7f1bd80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 -- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c570 msgr2=0x7f1bd8160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c570 0x7f1bd8160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 -- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bd80c8d40 msgr2=0x7f1bd80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bd80c8d40 0x7f1bd80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 -- 172.21.15.83:0/891143723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81f5fa0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bdd0cd640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d600 tx=0x7f1bd400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4004240 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd40043e0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.432+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4003860 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81f6280 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81f6840 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd809b990 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1bd4003a00 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bce7fc640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 0x7f1bbc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bdd8ce640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 0x7f1bbc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f1bd408f4f0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.436+0000 7f1bdd8ce640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 0x7f1bbc071910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000f70 tx=0x7f1bc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.440+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1bd405c3f0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.636+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bd80b89b0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bce7fc640 1 -- 172.21.15.83:0/891143723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5189 (secure 0 0 0) 0x7f1bd405bcf0 con 0x7f1bd80cf2e0 2024-05-17T22:05:43.630 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:43.630 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:05:29.852155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7407":{"gid":7407,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1996242614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1996242614},{"type":"v1","addr":"172.21.15.83:6836","nonce":1996242614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7407,"qdb_cluster":[7407]},"id":7}]} 2024-05-17T22:05:43.631 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 81 2024-05-17T22:05:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 msgr2=0x7f1bbc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 0x7f1bbc071910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000f70 tx=0x7f1bc8009040 comp rx=0 tx=0).stop 2024-05-17T22:05:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 msgr2=0x7f1bd80c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d600 tx=0x7f1bd400dac0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 shutdown_connections 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1bbc06f400 0x7f1bbc071910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bd80c8d40 0x7f1bd80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd80cf2e0 0x7f1bd80c8800 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 --2- 172.21.15.83:0/891143723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1bd809c570 0x7f1bd8160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 >> 172.21.15.83:0/891143723 conn(0x7f1bd80b5120 msgr2=0x7f1bd8076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 shutdown_connections 2024-05-17T22:05:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.640+0000 7f1bdfb32640 1 -- 172.21.15.83:0/891143723 wait complete. 2024-05-17T22:05:43.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T22:05:43.831 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:05:43.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:05:43.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 Processor -- start 2024-05-17T22:05:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 -- start start 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcb80b68b0 0x7fbcb80b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb80b7200 con 0x7fbcb80b0180 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb815cdd0 con 0x7fbcb815c9c0 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcbd362640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb80b5020 con 0x7fbcb80b68b0 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbcb80b68b0 0x7fbcb80b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbcb80b0180 0x7fbcb80b0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37806/0 (socket says 172.21.15.83:37806) 2024-05-17T22:05:43.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb77fe640 1 -- 172.21.15.83:0/976483633 learned_addr learned my addr 172.21.15.83:0/976483633 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb67fc640 1 -- 172.21.15.83:0/976483633 >> v1:172.21.15.155:6789/0 conn(0x7fbcb815c9c0 legacy=0x7fbcb80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb67fc640 1 -- 172.21.15.83:0/976483633 >> v1:172.21.15.83:6789/0 conn(0x7fbcb80b0180 legacy=0x7fbcb80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.988+0000 7fbcb67fc640 1 -- 172.21.15.83:0/976483633 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80b5630 con 0x7fbcb80b68b0 2024-05-17T22:05:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcb67fc640 1 --2- 172.21.15.83:0/976483633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b68b0 0x7fbcb80b6cc0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097c0 tx=0x7fbca801e560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f188a007bb662434 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcb5ffb640 1 -- 172.21.15.83:0/976483633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca80ad660 con 0x7fbcb80b68b0 2024-05-17T22:05:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcb5ffb640 1 -- 172.21.15.83:0/976483633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca80b2070 con 0x7fbcb80b68b0 2024-05-17T22:05:43.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcb5ffb640 1 -- 172.21.15.83:0/976483633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca80ada20 con 0x7fbcb80b68b0 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 -- 172.21.15.83:0/976483633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b68b0 msgr2=0x7fbcb80b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 --2- 172.21.15.83:0/976483633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b68b0 0x7fbcb80b6cc0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097c0 tx=0x7fbca801e560 comp rx=0 tx=0).stop 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 -- 172.21.15.83:0/976483633 shutdown_connections 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 --2- 172.21.15.83:0/976483633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b68b0 0x7fbcb80b6cc0 secure :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097c0 tx=0x7fbca801e560 comp rx=0 tx=0).stop 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 -- 172.21.15.83:0/976483633 >> 172.21.15.83:0/976483633 conn(0x7fbcb809b8b0 msgr2=0x7fbcb80b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 -- 172.21.15.83:0/976483633 shutdown_connections 2024-05-17T22:05:43.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.992+0000 7fbcbd362640 1 -- 172.21.15.83:0/976483633 wait complete. 2024-05-17T22:05:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 Processor -- start 2024-05-17T22:05:43.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- start start 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb815c9c0 0x7fbcb8141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb814d6b0 0x7fbcb81f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb809bf60 con 0x7fbcb80b0180 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb809bde0 con 0x7fbcb815c9c0 2024-05-17T22:05:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb806c790 con 0x7fbcb814d6b0 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45142/0 (socket says 172.21.15.83:45142) 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 -- 172.21.15.83:0/1000886390 learned_addr learned my addr 172.21.15.83:0/1000886390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 -- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb814d6b0 msgr2=0x7fbcb81f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb67fc640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb815c9c0 0x7fbcb8141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb77fe640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb814d6b0 0x7fbcb81f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb814d6b0 0x7fbcb81f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 -- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb815c9c0 msgr2=0x7fbcb8141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb815c9c0 0x7fbcb8141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 -- 172.21.15.83:0/1000886390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb80b4e50 con 0x7fbcb80b0180 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcb6ffd640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbca000b410 tx=0x7fbca000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0015030 con 0x7fbcb80b0180 2024-05-17T22:05:43.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca0004020 con 0x7fbcb80b0180 2024-05-17T22:05:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81415a0 con 0x7fbcb80b0180 2024-05-17T22:05:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004450 con 0x7fbcb80b0180 2024-05-17T22:05:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb8151e90 con 0x7fbcb80b0180 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:43.996+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb80b6cc0 con 0x7fbcb80b0180 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.000+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbca0018070 con 0x7fbcb80b0180 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.000+0000 7fbcaf7fe640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 0x7fbc98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.000+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fbca008ee20 con 0x7fbcb80b0180 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.000+0000 7fbcb67fc640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 0x7fbc98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.000+0000 7fbcb67fc640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 0x7fbc98071910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbca80b1000 tx=0x7fbca8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.004+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbca005bd10 con 0x7fbcb80b0180 2024-05-17T22:05:44.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcb8071d80 con 0x7fbcb80b0180 2024-05-17T22:05:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcaf7fe640 1 -- 172.21.15.83:0/1000886390 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5189 (secure 0 0 0) 0x7fbca005b610 con 0x7fbcb80b0180 2024-05-17T22:05:44.179 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:44.179 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:05:29.852155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7407":{"gid":7407,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1996242614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1996242614},{"type":"v1","addr":"172.21.15.83:6836","nonce":1996242614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7407,"qdb_cluster":[7407]},"id":7}]} 2024-05-17T22:05:44.179 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 81 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 msgr2=0x7fbc98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 0x7fbc98071910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbca80b1000 tx=0x7fbca8008040 comp rx=0 tx=0).stop 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 msgr2=0x7fbcb8150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbca000b410 tx=0x7fbca000b8d0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 shutdown_connections 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbcb814d6b0 0x7fbcb81f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbc9806f400 0x7fbc98071910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcb815c9c0 0x7fbcb8141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 --2- 172.21.15.83:0/1000886390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbcb80b0180 0x7fbcb8150aa0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 >> 172.21.15.83:0/1000886390 conn(0x7fbcb809b8b0 msgr2=0x7fbcb8157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 shutdown_connections 2024-05-17T22:05:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.188+0000 7fbcbd362640 1 -- 172.21.15.83:0/1000886390 wait complete. 2024-05-17T22:05:44.200 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-17T22:05:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.272+0000 7f7575bab080 1 Processor -- start 2024-05-17T22:05:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.272+0000 7f7575bab080 1 -- start start 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 0x55988d7d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d7926a0 0x55988d78e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55988d6de1b0 con 0x55988d84c370 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55988d739ef0 con 0x55988d791c00 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55988d863f10 con 0x55988d7926a0 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45160/0 (socket says 172.21.15.83:45160) 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 -- 172.21.15.83:0/1577189926 learned_addr learned my addr 172.21.15.83:0/1577189926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756f68d640 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 0x55988d7d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 -- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d7926a0 msgr2=0x55988d78e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d7926a0 0x55988d78e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 -- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 msgr2=0x55988d7d8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 0x55988d7d8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 -- 172.21.15.83:0/1577189926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d6b8c10 con 0x55988d84c370 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756f68d640 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 0x55988d7d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f756000a3d0 tx=0x7f756000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca813b70aad196a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756de8a640 1 -- 172.21.15.83:0/1577189926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560014620 con 0x55988d84c370 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756de8a640 1 -- 172.21.15.83:0/1577189926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756000f030 con 0x55988d84c370 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f756de8a640 1 -- 172.21.15.83:0/1577189926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75600149e0 con 0x55988d84c370 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- 172.21.15.83:0/1577189926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 msgr2=0x55988d7d2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f756000a3d0 tx=0x7f756000a4c0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- 172.21.15.83:0/1577189926 shutdown_connections 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d84c370 0x55988d7d2c60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d7926a0 0x55988d78e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 --2- 172.21.15.83:0/1577189926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d791c00 0x55988d7d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- 172.21.15.83:0/1577189926 >> 172.21.15.83:0/1577189926 conn(0x55988d77ae40 msgr2=0x55988d792c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- 172.21.15.83:0/1577189926 shutdown_connections 2024-05-17T22:05:44.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- 172.21.15.83:0/1577189926 wait complete. 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 Processor -- start 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.276+0000 7f7575bab080 1 -- start start 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 0x55988d8fe980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d7926a0 0x55988d7b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d84c370 0x55988d8fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55988d7bef30 con 0x55988d791c00 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55988d7bf7c0 con 0x55988d7926a0 2024-05-17T22:05:44.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55988d7d8a00 con 0x55988d84c370 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 0x55988d8fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 0x55988d8fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45168/0 (socket says 172.21.15.83:45168) 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 -- 172.21.15.83:0/1378550064 learned_addr learned my addr 172.21.15.83:0/1378550064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ee8c640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d7926a0 0x55988d7b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d84c370 0x55988d8fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 -- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d84c370 msgr2=0x55988d8fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55988d84c370 0x55988d8fd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 -- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d7926a0 msgr2=0x55988d7b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d7926a0 0x55988d7b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 -- 172.21.15.83:0/1378550064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988d8fda60 con 0x55988d791c00 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ee8c640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988d7926a0 0x55988d7b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756f68d640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 0x55988d8fe980 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f756801d7c0 tx=0x7f756801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 -- 172.21.15.83:0/1378550064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680a9650 con 0x55988d791c00 2024-05-17T22:05:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 -- 172.21.15.83:0/1378550064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75680a97f0 con 0x55988d791c00 2024-05-17T22:05:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55988d8fdd00 con 0x55988d791c00 2024-05-17T22:05:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 -- 172.21.15.83:0/1378550064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680a9ba0 con 0x55988d791c00 2024-05-17T22:05:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988d8fe240 con 0x55988d791c00 2024-05-17T22:05:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 -- 172.21.15.83:0/1378550064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75680c53f0 con 0x55988d791c00 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f755c06dde0 0x7f755c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ce88640 1 -- 172.21.15.83:0/1378550064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f756812ba50 con 0x55988d791c00 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ee8c640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f755c06dde0 0x7f755c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55988d7b7180 0x55988d7b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55988d8fd620 con 0x55988d7b7180 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55988d7b7180 0x55988d7b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.280+0000 7f756ee8c640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f755c06dde0 0x7f755c0702f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f756400b450 tx=0x7f7564005560 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f756fe8e640 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55988d7b7180 0x55988d7b7590 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f756fe8e640 1 -- 172.21.15.83:0/1378550064 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f7560014420 con 0x55988d7b7180 2024-05-17T22:05:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55988d7b7180 msgr2=0x55988d7b7590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55988d7b7180 0x55988d7b7590 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f755c06dde0 msgr2=0x7f755c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f755c06dde0 0x7f755c0702f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f756400b450 tx=0x7f7564005560 comp rx=0 tx=0).stop 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 msgr2=0x55988d8fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 --2- 172.21.15.83:0/1378550064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988d791c00 0x55988d8fe980 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f756801d7c0 tx=0x7f756801c1d0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f756fe8e640 1 -- 172.21.15.83:0/1378550064 reap_dead start 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 shutdown_connections 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 >> 172.21.15.83:0/1378550064 conn(0x55988d77ae40 msgr2=0x55988d7d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 shutdown_connections 2024-05-17T22:05:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.284+0000 7f7575bab080 1 -- 172.21.15.83:0/1378550064 wait complete. 2024-05-17T22:05:44.281 INFO:teuthology.orchestra.run.smithi083.stdout:����� 2024-05-17T22:05:44.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:05:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 Processor -- start 2024-05-17T22:05:44.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 -- start start 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3dd806deb0 0x7f3dd80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd8160f40 con 0x7f3dd8160b30 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd80b5b80 con 0x7f3dd8156c40 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3ddd093640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd80b6140 con 0x7f3dd806deb0 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3dd806deb0 0x7f3dd80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3dd806deb0 0x7f3dd80b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45182/0 (socket says 172.21.15.83:45182) 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd759e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3dd8160b30 0x7f3dd8070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37818/0 (socket says 172.21.15.83:37818) 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 -- 172.21.15.83:0/1104834484 learned_addr learned my addr 172.21.15.83:0/1104834484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:44.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 -- 172.21.15.83:0/1104834484 >> v1:172.21.15.155:6789/0 conn(0x7f3dd8156c40 legacy=0x7f3dd806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 -- 172.21.15.83:0/1104834484 >> v1:172.21.15.83:6789/0 conn(0x7f3dd8160b30 legacy=0x7f3dd8070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.424+0000 7f3dd659c640 1 -- 172.21.15.83:0/1104834484 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd80b5e60 con 0x7f3dd806deb0 2024-05-17T22:05:44.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3dd659c640 1 --2- 172.21.15.83:0/1104834484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd806deb0 0x7f3dd80b55a0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80b62c0 tx=0x7f3dc001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f06f4fd0762f872 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3dd5d9b640 1 -- 172.21.15.83:0/1104834484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00b1070 con 0x7f3dd806deb0 2024-05-17T22:05:44.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3dd5d9b640 1 -- 172.21.15.83:0/1104834484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00ac600 con 0x7f3dd806deb0 2024-05-17T22:05:44.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3dd5d9b640 1 -- 172.21.15.83:0/1104834484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00ac9d0 con 0x7f3dd806deb0 2024-05-17T22:05:44.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 -- 172.21.15.83:0/1104834484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd806deb0 msgr2=0x7f3dd80b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 --2- 172.21.15.83:0/1104834484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd806deb0 0x7f3dd80b55a0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80b62c0 tx=0x7f3dc001c4b0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 -- 172.21.15.83:0/1104834484 shutdown_connections 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 --2- 172.21.15.83:0/1104834484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd806deb0 0x7f3dd80b55a0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 -- 172.21.15.83:0/1104834484 >> 172.21.15.83:0/1104834484 conn(0x7f3dd8073670 msgr2=0x7f3dd8073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 -- 172.21.15.83:0/1104834484 shutdown_connections 2024-05-17T22:05:44.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.428+0000 7f3ddd093640 1 -- 172.21.15.83:0/1104834484 wait complete. 2024-05-17T22:05:44.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 Processor -- start 2024-05-17T22:05:44.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- start start 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 0x7f3dd814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dd8163790 0x7f3dd81512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd8071300 con 0x7f3dd8156c40 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd8071180 con 0x7f3dd8163790 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd816da50 con 0x7f3dd8160b30 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45188/0 (socket says 172.21.15.83:45188) 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 -- 172.21.15.83:0/3648467476 learned_addr learned my addr 172.21.15.83:0/3648467476 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd759e640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dd8163790 0x7f3dd81512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd659c640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 0x7f3dd814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 -- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 msgr2=0x7f3dd814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 0x7f3dd814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 -- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dd8163790 msgr2=0x7f3dd81512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dd8163790 0x7f3dd81512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 -- 172.21.15.83:0/3648467476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd8162b80 con 0x7f3dd8156c40 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd659c640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 0x7f3dd814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dd6d9d640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c450 tx=0x7f3dd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00052d0 con 0x7f3dd8156c40 2024-05-17T22:05:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd0005470 con 0x7f3dd8156c40 2024-05-17T22:05:44.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd8161ae0 con 0x7f3dd8156c40 2024-05-17T22:05:44.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0012850 con 0x7f3dd8156c40 2024-05-17T22:05:44.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd81620a0 con 0x7f3dd8156c40 2024-05-17T22:05:44.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.432+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd80b55a0 con 0x7f3dd8156c40 2024-05-17T22:05:44.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.436+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3dd00129f0 con 0x7f3dd8156c40 2024-05-17T22:05:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.436+0000 7f3dcf7fe640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 0x7f3db8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.436+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f3dd008f7b0 con 0x7f3dd8156c40 2024-05-17T22:05:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.436+0000 7f3dd659c640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 0x7f3db8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.436+0000 7f3dd659c640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 0x7f3db8071940 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ae000 tx=0x7f3dc0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:44.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.440+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3dd005c8a0 con 0x7f3dd8156c40 2024-05-17T22:05:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.624+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3dd8076820 con 0x7f3dd8156c40 2024-05-17T22:05:44.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3dcf7fe640 1 -- 172.21.15.83:0/3648467476 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f3dd005c1a0 con 0x7f3dd8156c40 2024-05-17T22:05:44.618 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 msgr2=0x7f3db8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 0x7f3db8071940 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00ae000 tx=0x7f3dc0008040 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 msgr2=0x7f3dd81611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c450 tx=0x7f3dd000c910 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 shutdown_connections 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f3db806f430 0x7f3db8071940 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dd8163790 0x7f3dd81512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dd8160b30 0x7f3dd814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 --2- 172.21.15.83:0/3648467476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8156c40 0x7f3dd81611e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 >> 172.21.15.83:0/3648467476 conn(0x7f3dd8073670 msgr2=0x7f3dd80af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 shutdown_connections 2024-05-17T22:05:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44.628+0000 7f3ddd093640 1 -- 172.21.15.83:0/3648467476 wait complete. 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:05:44.639 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:44.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:44.639 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:05:44.645 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:05:44.645 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.646 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42000sec preferred_lft 42000sec 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.647 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:05:44.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:44.648 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:05:44.698 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:05:44.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:44.699 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:05:44.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:05:44.751 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:05:44.751 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:44.751 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:44.751 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:05:44.751 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:05:44.751 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:05:44.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:05:44.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:44.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:44.869 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:05:44.914 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:05:44.914 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:05:44.915 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:44.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:44.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:05:44.970 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:05:44.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:05:44.971 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:45.018 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:05:45.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:05:45.018 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:45.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:45.124 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:45.173 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:45.174 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:45.175 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:45.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:45.176 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:45.223 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:05:45.223 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:05:45.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:45.283 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:45.331 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:45.332 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:45.333 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:45.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:45.334 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:45.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:05:45.380+0000 7f059f594580 -1 init, newargv = 0x562c62bc1cc0 newargc=15 2024-05-17T22:05:45.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46608]: starting ceph client 2024-05-17T22:05:45.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46608]: starting fuse 2024-05-17T22:05:46.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:05:46.495 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:05:46.543 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:05:46.543 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:05:46.543 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:05:46.543 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:05:46.543 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:05:46.544 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:05:46.545 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:46.546 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:05:46.546 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:46.546 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:05:46.546 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:46.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:46.547 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:05:46.595 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:05:46.595 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:05:46.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:05:46.595 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:05:46.595 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:05:46.595 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:05:46.596 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:05:46.597 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:46.719 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.46608.asok 2024-05-17T22:05:46.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:46.730 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46608.asok 2024-05-17T22:05:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:46.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46608.asok status 2024-05-17T22:05:46.913 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:05:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "46608", 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7611, 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:05:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 7611 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2268140467 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2268140467 2024-05-17T22:05:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.7611 192.168.0.1:0/2268140467", 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2268140467", 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 81, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 96, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:05:46.916 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:46.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:46.929 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:05:46.929 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:05:46.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:46.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:46.936 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:46.990 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:46.990 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:47.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:47.039 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-17T22:05:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records in 2024-05-17T22:05:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records out 2024-05-17T22:05:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0690882 s, 91.1 MB/s 2024-05-17T22:05:47.167 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:47.167 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:47.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:47.173 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-17T22:05:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records in 2024-05-17T22:05:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:6+0 records out 2024-05-17T22:05:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0634317 s, 99.2 MB/s 2024-05-17T22:05:47.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:47.287 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-17T22:05:47.298 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2024-05-17T22:05:47.298 INFO:teuthology.orchestra.run.smithi083.stdout:./file2_sixmegs 2024-05-17T22:05:47.298 INFO:teuthology.orchestra.run.smithi083.stdout:./file3_sixmegs 2024-05-17T22:05:47.298 INFO:teuthology.orchestra.run.smithi083.stdout:./file1_sixmegs 2024-05-17T22:05:47.299 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:05:47.299 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:47.300 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:05:47.300 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:47.300 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-17T22:05:47.300 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:47.401 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:05:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-17T22:05:47.411 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:47.518 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T22:05:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:47.527 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:05:47.528 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:47.528 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:05:47.528 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:47.528 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-17T22:05:47.528 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:47.633 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2024-05-17T22:05:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-17T22:05:47.643 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:05:47.749 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:05:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:05:47.759 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:47.759 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:47.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:05:48.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 Processor -- start 2024-05-17T22:05:48.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 -- start start 2024-05-17T22:05:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ed00d5210 0x7f7ed00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed00d4170 con 0x7f7ed00b4810 2024-05-17T22:05:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed00d4450 con 0x7f7ed00d2000 2024-05-17T22:05:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed70e6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed00d5620 con 0x7f7ed00d5210 2024-05-17T22:05:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed4e82640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7ed00b4810 0x7f7ed015fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58000/0 (socket says 172.21.15.83:58000) 2024-05-17T22:05:48.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ed4e82640 1 -- 172.21.15.83:0/3228094622 learned_addr learned my addr 172.21.15.83:0/3228094622 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ecffff640 1 --2- 172.21.15.83:0/3228094622 >> v2:172.21.15.155:3300/0 conn(0x7f7ed00d5210 0x7f7ed00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028893216 0 0) 0x7f7ed00d4170 con 0x7f7ed00b4810 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.016+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb8003530 con 0x7f7ed00b4810 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2976825330 0 0) 0x7f7eb8003530 con 0x7f7ed00b4810 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 >> v2:172.21.15.155:3300/0 conn(0x7f7ed00d5210 msgr2=0x7f7ed00d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 --2- 172.21.15.83:0/3228094622 >> v2:172.21.15.155:3300/0 conn(0x7f7ed00d5210 0x7f7ed00d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 >> v1:172.21.15.155:6789/0 conn(0x7f7ed00d2000 legacy=0x7f7ed00d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00d4730 con 0x7f7ed00b4810 2024-05-17T22:05:48.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f7ec0001700 con 0x7f7ed00b4810 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 >> v1:172.21.15.83:6789/0 conn(0x7f7ed00b4810 legacy=0x7f7ed015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8003a60 0x7f7eb8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00d5620 con 0x7f7eb8005260 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb800c1f0 con 0x7f7eb8004810 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb800c3f0 con 0x7f7eb8003a60 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecffff640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed4e82640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8003a60 0x7f7eb8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 -- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8003a60 msgr2=0x7f7eb8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8003a60 0x7f7eb8003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 -- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb8004810 msgr2=0x7f7eb8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 -- 172.21.15.83:0/3228094622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8003530 con 0x7f7eb8005260 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 -- 172.21.15.83:0/3228094622 reap_dead start 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed5683640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00d3880 tx=0x7f7ec801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb8f3c2f235ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8001ae0 con 0x7f7eb8005260 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8003b80 con 0x7f7eb8005260 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ecf7fe640 1 -- 172.21.15.83:0/3228094622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80050d0 con 0x7f7eb8005260 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed70e6640 1 -- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 msgr2=0x7f7eb800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.020+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00d3880 tx=0x7f7ec801c750 comp rx=0 tx=0).stop 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- 172.21.15.83:0/3228094622 shutdown_connections 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/3228094622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- 172.21.15.83:0/3228094622 >> 172.21.15.83:0/3228094622 conn(0x7f7ed006e010 msgr2=0x7f7ed006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- 172.21.15.83:0/3228094622 shutdown_connections 2024-05-17T22:05:48.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- 172.21.15.83:0/3228094622 wait complete. 2024-05-17T22:05:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 Processor -- start 2024-05-17T22:05:48.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- start start 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8005260 0x7f7ed0158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed0153950 0x7f7ed0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00d4cf0 con 0x7f7ed01569c0 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed00d3fd0 con 0x7f7ed0153950 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed70e6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed00d0290 con 0x7f7eb8005260 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43772/0 (socket says 172.21.15.83:43772) 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 -- 172.21.15.83:0/808121247 learned_addr learned my addr 172.21.15.83:0/808121247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ecffff640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed0153950 0x7f7ed0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed4e82640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8005260 0x7f7ed0158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 -- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8005260 msgr2=0x7f7ed0158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8005260 0x7f7ed0158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 -- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed0153950 msgr2=0x7f7ed0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed0153950 0x7f7ed0156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 -- 172.21.15.83:0/808121247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00b3b00 con 0x7f7ed01569c0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.024+0000 7f7ed5683640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7ec801cff0 tx=0x7f7eb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8006d70 con 0x7f7ed01569c0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80050d0 con 0x7f7ed01569c0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed0156dd0 con 0x7f7ed01569c0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80054a0 con 0x7f7ed01569c0 2024-05-17T22:05:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed00b3d70 con 0x7f7ed01569c0 2024-05-17T22:05:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed00d0620 con 0x7f7ed01569c0 2024-05-17T22:05:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ec8005640 con 0x7f7ed01569c0 2024-05-17T22:05:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 0x7f7ebc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec8129550 con 0x7f7ed01569c0 2024-05-17T22:05:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ed4e82640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 0x7f7ebc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.028+0000 7f7ed4e82640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 0x7f7ebc071940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0001060 tx=0x7f7ec0000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.032+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7ed0156dd0 con 0x7f7ed01569c0 2024-05-17T22:05:48.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.208+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7ed00b4d20 con 0x7f7ed01569c0 2024-05-17T22:05:48.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.208+0000 7f7ecd7fa640 1 -- 172.21.15.83:0/808121247 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f7ec80bf060 con 0x7f7ed01569c0 2024-05-17T22:05:48.200 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:05:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 msgr2=0x7f7ebc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 0x7f7ebc071940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0001060 tx=0x7f7ec0000ee0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 msgr2=0x7f7ed01620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7ec801cff0 tx=0x7f7eb8005840 comp rx=0 tx=0).stop 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 shutdown_connections 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed01569c0 0x7f7ed01620b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7ebc06f430 0x7f7ebc071940 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed0153950 0x7f7ed0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 --2- 172.21.15.83:0/808121247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7eb8005260 0x7f7ed0158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 >> 172.21.15.83:0/808121247 conn(0x7f7ed006e010 msgr2=0x7f7ed006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 shutdown_connections 2024-05-17T22:05:48.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.212+0000 7f7ed70e6640 1 -- 172.21.15.83:0/808121247 wait complete. 2024-05-17T22:05:48.222 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:48.222 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:48.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:05:48.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 Processor -- start 2024-05-17T22:05:48.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 -- start start 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fd0143f20 0x7f4fd0072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd00737d0 con 0x7f4fd00733c0 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd00b4110 con 0x7f4fd0143f20 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd76cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd0070770 con 0x7f4fd00b3d00 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fd0143f20 0x7f4fd0072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fd0143f20 0x7f4fd0072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43782/0 (socket says 172.21.15.83:43782) 2024-05-17T22:05:48.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 -- 172.21.15.83:0/1041602562 learned_addr learned my addr 172.21.15.83:0/1041602562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd5468640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 -- 172.21.15.83:0/1041602562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 msgr2=0x7f4fd00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 -- 172.21.15.83:0/1041602562 >> v1:172.21.15.155:6789/0 conn(0x7f4fd00733c0 legacy=0x7f4fd006eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd4c67640 1 -- 172.21.15.83:0/1041602562 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd00d10d0 con 0x7f4fd0143f20 2024-05-17T22:05:48.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.432+0000 7f4fd5468640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd4c67640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd0143f20 0x7f4fd0072e80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009870 tx=0x7f4fc001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4757d25f900afde2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fc7fff640 1 -- 172.21.15.83:0/1041602562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00ab650 con 0x7f4fd0143f20 2024-05-17T22:05:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fc7fff640 1 -- 172.21.15.83:0/1041602562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc00b0070 con 0x7f4fd0143f20 2024-05-17T22:05:48.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fc7fff640 1 -- 172.21.15.83:0/1041602562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00aba10 con 0x7f4fd0143f20 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 -- 172.21.15.83:0/1041602562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd0143f20 msgr2=0x7f4fd0072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd0143f20 0x7f4fd0072e80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009870 tx=0x7f4fc001c560 comp rx=0 tx=0).stop 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 -- 172.21.15.83:0/1041602562 shutdown_connections 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd0143f20 0x7f4fd0072e80 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/1041602562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 -- 172.21.15.83:0/1041602562 >> 172.21.15.83:0/1041602562 conn(0x7f4fd006cd90 msgr2=0x7f4fd00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 -- 172.21.15.83:0/1041602562 shutdown_connections 2024-05-17T22:05:48.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.436+0000 7f4fd76cc640 1 -- 172.21.15.83:0/1041602562 wait complete. 2024-05-17T22:05:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 Processor -- start 2024-05-17T22:05:48.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- start start 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fd0143f20 0x7f4fd013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd0070c00 con 0x7f4fd00733c0 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd0144330 con 0x7f4fd0143f20 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00aeaf0 con 0x7f4fd00b3d00 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43792/0 (socket says 172.21.15.83:43792) 2024-05-17T22:05:48.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 -- 172.21.15.83:0/3513955165 learned_addr learned my addr 172.21.15.83:0/3513955165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd4c67640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5c69640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fd0143f20 0x7f4fd013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 -- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 msgr2=0x7f4fd00ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 -- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fd0143f20 msgr2=0x7f4fd013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fd0143f20 0x7f4fd013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 -- 172.21.15.83:0/3513955165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd01f6fa0 con 0x7f4fd00733c0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd5468640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c820 tx=0x7f4fcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc015070 con 0x7f4fd00733c0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fcc007a30 con 0x7f4fd00733c0 2024-05-17T22:05:48.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc007e40 con 0x7f4fd00733c0 2024-05-17T22:05:48.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd00add70 con 0x7f4fd00733c0 2024-05-17T22:05:48.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.440+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd00af020 con 0x7f4fd00733c0 2024-05-17T22:05:48.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd00d0a10 con 0x7f4fd00733c0 2024-05-17T22:05:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4fcc018070 con 0x7f4fd00733c0 2024-05-17T22:05:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fc5ffb640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 0x7f4fb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f4fcc08f1c0 con 0x7f4fd00733c0 2024-05-17T22:05:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fd4c67640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 0x7f4fb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.444+0000 7f4fd4c67640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 0x7f4fb4071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00af000 tx=0x7f4fc00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.448+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4fd00add70 con 0x7f4fd00733c0 2024-05-17T22:05:48.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4fd0141150 con 0x7f4fd00733c0 2024-05-17T22:05:48.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fc5ffb640 1 -- 172.21.15.83:0/3513955165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) ==== 80+0+0 (secure 0 0 0) 0x7f4fcc05c140 con 0x7f4fd00733c0 2024-05-17T22:05:48.611 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 msgr2=0x7f4fb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 0x7f4fb4071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00af000 tx=0x7f4fc00a7000 comp rx=0 tx=0).stop 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 msgr2=0x7f4fd015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c820 tx=0x7f4fcc00cce0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 shutdown_connections 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4fb406f400 0x7f4fb4071910 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4fd0143f20 0x7f4fd013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4fd00b3d00 0x7f4fd00ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 --2- 172.21.15.83:0/3513955165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd00733c0 0x7f4fd015d510 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.620+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 >> 172.21.15.83:0/3513955165 conn(0x7f4fd006cd90 msgr2=0x7f4fd00d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.624+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 shutdown_connections 2024-05-17T22:05:48.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.624+0000 7f4fd76cc640 1 -- 172.21.15.83:0/3513955165 wait complete. 2024-05-17T22:05:48.631 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:05:48.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:48.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:48.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[46608]: fuse finished with error 0 and tester_r 0 2024-05-17T22:05:48.645 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:05:48.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:05:48.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:48.732 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:05:48.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:05:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 Processor -- start 2024-05-17T22:05:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 -- start start 2024-05-17T22:05:48.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f111c1461c0 0x7f111c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f111c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111c1465d0 con 0x7f111c070510 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f111c06ef30 con 0x7f111c1461c0 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1122704640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f111c06f210 con 0x7f111c15e6b0 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1120ca1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f111c070510 0x7f111c1605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58004/0 (socket says 172.21.15.83:58004) 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1120ca1640 1 -- 172.21.15.83:0/2205743696 learned_addr learned my addr 172.21.15.83:0/2205743696 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111bfff640 1 --2- 172.21.15.83:0/2205743696 >> v2:172.21.15.155:3300/0 conn(0x7f111c1461c0 0x7f111c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971216569 0 0) 0x7f111c1465d0 con 0x7f111c070510 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f111c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1108003530 con 0x7f111c070510 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 854062910 0 0) 0x7f1108003530 con 0x7f111c070510 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 msgr2=0x7f111c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f111c06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 >> v2:172.21.15.155:3300/0 conn(0x7f111c1461c0 msgr2=0x7f111c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 --2- 172.21.15.83:0/2205743696 >> v2:172.21.15.155:3300/0 conn(0x7f111c1461c0 0x7f111c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c070920 con 0x7f111c070510 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1114001a80 con 0x7f111c070510 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 >> v1:172.21.15.83:6789/0 conn(0x7f111c070510 legacy=0x7f111c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1108003a10 0x7f1108003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1108004810 0x7f1108004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f1108005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06ef30 con 0x7f1108004810 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06f210 con 0x7f1108003a10 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f110800bab0 con 0x7f111c15e6b0 2024-05-17T22:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1108004810 0x7f1108004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f111bfff640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1108003a10 0x7f1108003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1120ca1640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f1108005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 -- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 msgr2=0x7f1108005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f1108005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 -- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1108003a10 msgr2=0x7f1108003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1108003a10 0x7f1108003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f11137fe640 1 -- 172.21.15.83:0/2205743696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1108003530 con 0x7f1108004810 2024-05-17T22:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.920+0000 7f1120ca1640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c15e6b0 0x7f1108005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f11137fe640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1108004810 0x7f1108004c20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f110c009ae0 tx=0x7f110c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bfa951dd0e4cda server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0b2070 con 0x7f1108004810 2024-05-17T22:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c0a5680 con 0x7f1108004810 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f111b7fe640 1 -- 172.21.15.83:0/2205743696 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0096b0 con 0x7f1108004810 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 -- 172.21.15.83:0/2205743696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1108004810 msgr2=0x7f1108004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 --2- 172.21.15.83:0/2205743696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1108004810 0x7f1108004c20 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f110c009ae0 tx=0x7f110c01ca90 comp rx=0 tx=0).stop 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1120ca1640 1 -- 172.21.15.83:0/2205743696 reap_dead start 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 -- 172.21.15.83:0/2205743696 shutdown_connections 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 -- 172.21.15.83:0/2205743696 >> 172.21.15.83:0/2205743696 conn(0x7f111c06cd90 msgr2=0x7f111c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 -- 172.21.15.83:0/2205743696 shutdown_connections 2024-05-17T22:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.924+0000 7f1122704640 1 -- 172.21.15.83:0/2205743696 wait complete. 2024-05-17T22:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 Processor -- start 2024-05-17T22:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- start start 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c1e7ce0 0x7f111c1e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c14d8c0 0x7f111c13eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06fab0 con 0x7f111c072e30 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f111c06f7d0 con 0x7f111c1e7ce0 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f111c160cb0 con 0x7f111c14d8c0 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43818/0 (socket says 172.21.15.83:43818) 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 -- 172.21.15.83:0/1683809467 learned_addr learned my addr 172.21.15.83:0/1683809467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f111bfff640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c1e7ce0 0x7f111c1e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f11137fe640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c14d8c0 0x7f111c13eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 -- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c14d8c0 msgr2=0x7f111c13eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c14d8c0 0x7f111c13eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 -- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c1e7ce0 msgr2=0x7f111c1e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c1e7ce0 0x7f111c1e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 -- 172.21.15.83:0/1683809467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c073780 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1120ca1640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1114004bd0 tx=0x7f1114010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111401a070 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1114011040 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1e8210 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400ece0 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1e6860 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.928+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c15e170 con 0x7f111c072e30 2024-05-17T22:05:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.932+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f111401d070 con 0x7f111c072e30 2024-05-17T22:05:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.932+0000 7f11197fa640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.932+0000 7f111bfff640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.932+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f1114091980 con 0x7f111c072e30 2024-05-17T22:05:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.932+0000 7f111bfff640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 0x7f10fc071910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f111c0b11f0 tx=0x7f1104008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:48.936+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f111c073780 con 0x7f111c072e30 2024-05-17T22:05:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.116+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f111c0b41a0 con 0x7f111c072e30 2024-05-17T22:05:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.116+0000 7f11197fa640 1 -- 172.21.15.83:0/1683809467 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5189 (secure 0 0 0) 0x7f111405ea50 con 0x7f111c072e30 2024-05-17T22:05:49.109 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:49.109 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:05:29.852155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7407":{"gid":7407,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1996242614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1996242614},{"type":"v1","addr":"172.21.15.83:6836","nonce":1996242614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7407,"qdb_cluster":[7407]},"id":7}]} 2024-05-17T22:05:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 81 2024-05-17T22:05:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 msgr2=0x7f10fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 0x7f10fc071910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f111c0b11f0 tx=0x7f1104008040 comp rx=0 tx=0).stop 2024-05-17T22:05:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 msgr2=0x7f111c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1114004bd0 tx=0x7f1114010e60 comp rx=0 tx=0).stop 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 shutdown_connections 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f111c072e30 0x7f111c073240 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f111c14d8c0 0x7f111c13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 --2- 172.21.15.83:0/1683809467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f111c1e7ce0 0x7f111c1e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 >> 172.21.15.83:0/1683809467 conn(0x7f111c06cd90 msgr2=0x7f111c0b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 shutdown_connections 2024-05-17T22:05:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.120+0000 7f1122704640 1 -- 172.21.15.83:0/1683809467 wait complete. 2024-05-17T22:05:49.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-17T22:05:49.306 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0} 2024-05-17T22:05:49.306 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-05-17T22:05:49.306 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:05:49.307 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:05:49.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:05:49.312 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi083.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-17T22:05:49.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:05:49.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:49.312 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:05:49.358 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-17T22:05:49.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:05:49.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-17T22:05:49.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:49.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-17T22:05:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 Processor -- start 2024-05-17T22:05:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 -- start start 2024-05-17T22:05:49.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde400b77c0 0x7fde40160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde400b5cf0 0x7fde4006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde400b1790 con 0x7fde4015d940 2024-05-17T22:05:49.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fde4006d720 con 0x7fde400b77c0 2024-05-17T22:05:49.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde475d9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde4006c790 con 0x7fde400b5cf0 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde400b77c0 0x7fde40160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde400b77c0 0x7fde40160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43830/0 (socket says 172.21.15.83:43830) 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 -- 172.21.15.83:0/3533615637 learned_addr learned my addr 172.21.15.83:0/3533615637 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 -- 172.21.15.83:0/3533615637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde400b5cf0 msgr2=0x7fde4006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45b76640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde400b5cf0 0x7fde4006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde400b5cf0 0x7fde4006d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 -- 172.21.15.83:0/3533615637 >> v1:172.21.15.155:6789/0 conn(0x7fde4015d940 legacy=0x7fde400b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.572+0000 7fde45375640 1 -- 172.21.15.83:0/3533615637 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde400b1a70 con 0x7fde400b77c0 2024-05-17T22:05:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde45375640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40160cb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fde40073430 tx=0x7fde3001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6da56c87b100 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde37fff640 1 -- 172.21.15.83:0/3533615637 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300b0070 con 0x7fde400b77c0 2024-05-17T22:05:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde37fff640 1 -- 172.21.15.83:0/3533615637 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde300ab600 con 0x7fde400b77c0 2024-05-17T22:05:49.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde37fff640 1 -- 172.21.15.83:0/3533615637 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300ab9d0 con 0x7fde400b77c0 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 -- 172.21.15.83:0/3533615637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 msgr2=0x7fde40160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40160cb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fde40073430 tx=0x7fde3001c4c0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 -- 172.21.15.83:0/3533615637 shutdown_connections 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde400b5cf0 0x7fde4006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 --2- 172.21.15.83:0/3533615637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40160cb0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 -- 172.21.15.83:0/3533615637 >> 172.21.15.83:0/3533615637 conn(0x7fde4009b8b0 msgr2=0x7fde400b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 -- 172.21.15.83:0/3533615637 shutdown_connections 2024-05-17T22:05:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.576+0000 7fde475d9640 1 -- 172.21.15.83:0/3533615637 wait complete. 2024-05-17T22:05:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 Processor -- start 2024-05-17T22:05:49.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 -- start start 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde4015d940 0x7fde40150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 0x7fde40156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde400aff50 con 0x7fde400b77c0 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde400afdd0 con 0x7fde4014c3a0 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde475d9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde400b00d0 con 0x7fde4015d940 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43836/0 (socket says 172.21.15.83:43836) 2024-05-17T22:05:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 -- 172.21.15.83:0/2167721566 learned_addr learned my addr 172.21.15.83:0/2167721566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45b76640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 0x7fde40156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7f8e4c57f640 1 Processor -- start 2024-05-17T22:05:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde44b74640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde4015d940 0x7fde40150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7f8e4c57f640 1 -- start start 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 -- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde4015d940 msgr2=0x7fde40150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde4015d940 0x7fde40150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 -- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 msgr2=0x7fde40156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 0x7fde40156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45375640 1 -- 172.21.15.83:0/2167721566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4014d5f0 con 0x7fde400b77c0 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.580+0000 7fde45b76640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 0x7fde40156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde45375640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fde3001cd60 tx=0x7fde3001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 -- 172.21.15.83:0/2167721566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300b0040 con 0x7fde400b77c0 2024-05-17T22:05:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 -- 172.21.15.83:0/2167721566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde300bc070 con 0x7fde400b77c0 2024-05-17T22:05:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4014f370 con 0x7fde400b77c0 2024-05-17T22:05:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 -- 172.21.15.83:0/2167721566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300b6840 con 0x7fde400b77c0 2024-05-17T22:05:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4014f930 con 0x7fde400b77c0 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4c57f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4c57f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e440d1d80 0x7f8e44071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4c57f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e44161c60 con 0x7f8e44071b70 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4c57f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4415df10 con 0x7f8e440d1d80 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4c57f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e440cec90 con 0x7f8e440ce880 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4a31b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4a31b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:60716/0 (socket says 172.21.15.83:60716) 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e49b1a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e440d1d80 0x7f8e44071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4a31b640 1 -- 172.21.15.83:0/2875758258 learned_addr learned my addr 172.21.15.83:0/2875758258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e49b1a640 1 -- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 msgr2=0x7f8e44161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e49b1a640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e49b1a640 1 -- 172.21.15.83:0/2875758258 >> v1:172.21.15.155:6789/0 conn(0x7f8e44071b70 legacy=0x7f8e4415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e49b1a640 1 -- 172.21.15.83:0/2875758258 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e44161f40 con 0x7f8e440d1d80 2024-05-17T22:05:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 -- 172.21.15.83:0/2167721566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde300b69e0 con 0x7fde400b77c0 2024-05-17T22:05:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7f8e4a31b640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 0x7fde24071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde35ffb640 1 -- 172.21.15.83:0/2167721566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fde301295a0 con 0x7fde400b77c0 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde44b74640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 0x7fde24071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.584+0000 7fde44b74640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 0x7fde24071940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fde401fc530 tx=0x7fde3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 Processor -- start 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- start start 2024-05-17T22:05:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e49b1a640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440d1d80 0x7f8e44071630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e3801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5fdebb76b41b59 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55737268f240 0x5573727d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573725e4220 con 0x557372680f60 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5573725d3dc0 con 0x55737266adf0 2024-05-17T22:05:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5573725d7e00 con 0x55737268f240 2024-05-17T22:05:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e49319640 1 -- 172.21.15.83:0/2875758258 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380b0070 con 0x7f8e440d1d80 2024-05-17T22:05:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e49319640 1 -- 172.21.15.83:0/2875758258 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e380ab600 con 0x7f8e440d1d80 2024-05-17T22:05:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e49319640 1 -- 172.21.15.83:0/2875758258 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e380ab9d0 con 0x7f8e440d1d80 2024-05-17T22:05:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43838/0 (socket says 172.21.15.83:43838) 2024-05-17T22:05:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 -- 172.21.15.83:0/822238492 learned_addr learned my addr 172.21.15.83:0/822238492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde45b76640 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 -- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55737268f240 msgr2=0x5573727d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55737268f240 0x5573727d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 -- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 msgr2=0x5573727d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 -- 172.21.15.83:0/822238492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573725afbf0 con 0x557372680f60 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde44b74640 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d7e0 tx=0x7fde3c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1aff7fa48ed922 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde2affd640 1 -- 172.21.15.83:0/822238492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c024070 con 0x557372680f60 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde2affd640 1 -- 172.21.15.83:0/822238492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c0149b0 con 0x557372680f60 2024-05-17T22:05:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde2affd640 1 -- 172.21.15.83:0/822238492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c014d60 con 0x557372680f60 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 -- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440d1d80 msgr2=0x7f8e44071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440d1d80 0x7f8e44071630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e3801c440 comp rx=0 tx=0).stop 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 -- 172.21.15.83:0/2875758258 shutdown_connections 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440d1d80 0x7f8e44071630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 --2- 172.21.15.83:0/2875758258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440ce880 0x7f8e44161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 -- 172.21.15.83:0/2875758258 >> 172.21.15.83:0/2875758258 conn(0x7f8e440b45e0 msgr2=0x7f8e440b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 -- 172.21.15.83:0/2875758258 shutdown_connections 2024-05-17T22:05:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7f8e4c57f640 1 -- 172.21.15.83:0/2875758258 wait complete. 2024-05-17T22:05:49.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- 172.21.15.83:0/822238492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 msgr2=0x5573727d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d7e0 tx=0x7fde3c010000 comp rx=0 tx=0).stop 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- 172.21.15.83:0/822238492 shutdown_connections 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557372680f60 0x5573727d2e90 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 --2- 172.21.15.83:0/822238492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55737268f240 0x5573727d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- 172.21.15.83:0/822238492 >> 172.21.15.83:0/822238492 conn(0x557372295810 msgr2=0x5573727870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- 172.21.15.83:0/822238492 shutdown_connections 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.588+0000 7fde4e853000 1 -- 172.21.15.83:0/822238492 wait complete. 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- start start 2024-05-17T22:05:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 Processor -- start 2024-05-17T22:05:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 Processor -- start 2024-05-17T22:05:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- start start 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573727d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55737255cf30 con 0x55737268f240 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5573725d0740 con 0x55737266adf0 2024-05-17T22:05:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55737255acc0 con 0x557372680f60 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e44071b70 0x7f8e44151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440ce880 0x7f8e44156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440d1d80 0x7f8e44156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4415eee0 con 0x7f8e44071b70 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4415ed60 con 0x7f8e440ce880 2024-05-17T22:05:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8e44157780 con 0x7f8e440d1d80 2024-05-17T22:05:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 10 client.0 _ll_register_callbacks cb 0x5573727746a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e44071b70 0x7f8e44151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e44071b70 0x7f8e44151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43860/0 (socket says 172.21.15.83:43860) 2024-05-17T22:05:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 -- 172.21.15.83:0/3248376065 learned_addr learned my addr 172.21.15.83:0/3248376065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43852/0 (socket says 172.21.15.83:43852) 2024-05-17T22:05:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 -- 172.21.15.83:0/3964145436 learned_addr learned my addr 172.21.15.83:0/3964145436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45375640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde44b74640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573727d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4ab1c640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440d1d80 0x7f8e44156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 -- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440d1d80 msgr2=0x7f8e44156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440d1d80 0x7f8e44156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 -- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440ce880 msgr2=0x7f8e44156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e440ce880 0x7f8e44156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 -- 172.21.15.83:0/3248376065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e441520f0 con 0x7f8e44071b70 2024-05-17T22:05:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4ab1c640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e440d1d80 0x7f8e44156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 -- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 msgr2=0x5573727d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573727d9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 -- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 msgr2=0x5573727d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 -- 172.21.15.83:0/3964145436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737258d460 con 0x55737268f240 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde44b74640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573727d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45375640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:05:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4a31b640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e44071b70 0x7f8e44151bb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e930 tx=0x7f8e4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde45b76640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fde38010b00 tx=0x7fde3800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36efe8bb2571f45a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40017070 con 0x7f8e44071b70 2024-05-17T22:05:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4000ca00 con 0x7f8e44071b70 2024-05-17T22:05:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- 172.21.15.83:0/3248376065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e440cba70 con 0x7f8e44071b70 2024-05-17T22:05:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cdd0 con 0x7f8e44071b70 2024-05-17T22:05:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- 172.21.15.83:0/3248376065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e440cc030 con 0x7f8e44071b70 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3800ed30 con 0x55737268f240 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fde38012450 con 0x55737268f240 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3c0149b0 con 0x55737268f240 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fde38012690 con 0x55737268f240 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 10 client.7644 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x5573725929e0 con 0x55737268f240 2024-05-17T22:05:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.592+0000 7f8e4c57f640 1 -- 172.21.15.83:0/3248376065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8e44071630 con 0x7f8e44071b70 2024-05-17T22:05:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c0149b0 con 0x55737268f240 2024-05-17T22:05:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7fde3800f040 con 0x55737268f240 2024-05-17T22:05:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 10 client.7644 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 10 client.7644 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 20 client.7644 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55737227ab80 0x5573727e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 4 client.7644 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55737256fd40 con 0x55737227ab80 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde45375640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55737227ab80 0x5573727e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 10 client.7644 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8e4001a070 con 0x7f8e44071b70 2024-05-17T22:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde45375640 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55737227ab80 0x5573727e31b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469b8f9983745de5 server_cookie=6145dafc5d41ecdd in_seq=0 out_seq=0 2024-05-17T22:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e36ffd640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8e2806f400 0x7f8e28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e49b1a640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8e2806f400 0x7f8e28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f8e4008f1a0 con 0x7f8e44071b70 2024-05-17T22:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 1 -- 172.21.15.83:0/3964145436 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55737256fd40 con 0x55737227ab80 2024-05-17T22:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 10 client.7644 handle_command_reply: tid=0 2024-05-17T22:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 client.7644 shutdown 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f8e40091820 con 0x7f8e44071b70 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 msgr2=0x5573727d9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fde38010b00 tx=0x7fde3800e2f0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 shutdown_connections 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55737227ab80 0x5573727e31b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x5573727d9850 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e4c57f640 1 -- 172.21.15.83:0/3248376065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f8e44161540 con 0x7f8e44071b70 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573727d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 --2- 172.21.15.83:0/3964145436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x5573727d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde1ffff640 0 client.7644 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 >> 172.21.15.83:0/3964145436 conn(0x557372295810 msgr2=0x5573727870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 shutdown_connections 2024-05-17T22:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 1 -- 172.21.15.83:0/3964145436 wait complete. 2024-05-17T22:05:49.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7f8e49b1a640 1 --2- 172.21.15.83:0/3248376065 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8e2806f400 0x7f8e28071910 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8e380ad000 tx=0x7f8e38004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.596+0000 7fde4e853000 20 client.7644 trim_cache size 0 max 16384 2024-05-17T22:05:49.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.612+0000 7fde4e853000 1 Processor -- start 2024-05-17T22:05:49.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.612+0000 7fde4e853000 1 -- start start 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x557372840de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 0x557372841320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde401fd270 con 0x55737266adf0 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde401fc5d0 con 0x55737268f240 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde401fcc20 con 0x557372680f60 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43866/0 (socket says 172.21.15.83:43866) 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 -- 172.21.15.83:0/2855050746 learned_addr learned my addr 172.21.15.83:0/2855050746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45b76640 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 0x557372841320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 -- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 msgr2=0x557372840de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x557372840de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 -- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 msgr2=0x557372841320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 0x557372841320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 -- 172.21.15.83:0/2855050746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557372658a10 con 0x55737266adf0 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45b76640 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 0x557372841320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde45375640 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fde300f5b20 tx=0x7fde30009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696d5014699557d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde2a7fc640 1 -- 172.21.15.83:0/2855050746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300fa1d0 con 0x55737266adf0 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde2a7fc640 1 -- 172.21.15.83:0/2855050746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde300febf0 con 0x55737266adf0 2024-05-17T22:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde2a7fc640 1 -- 172.21.15.83:0/2855050746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300ab520 con 0x55737266adf0 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- 172.21.15.83:0/2855050746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 msgr2=0x5573728408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fde300f5b20 tx=0x7fde30009240 comp rx=0 tx=0).stop 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- 172.21.15.83:0/2855050746 shutdown_connections 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737268f240 0x557372841320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x557372840de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 --2- 172.21.15.83:0/2855050746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737266adf0 0x5573728408a0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- 172.21.15.83:0/2855050746 >> 172.21.15.83:0/2855050746 conn(0x557372295810 msgr2=0x55737278d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- 172.21.15.83:0/2855050746 shutdown_connections 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- 172.21.15.83:0/2855050746 wait complete. 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 -- start start 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.616+0000 7fde4e853000 1 Processor -- start 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x557372844e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573728453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573725a2dd0 con 0x55737268f240 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5573725e2ff0 con 0x55737266adf0 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5573725d7110 con 0x557372680f60 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 10 client.0 _ll_register_callbacks cb 0x7fde38007790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43868/0 (socket says 172.21.15.83:43868) 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 -- 172.21.15.83:0/1127993111 learned_addr learned my addr 172.21.15.83:0/1127993111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45375640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x557372844e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 -- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 msgr2=0x5573728453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573728453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 -- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 msgr2=0x557372844e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x557372844e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 -- 172.21.15.83:0/1127993111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557372590dc0 con 0x55737268f240 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45b76640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f90 tx=0x7fde38004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9610c44df54f0e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38012ac0 con 0x55737268f240 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fde38002040 con 0x55737268f240 2024-05-17T22:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde300febf0 con 0x55737268f240 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fde38038050 con 0x55737268f240 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 10 client.7650 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55737257b3e0 con 0x55737268f240 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde300febf0 con 0x55737268f240 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7fde3803b020 con 0x55737268f240 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 10 client.7650 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 10 client.7650 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 20 client.7650 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x557372268be0 0x557372800790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 4 client.7650 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5573725e6170 con 0x557372268be0 2024-05-17T22:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45375640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x557372268be0 0x557372800790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 10 client.7650 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde45375640 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x557372268be0 0x557372800790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6552232051ee4cb server_cookie=e8c7192cab72cd45 in_seq=0 out_seq=0 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 1 -- 172.21.15.83:0/1127993111 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573725e6170 con 0x557372268be0 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde1ffff640 10 client.7650 handle_command_reply: tid=0 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 client.7650 shutdown 2024-05-17T22:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 msgr2=0x557372845910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f90 tx=0x7fde38004a50 comp rx=0 tx=0).stop 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 shutdown_connections 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x557372268be0 0x557372800790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55737268f240 0x557372845910 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557372680f60 0x5573728453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 --2- 172.21.15.83:0/1127993111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55737266adf0 0x557372844e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.620+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 >> 172.21.15.83:0/1127993111 conn(0x557372295810 msgr2=0x5573727df580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde1ffff640 0 client.7650 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 shutdown_connections 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde4e853000 1 -- 172.21.15.83:0/1127993111 wait complete. 2024-05-17T22:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde4e853000 20 client.7650 trim_cache size 0 max 16384 2024-05-17T22:05:49.620 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:05:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e6439c7a-fab3-4063-a2d8-d6bbc259f4b9", 2024-05-17T22:05:49.620 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:05:49.620 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 msgr2=0x7fde24071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 0x7fde24071940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fde401fc530 tx=0x7fde3c008040 comp rx=0 tx=0).stop 2024-05-17T22:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 msgr2=0x7fde40154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fde3001cd60 tx=0x7fde3001ad60 comp rx=0 tx=0).stop 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 shutdown_connections 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f430 0x7fde24071940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4014c3a0 0x7fde40156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde4015d940 0x7fde40150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 --2- 172.21.15.83:0/2167721566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde400b77c0 0x7fde40154a80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 >> 172.21.15.83:0/2167721566 conn(0x7fde4009b8b0 msgr2=0x7fde400b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 shutdown_connections 2024-05-17T22:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.624+0000 7fde475d9640 1 -- 172.21.15.83:0/2167721566 wait complete. 2024-05-17T22:05:49.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:05:49.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-17T22:05:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 Processor -- start 2024-05-17T22:05:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 -- start start 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffba00d2140 0x7ffba0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffba00d3ae0 0x7ffba00d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ffba006f630 con 0x7ffba00d3ae0 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffba00d3ef0 con 0x7ffba00d2140 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba6dd5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00d2670 con 0x7ffba015d6f0 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffba00d3ae0 0x7ffba00d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffba00d3ae0 0x7ffba00d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43870/0 (socket says 172.21.15.83:43870) 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 -- 172.21.15.83:0/2576422824 learned_addr learned my addr 172.21.15.83:0/2576422824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba4b71640 1 --2- 172.21.15.83:0/2576422824 >> v2:172.21.15.155:3300/0 conn(0x7ffba00d2140 0x7ffba0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 -- 172.21.15.83:0/2576422824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 msgr2=0x7ffba00d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 -- 172.21.15.83:0/2576422824 >> v2:172.21.15.155:3300/0 conn(0x7ffba00d2140 msgr2=0x7ffba0160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 --2- 172.21.15.83:0/2576422824 >> v2:172.21.15.155:3300/0 conn(0x7ffba00d2140 0x7ffba0160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.788+0000 7ffba5372640 1 -- 172.21.15.83:0/2576422824 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00d2950 con 0x7ffba00d3ae0 2024-05-17T22:05:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba5372640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba00d0320 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ffba00d1130 tx=0x7ffb9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cb6033c905db12 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffb9f7fe640 1 -- 172.21.15.83:0/2576422824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900b1070 con 0x7ffba00d3ae0 2024-05-17T22:05:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffb9f7fe640 1 -- 172.21.15.83:0/2576422824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb900ac630 con 0x7ffba00d3ae0 2024-05-17T22:05:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffb9f7fe640 1 -- 172.21.15.83:0/2576422824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900a8050 con 0x7ffba00d3ae0 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 -- 172.21.15.83:0/2576422824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 msgr2=0x7ffba00d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba00d0320 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ffba00d1130 tx=0x7ffb9001c520 comp rx=0 tx=0).stop 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 -- 172.21.15.83:0/2576422824 shutdown_connections 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba00d0320 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/2576422824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00d35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/2576422824 >> v2:172.21.15.155:3300/0 conn(0x7ffba00d2140 0x7ffba0160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 -- 172.21.15.83:0/2576422824 >> 172.21.15.83:0/2576422824 conn(0x7ffba0071180 msgr2=0x7ffba0071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 -- 172.21.15.83:0/2576422824 shutdown_connections 2024-05-17T22:05:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.792+0000 7ffba6dd5640 1 -- 172.21.15.83:0/2576422824 wait complete. 2024-05-17T22:05:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 Processor -- start 2024-05-17T22:05:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- start start 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba00d2140 0x7ffba00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffba015dc80 con 0x7ffba00d3ae0 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffba015db00 con 0x7ffba00d2140 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffba01638b0 con 0x7ffba015d6f0 2024-05-17T22:05:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43880/0 (socket says 172.21.15.83:43880) 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 -- 172.21.15.83:0/1909967999 learned_addr learned my addr 172.21.15.83:0/1909967999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba4b71640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba00d2140 0x7ffba00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba5372640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 -- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 msgr2=0x7ffba00b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 -- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba00d2140 msgr2=0x7ffba00b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba00d2140 0x7ffba00b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 -- 172.21.15.83:0/1909967999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00b6830 con 0x7ffba00d3ae0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007e90 tx=0x7ffb9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9d7fa640 1 -- 172.21.15.83:0/1909967999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94014620 con 0x7ffba00d3ae0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9d7fa640 1 -- 172.21.15.83:0/1909967999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb94019070 con 0x7ffba00d3ae0 2024-05-17T22:05:49.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba00b6b10 con 0x7ffba00d3ae0 2024-05-17T22:05:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffb9d7fa640 1 -- 172.21.15.83:0/1909967999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94014a20 con 0x7ffba00d3ae0 2024-05-17T22:05:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.796+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba00b70d0 con 0x7ffba00d3ae0 2024-05-17T22:05:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 Processor -- start 2024-05-17T22:05:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 -- start start 2024-05-17T22:05:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffb9d7fa640 1 -- 172.21.15.83:0/1909967999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffb94014bc0 con 0x7ffba00d3ae0 2024-05-17T22:05:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffb9d7fa640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 0x7ffb8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba5372640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 0x7ffb8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffb9d7fa640 1 -- 172.21.15.83:0/1909967999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9408fc10 con 0x7ffba00d3ae0 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba5372640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 0x7ffb8c071910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffb900ae000 tx=0x7ffb900a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae74a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae74a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae5e8a90 con 0x55d1ae5e3f80 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae55c120 con 0x55d1ae5d0880 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d1ae54bcc0 con 0x55d1ae5d1090 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43890/0 (socket says 172.21.15.83:43890) 2024-05-17T22:05:49.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 -- 172.21.15.83:0/3827426426 learned_addr learned my addr 172.21.15.83:0/3827426426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 -- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 msgr2=0x55d1ae74a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae74a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 -- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 msgr2=0x55d1ae74a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae74a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae74a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.800+0000 7ffba4b71640 1 -- 172.21.15.83:0/3827426426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ae508110 con 0x55d1ae5e3f80 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009ae0 tx=0x7ffb98009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81104e5ea6d352bf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffb867fc640 1 -- 172.21.15.83:0/3827426426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98019070 con 0x55d1ae5e3f80 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffb867fc640 1 -- 172.21.15.83:0/3827426426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98011bc0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffb867fc640 1 -- 172.21.15.83:0/3827426426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98015460 con 0x55d1ae5e3f80 2024-05-17T22:05:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- 172.21.15.83:0/3827426426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 msgr2=0x55d1ae749e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009ae0 tx=0x7ffb98009190 comp rx=0 tx=0).stop 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- 172.21.15.83:0/3827426426 shutdown_connections 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae74a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae74a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3827426426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae749e20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- 172.21.15.83:0/3827426426 >> 172.21.15.83:0/3827426426 conn(0x55d1ae61f010 msgr2=0x55d1ae6ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- 172.21.15.83:0/3827426426 shutdown_connections 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- 172.21.15.83:0/3827426426 wait complete. 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- start start 2024-05-17T22:05:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 Processor -- start 2024-05-17T22:05:49.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae706a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae706fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae507ec0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae54fd00 con 0x55d1ae5d0880 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d1ae215680 con 0x55d1ae5d1090 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffbae04f000 10 client.0 _ll_register_callbacks cb 0x55d1ae6031c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffba5372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffba5372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43900/0 (socket says 172.21.15.83:43900) 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffba5372640 1 -- 172.21.15.83:0/3282855105 learned_addr learned my addr 172.21.15.83:0/3282855105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae706a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.804+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae706fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 -- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 msgr2=0x55d1ae706fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae706fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 -- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 msgr2=0x55d1ae706a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae706a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 -- 172.21.15.83:0/3282855105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ae50a890 con 0x55d1ae5e3f80 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae706fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba5372640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffb900af040 tx=0x7ffb900192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a03d02b38a33b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900c6060 con 0x55d1ae5e3f80 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7ffb900b7650 con 0x55d1ae5e3f80 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb98011bc0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffb900cc050 con 0x55d1ae5e3f80 2024-05-17T22:05:49.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 10 client.7662 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55d1ae50c650 con 0x55d1ae5e3f80 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98011bc0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7ffb900cf020 con 0x55d1ae5e3f80 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 10 client.7662 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 10 client.7662 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 20 client.7662 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1f2930 0x55d1ae750e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 4 client.7662 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ae4e7c40 con 0x55d1ae1f2930 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1f2930 0x55d1ae750e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 10 client.7662 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1f2930 0x55d1ae750e90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50fb76e816bec11 server_cookie=48534d42889ead82 in_seq=0 out_seq=0 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3282855105 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1ae4e7c40 con 0x55d1ae1f2930 2024-05-17T22:05:49.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 10 client.7662 handle_command_reply: tid=0 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 client.7662 shutdown 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 msgr2=0x55d1ae70bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffb900af040 tx=0x7ffb900192c0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 shutdown_connections 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1f2930 0x55d1ae750e90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae70bbe0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d1090 0x55d1ae706fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffb7f7fe640 0 client.7662 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3282855105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d0880 0x55d1ae706a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 >> 172.21.15.83:0/3282855105 conn(0x55d1ae61f010 msgr2=0x55d1ae6ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 shutdown_connections 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 1 -- 172.21.15.83:0/3282855105 wait complete. 2024-05-17T22:05:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.808+0000 7ffbae04f000 20 client.7662 trim_cache size 0 max 16384 2024-05-17T22:05:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 Processor -- start 2024-05-17T22:05:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- start start 2024-05-17T22:05:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d0880 0x55d1ae702260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae760410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffba02054a0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0204660 con 0x55d1ae5d1090 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffba0161680 con 0x55d1ae5d0880 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43912/0 (socket says 172.21.15.83:43912) 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 -- 172.21.15.83:0/667251844 learned_addr learned my addr 172.21.15.83:0/667251844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae760410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba4b71640 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d0880 0x55d1ae702260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 -- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d0880 msgr2=0x55d1ae702260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d0880 0x55d1ae702260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 -- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 msgr2=0x55d1ae760410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae760410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 -- 172.21.15.83:0/667251844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ae4eb8c0 con 0x55d1ae5e3f80 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffba5372640 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb9001c350 tx=0x7ffb900be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece1ae34f5653f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffb85ffb640 1 -- 172.21.15.83:0/667251844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900d5070 con 0x55d1ae5e3f80 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffb85ffb640 1 -- 172.21.15.83:0/667251844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb900b7450 con 0x55d1ae5e3f80 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffb85ffb640 1 -- 172.21.15.83:0/667251844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900b1040 con 0x55d1ae5e3f80 2024-05-17T22:05:49.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- 172.21.15.83:0/667251844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 msgr2=0x55d1ae760950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffb9001c350 tx=0x7ffb900be000 comp rx=0 tx=0).stop 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- 172.21.15.83:0/667251844 shutdown_connections 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5e3f80 0x55d1ae760950 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae760410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 --2- 172.21.15.83:0/667251844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5d0880 0x55d1ae702260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- 172.21.15.83:0/667251844 >> 172.21.15.83:0/667251844 conn(0x55d1ae61f010 msgr2=0x55d1ae7494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- 172.21.15.83:0/667251844 shutdown_connections 2024-05-17T22:05:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.828+0000 7ffbae04f000 1 -- 172.21.15.83:0/667251844 wait complete. 2024-05-17T22:05:49.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- start start 2024-05-17T22:05:49.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 Processor -- start 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae7bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5e3f80 0x55d1ae700bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae512f10 con 0x55d1ae5d0880 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1ae55a620 con 0x55d1ae5d1090 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55d1ae554330 con 0x55d1ae5e3f80 2024-05-17T22:05:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 10 client.0 _ll_register_callbacks cb 0x7ffb900af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43924/0 (socket says 172.21.15.83:43924) 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 -- 172.21.15.83:0/3753760315 learned_addr learned my addr 172.21.15.83:0/3753760315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba5372640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5e3f80 0x55d1ae700bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae7bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 -- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5e3f80 msgr2=0x55d1ae700bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5e3f80 0x55d1ae700bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 -- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 msgr2=0x55d1ae7bd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae7bd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 -- 172.21.15.83:0/3753760315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ae506fa0 con 0x55d1ae5d0880 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffba4b71640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffb98018000 tx=0x7ffb98004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4023c0d9d60653c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98030020 con 0x55d1ae5d0880 2024-05-17T22:05:49.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7ffb98004240 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb900b7450 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffb9803a050 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 10 client.7668 fetch_fsmap learned FSMap version 81 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55d1ae206220 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb900b7450 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7ffb9803d020 con 0x55d1ae5d0880 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 10 client.7668 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:05:49.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 10 client.7668 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 20 client.7668 populate_metadata read hostname 'smithi083' 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1e0990 0x55d1ae701100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 4 client.7668 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1ae55e070 con 0x55d1ae1e0990 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1e0990 0x55d1ae701100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb7f7fe640 10 client.7668 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.832+0000 7ffb9ffff640 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1e0990 0x55d1ae701100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8b15a64c4e26d server_cookie=a92447c1f537bec4 in_seq=0 out_seq=0 2024-05-17T22:05:49.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffb7f7fe640 1 -- 172.21.15.83:0/3753760315 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55d1ae55e070 con 0x55d1ae1e0990 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffb7f7fe640 10 client.7668 handle_command_reply: tid=0 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 client.7668 shutdown 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 msgr2=0x55d1ae7bcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffb98018000 tx=0x7ffb98004740 comp rx=0 tx=0).stop 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 shutdown_connections 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x55d1ae1e0990 0x55d1ae701100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55d1ae5e3f80 0x55d1ae700bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1ae5d1090 0x55d1ae7bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffb7f7fe640 0 client.7668 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:05:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 --2- 172.21.15.83:0/3753760315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ae5d0880 0x55d1ae7bcdd0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 >> 172.21.15.83:0/3753760315 conn(0x55d1ae61f010 msgr2=0x55d1ae7492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 shutdown_connections 2024-05-17T22:05:49.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 1 -- 172.21.15.83:0/3753760315 wait complete. 2024-05-17T22:05:49.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffbae04f000 20 client.7668 trim_cache size 0 max 16384 2024-05-17T22:05:49.839 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:05:49.839 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:05:49.839 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: "e6439c7a-fab3-4063-a2d8-d6bbc259f4b9": { 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e6439c7a-fab3-4063-a2d8-d6bbc259f4b9", 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:05:49.840 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 msgr2=0x7ffb8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 0x7ffb8c071910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ffb900ae000 tx=0x7ffb900a7000 comp rx=0 tx=0).stop 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 msgr2=0x7ffba0155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007e90 tx=0x7ffb9400e4b0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 shutdown_connections 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffba015d6f0 0x7ffba00b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffba00d3ae0 0x7ffba0155e30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffb8c06f400 0x7ffb8c071910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 --2- 172.21.15.83:0/1909967999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba00d2140 0x7ffba00b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:05:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 >> 172.21.15.83:0/1909967999 conn(0x7ffba0071180 msgr2=0x7ffba0071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:05:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 shutdown_connections 2024-05-17T22:05:49.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:49.836+0000 7ffba6dd5640 1 -- 172.21.15.83:0/1909967999 wait complete. 2024-05-17T22:05:49.846 INFO:tasks.cephfs.filesystem:scrub status for tag:e6439c7a-fab3-4063-a2d8-d6bbc259f4b9 - {'path': '/', 'tag': 'e6439c7a-fab3-4063-a2d8-d6bbc259f4b9', 'options': 'recursive,repair'} 2024-05-17T22:05:51.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:51.248+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 7 ==== log(4 entries from seq 1 at 2024-05-17T22:05:49.625666+0000) ==== 695+0+0 (secure 0 0 0) 0x7f8e4008d7e0 con 0x7f8e44071b70 2024-05-17T22:05:51.238 INFO:teuthology.orchestra.run.smithi083.stdout: cluster: 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: id: 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: health: HEALTH_OK 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: services: 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: mgr: y(active, since 8m), standbys: x 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: mds: 1/1 daemons up, 3 standby 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: data: 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: volumes: 1/1 healthy 2024-05-17T22:05:51.239 INFO:teuthology.orchestra.run.smithi083.stdout: pools: 3 pools, 97 pgs 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: objects: 30 objects, 19 MiB 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: pgs: 97 active+clean 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: io: 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: client: 85 B/s rd, 1.5 MiB/s wr, 0 op/s rd, 3 op/s wr 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:05:49.625666+0000 mds.a [INF] scrub queued for path: / 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:05:49.625670+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:05:49.625674+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-17T22:05:51.240 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:05:49.627329+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T22:05:54.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:05:54.256+0000 7f8e36ffd640 1 -- 172.21.15.83:0/3248376065 <== mon.0 v2:172.21.15.83:3300/0 8 ==== log(1 entries from seq 5 at 2024-05-17T22:05:52.442947+0000) ==== 189+0+0 (secure 0 0 0) 0x7f8e40057c30 con 0x7f8e44071b70 2024-05-17T22:05:54.247 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:05:52.442947+0000 mds.a [INF] scrub summary: idle 2024-05-17T22:06:01.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:06:01.364 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:06:19.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:06:19.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-17T22:06:20.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.019+0000 7f70bd52b640 1 Processor -- start 2024-05-17T22:06:20.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.019+0000 7f70bd52b640 1 -- start start 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70bd52b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f70b80cf830 0x7f70b80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70bd52b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70b815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70bd52b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b80b5640 con 0x7f70b815d530 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70bd52b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f70b80b58e0 con 0x7f70b80cf830 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70bd52b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70b806f060 con 0x7f70b80cdf40 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70b815d530 0x7f70b806f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40970/0 (socket says 172.21.15.83:40970) 2024-05-17T22:06:20.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b77fe640 1 -- 172.21.15.83:0/889942372 learned_addr learned my addr 172.21.15.83:0/889942372 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b67fc640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70b815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/889942372 >> v2:172.21.15.155:3300/0 conn(0x7f70b80cf830 0x7f70b80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923116229 0 0) 0x7f70b80b5640 con 0x7f70b815d530 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70a0003530 con 0x7f70b815d530 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 761826516 0 0) 0x7f70a0003530 con 0x7f70b815d530 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 msgr2=0x7f70b815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70b815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 >> v2:172.21.15.155:3300/0 conn(0x7f70b80cf830 msgr2=0x7f70b80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 --2- 172.21.15.83:0/889942372 >> v2:172.21.15.155:3300/0 conn(0x7f70b80cf830 0x7f70b80b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b806f340 con 0x7f70b815d530 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f70b0001aa0 con 0x7f70b815d530 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 >> v1:172.21.15.83:6789/0 conn(0x7f70b815d530 legacy=0x7f70b806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70a0003a10 0x7f70a0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0004d50 0x7f70a0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70b80b58e0 con 0x7f70a0003a10 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b806f060 con 0x7f70a0004d50 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70a000ba80 con 0x7f70b80cdf40 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70a0003a10 0x7f70a0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b67fc640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b77fe640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0004d50 0x7f70a0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 -- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 msgr2=0x7f70a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b80cdf40 0x7f70a0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 -- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0004d50 msgr2=0x7f70a0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0004d50 0x7f70a0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b6ffd640 1 -- 172.21.15.83:0/889942372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a0003530 con 0x7f70a0003a10 2024-05-17T22:06:20.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.023+0000 7f70b77fe640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0004d50 0x7f70a0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70a0003a10 0x7f70a0003e80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ae0 tx=0x7f70a801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb13901f966bdcee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80b2070 con 0x7f70a0003a10 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a80a5590 con 0x7f70a0003a10 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70b5ffb640 1 -- 172.21.15.83:0/889942372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80096b0 con 0x7f70a0003a10 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- 172.21.15.83:0/889942372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70a0003a10 msgr2=0x7f70a0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 --2- 172.21.15.83:0/889942372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70a0003a10 0x7f70a0003e80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ae0 tx=0x7f70a801cab0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70b77fe640 1 -- 172.21.15.83:0/889942372 reap_dead start 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- 172.21.15.83:0/889942372 shutdown_connections 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- 172.21.15.83:0/889942372 >> 172.21.15.83:0/889942372 conn(0x7f70b806cd90 msgr2=0x7f70b80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- 172.21.15.83:0/889942372 shutdown_connections 2024-05-17T22:06:20.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- 172.21.15.83:0/889942372 wait complete. 2024-05-17T22:06:20.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 Processor -- start 2024-05-17T22:06:20.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.027+0000 7f70bd52b640 1 -- start start 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0003a10 0x7f70b80aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b815e6f0 0x7f70b80aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8073210 con 0x7f70b815efa0 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8073090 con 0x7f70a0003a10 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70b8073390 con 0x7f70b815e6f0 2024-05-17T22:06:20.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38056/0 (socket says 172.21.15.83:38056) 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 -- 172.21.15.83:0/421852088 learned_addr learned my addr 172.21.15.83:0/421852088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0003a10 0x7f70b80aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b77fe640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b815e6f0 0x7f70b80aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 -- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b815e6f0 msgr2=0x7f70b80aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b815e6f0 0x7f70b80aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 -- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0003a10 msgr2=0x7f70b80aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0003a10 0x7f70b80aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 -- 172.21.15.83:0/421852088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b81f5a30 con 0x7f70b815efa0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b67fc640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f70ac010850 tx=0x7f70ac010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 -- 172.21.15.83:0/421852088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00f040 con 0x7f70b815efa0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 -- 172.21.15.83:0/421852088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ac00eb60 con 0x7f70b815efa0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b81f5d10 con 0x7f70b815efa0 2024-05-17T22:06:20.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 -- 172.21.15.83:0/421852088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac01c400 con 0x7f70b815efa0 2024-05-17T22:06:20.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b815e1d0 con 0x7f70b815efa0 2024-05-17T22:06:20.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 -- 172.21.15.83:0/421852088 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f70ac01c5a0 con 0x7f70b815efa0 2024-05-17T22:06:20.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 0x7f70a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f709f7fe640 1 -- 172.21.15.83:0/421852088 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f70ac08f890 con 0x7f70b815efa0 2024-05-17T22:06:20.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.031+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 0x7f70a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 0x7f70a4071940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ae0 tx=0x7f70a80046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 Processor -- start 2024-05-17T22:06:20.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- start start 2024-05-17T22:06:20.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb258360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bdb0f6a90 con 0x555bdb0f1f80 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555bdb06a120 con 0x555bdb0df090 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555bdb059cc0 con 0x555bdb0de880 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38068/0 (socket says 172.21.15.83:38068) 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 -- 172.21.15.83:0/2377351510 learned_addr learned my addr 172.21.15.83:0/2377351510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b67fc640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb258360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b77fe640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 -- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 msgr2=0x555bdb258360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb258360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 -- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 msgr2=0x555bdb2588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 -- 172.21.15.83:0/2377351510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdb016110 con 0x555bdb0f1f80 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b77fe640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f70a801cf90 tx=0x7f70a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e345e2d8766527 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f709cff9640 1 -- 172.21.15.83:0/2377351510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80c7070 con 0x555bdb0f1f80 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f709cff9640 1 -- 172.21.15.83:0/2377351510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a80036e0 con 0x555bdb0f1f80 2024-05-17T22:06:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f709cff9640 1 -- 172.21.15.83:0/2377351510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80b2040 con 0x555bdb0f1f80 2024-05-17T22:06:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- 172.21.15.83:0/2377351510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 msgr2=0x555bdb257e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f70a801cf90 tx=0x7f70a8004740 comp rx=0 tx=0).stop 2024-05-17T22:06:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- 172.21.15.83:0/2377351510 shutdown_connections 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb258360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2377351510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb257e20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- 172.21.15.83:0/2377351510 >> 172.21.15.83:0/2377351510 conn(0x555bdb12d010 msgr2=0x555bdb20d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- 172.21.15.83:0/2377351510 shutdown_connections 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.035+0000 7f70c47a5000 1 -- 172.21.15.83:0/2377351510 wait complete. 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- start start 2024-05-17T22:06:20.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 Processor -- start 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 0x555bdb214a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 0x555bdb219be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bdb015ec0 con 0x555bdb0df090 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555bdb05dd00 con 0x555bdb0de880 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555bdad23680 con 0x555bdb0f1f80 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 10 client.0 _ll_register_callbacks cb 0x555bdb1111c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38072/0 (socket says 172.21.15.83:38072) 2024-05-17T22:06:20.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 -- 172.21.15.83:0/2947937163 learned_addr learned my addr 172.21.15.83:0/2947937163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b77fe640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 0x555bdb219be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 0x555bdb214a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 -- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 msgr2=0x555bdb219be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 0x555bdb219be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 -- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 msgr2=0x555bdb214a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 0x555bdb214a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 -- 172.21.15.83:0/2947937163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdb018890 con 0x555bdb0df090 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 0x555bdb214a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b77fe640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 0x555bdb219be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b67fc640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f70ac010820 tx=0x7f70ac05bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0c95924c36acd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:20.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac08b060 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f70ac05b570 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a80036e0 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f70ac075070 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 10 client.7680 fetch_fsmap learned FSMap version 81 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x555bdb01a650 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a80036e0 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f70ac024030 con 0x555bdb0df090 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 10 client.7680 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 10 client.7680 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 20 client.7680 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdad00930 0x555bdb25ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 4 client.7680 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bdaff5c40 con 0x555bdad00930 2024-05-17T22:06:20.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.039+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdad00930 0x555bdb25ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f7095ffb640 10 client.7680 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdad00930 0x555bdb25ee90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bc4501f5bca18f server_cookie=ce51caeafe7e4d0d in_seq=0 out_seq=0 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f7095ffb640 1 -- 172.21.15.83:0/2947937163 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bdaff5c40 con 0x555bdad00930 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f7095ffb640 10 client.7680 handle_command_reply: tid=0 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 client.7680 shutdown 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 msgr2=0x555bdb214fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f70ac010820 tx=0x7f70ac05bc20 comp rx=0 tx=0).stop 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 shutdown_connections 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdad00930 0x555bdb25ee90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0f1f80 0x555bdb219be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f7095ffb640 0 client.7680 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb214fb0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 --2- 172.21.15.83:0/2947937163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0de880 0x555bdb214a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 >> 172.21.15.83:0/2947937163 conn(0x555bdb12d010 msgr2=0x555bdb20d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 shutdown_connections 2024-05-17T22:06:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 1 -- 172.21.15.83:0/2947937163 wait complete. 2024-05-17T22:06:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.043+0000 7f70c47a5000 20 client.7680 trim_cache size 0 max 16384 2024-05-17T22:06:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.059+0000 7f70c47a5000 1 Processor -- start 2024-05-17T22:06:20.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.059+0000 7f70c47a5000 1 -- start start 2024-05-17T22:06:20.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0f1f80 0x555bdb2c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70b82021d0 con 0x555bdb0df090 2024-05-17T22:06:20.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b82011f0 con 0x555bdb0f1f80 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f70b81f9230 con 0x555bdb0de880 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38086/0 (socket says 172.21.15.83:38086) 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 -- 172.21.15.83:0/3944725380 learned_addr learned my addr 172.21.15.83:0/3944725380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 -- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 msgr2=0x555bdb2c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 -- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0f1f80 msgr2=0x555bdb2c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0f1f80 0x555bdb2c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 -- 172.21.15.83:0/3944725380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bdaff98c0 con 0x555bdb0df090 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70b67fc640 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f70ac05b810 tx=0x7f70ac016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e43f56c264292e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f7097fff640 1 -- 172.21.15.83:0/3944725380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac08b030 con 0x555bdb0df090 2024-05-17T22:06:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f7097fff640 1 -- 172.21.15.83:0/3944725380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ac05b590 con 0x555bdb0df090 2024-05-17T22:06:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f7097fff640 1 -- 172.21.15.83:0/3944725380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac024de0 con 0x555bdb0df090 2024-05-17T22:06:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- 172.21.15.83:0/3944725380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 msgr2=0x555bdb2c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f70ac05b810 tx=0x7f70ac016070 comp rx=0 tx=0).stop 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- 172.21.15.83:0/3944725380 shutdown_connections 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0f1f80 0x555bdb2c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0df090 0x555bdb2c70f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 --2- 172.21.15.83:0/3944725380 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- 172.21.15.83:0/3944725380 >> 172.21.15.83:0/3944725380 conn(0x555bdb12d010 msgr2=0x555bdb213a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- 172.21.15.83:0/3944725380 shutdown_connections 2024-05-17T22:06:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- 172.21.15.83:0/3944725380 wait complete. 2024-05-17T22:06:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 -- start start 2024-05-17T22:06:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.063+0000 7f70c47a5000 1 Processor -- start 2024-05-17T22:06:20.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bdb05d010 con 0x555bdb0f1f80 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8200ba0 con 0x555bdb0df090 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555bdad23680 con 0x555bdb0de880 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 10 client.0 _ll_register_callbacks cb 0x7f70ac0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38094/0 (socket says 172.21.15.83:38094) 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 -- 172.21.15.83:0/1462398363 learned_addr learned my addr 172.21.15.83:0/1462398363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b67fc640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 -- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 msgr2=0x555bdb2cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 -- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 msgr2=0x555bdb2cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 -- 172.21.15.83:0/1462398363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bdb014fa0 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b77fe640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f70b815a970 tx=0x7f70b000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eee2650ee7339da server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b000edf0 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f70b0004b70 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ac05b590 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f70b0021050 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 10 client.7686 fetch_fsmap learned FSMap version 81 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x555bdad14220 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ac05b590 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f70b0024020 con 0x555bdb0f1f80 2024-05-17T22:06:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 10 client.7686 fetch_fsmap finished waiting for FSMap version 81 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 10 client.7686 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 20 client.7686 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdacee990 0x555bdb20f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 4 client.7686 mds_command: new command op to 7407 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 --> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bdb06c070 con 0x555bdacee990 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdacee990 0x555bdb20f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 10 client.7686 ms_handle_connect on v2:172.21.15.83:6834/1996242614 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70b6ffd640 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdacee990 0x555bdb20f770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2db6e0dfcc1beb3 server_cookie=523ad30bdf02871 in_seq=0 out_seq=0 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 1 -- 172.21.15.83:0/1462398363 <== mds.0 v2:172.21.15.83:6834/1996242614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bdb06c070 con 0x555bdacee990 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 10 client.7686 handle_command_reply: tid=0 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 client.7686 shutdown 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 msgr2=0x555bdb20f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f70b815a970 tx=0x7f70b000e590 comp rx=0 tx=0).stop 2024-05-17T22:06:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 shutdown_connections 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:6834/1996242614,v1:172.21.15.83:6836/1996242614] conn(0x555bdacee990 0x555bdb20f770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bdb0f1f80 0x555bdb20f230 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bdb0df090 0x555bdb2cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f7095ffb640 0 client.7686 ms_handle_reset on v2:172.21.15.83:6834/1996242614 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 --2- 172.21.15.83:0/1462398363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555bdb0de880 0x555bdb2cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 >> 172.21.15.83:0/1462398363 conn(0x555bdb12d010 msgr2=0x555bdb2143e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 shutdown_connections 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 1 -- 172.21.15.83:0/1462398363 wait complete. 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.067+0000 7f70c47a5000 20 client.7686 trim_cache size 0 max 16384 2024-05-17T22:06:20.066 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:20.067 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:06:20.067 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:06:20.067 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 msgr2=0x7f70a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 0x7f70a4071940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009ae0 tx=0x7f70a80046d0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 msgr2=0x7f70b81f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f70ac010850 tx=0x7f70ac010d10 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 shutdown_connections 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70b815e6f0 0x7f70b80aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70b815efa0 0x7f70b81f5050 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f70a406f430 0x7f70a4071940 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 --2- 172.21.15.83:0/421852088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70a0003a10 0x7f70b80aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 >> 172.21.15.83:0/421852088 conn(0x7f70b806cd90 msgr2=0x7f70b8075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 shutdown_connections 2024-05-17T22:06:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.071+0000 7f70bd52b640 1 -- 172.21.15.83:0/421852088 wait complete. 2024-05-17T22:06:20.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:06:20.183 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:06:20.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 Processor -- start 2024-05-17T22:06:20.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 -- start start 2024-05-17T22:06:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f364c06eec0 0x7f364c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f364c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364c06f2d0 con 0x7f364c15be70 2024-05-17T22:06:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f364c06f5a0 con 0x7f364c06eec0 2024-05-17T22:06:20.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3653977640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f364c06f880 con 0x7f364c0b44c0 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3650f12640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f364c15be70 0x7f364c1601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:40986/0 (socket says 172.21.15.83:40986) 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3650f12640 1 -- 172.21.15.83:0/867885749 learned_addr learned my addr 172.21.15.83:0/867885749 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3651713640 1 --2- 172.21.15.83:0/867885749 >> v2:172.21.15.155:3300/0 conn(0x7f364c06eec0 0x7f364c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3651f14640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f364c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615396224 0 0) 0x7f364c06f2d0 con 0x7f364c15be70 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3638003530 con 0x7f364c15be70 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1129812678 0 0) 0x7f3638003530 con 0x7f364c15be70 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 msgr2=0x7f364c0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f364c0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 >> v2:172.21.15.155:3300/0 conn(0x7f364c06eec0 msgr2=0x7f364c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 --2- 172.21.15.83:0/867885749 >> v2:172.21.15.155:3300/0 conn(0x7f364c06eec0 0x7f364c0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c06fb60 con 0x7f364c15be70 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f363c001700 con 0x7f364c15be70 2024-05-17T22:06:20.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 >> v1:172.21.15.83:6789/0 conn(0x7f364c15be70 legacy=0x7f364c1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638003a10 0x7f3638003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3638004810 0x7f3638004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f3638005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f364c06f5a0 con 0x7f3638004810 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f364c06f880 con 0x7f3638003a10 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f363800dd10 con 0x7f364c0b44c0 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.347+0000 7f3650f12640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3638004810 0x7f3638004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3651713640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638003a10 0x7f3638003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 -- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 msgr2=0x7f3638005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3651f14640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f3638005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c0b44c0 0x7f3638005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 -- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638003a10 msgr2=0x7f3638003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638003a10 0x7f3638003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 -- 172.21.15.83:0/867885749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3638003530 con 0x7f3638004810 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3650f12640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3638004810 0x7f3638004c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f364c0b56c0 tx=0x7f363c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f16a229f19a6e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c007d60 con 0x7f3638004810 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363c0a9070 con 0x7f3638004810 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3643fff640 1 -- 172.21.15.83:0/867885749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c005d20 con 0x7f3638004810 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 -- 172.21.15.83:0/867885749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3638004810 msgr2=0x7f3638004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 --2- 172.21.15.83:0/867885749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3638004810 0x7f3638004c20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f364c0b56c0 tx=0x7f363c01cc40 comp rx=0 tx=0).stop 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3651f14640 1 -- 172.21.15.83:0/867885749 reap_dead start 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 -- 172.21.15.83:0/867885749 shutdown_connections 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 -- 172.21.15.83:0/867885749 >> 172.21.15.83:0/867885749 conn(0x7f364c06cd90 msgr2=0x7f364c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 -- 172.21.15.83:0/867885749 shutdown_connections 2024-05-17T22:06:20.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.351+0000 7f3653977640 1 -- 172.21.15.83:0/867885749 wait complete. 2024-05-17T22:06:20.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 Processor -- start 2024-05-17T22:06:20.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- start start 2024-05-17T22:06:20.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638004810 0x7f364c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c1fc7a0 0x7f364c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f364c160a40 con 0x7f364c0b2400 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f364c1608c0 con 0x7f3638004810 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f364c072e80 con 0x7f364c1fc7a0 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38122/0 (socket says 172.21.15.83:38122) 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 -- 172.21.15.83:0/913496569 learned_addr learned my addr 172.21.15.83:0/913496569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651713640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638004810 0x7f364c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3650f12640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c1fc7a0 0x7f364c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 -- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c1fc7a0 msgr2=0x7f364c149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c1fc7a0 0x7f364c149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 -- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638004810 msgr2=0x7f364c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638004810 0x7f364c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 -- 172.21.15.83:0/913496569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c0b2d50 con 0x7f364c0b2400 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3651f14640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f364c074090 tx=0x7f364400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364401c070 con 0x7f364c0b2400 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644017400 con 0x7f364c0b2400 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c152fa0 con 0x7f364c0b2400 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440177f0 con 0x7f364c0b2400 2024-05-17T22:06:20.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.355+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c153560 con 0x7f364c0b2400 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364c0b08a0 con 0x7f364c0b2400 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f364401f070 con 0x7f364c0b2400 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3641ffb640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 0x7f3630071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3651713640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 0x7f3630071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7f364408f240 con 0x7f364c0b2400 2024-05-17T22:06:20.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.359+0000 7f3651713640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 0x7f3630071940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f364800aab0 tx=0x7f3648008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.363+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f364c0b2d50 con 0x7f364c0b2400 2024-05-17T22:06:20.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.555+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f364c0b01c0 con 0x7f364c0b2400 2024-05-17T22:06:20.548 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:06:20.555+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:06:20.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.559+0000 7f3641ffb640 1 -- 172.21.15.83:0/913496569 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7f3644012070 con 0x7f364c0b2400 2024-05-17T22:06:20.550 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 msgr2=0x7f3630071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 0x7f3630071940 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f364800aab0 tx=0x7f3648008040 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 msgr2=0x7f364c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f364c074090 tx=0x7f364400f4a0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 shutdown_connections 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f364c0b2400 0x7f364c0b2810 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f364c1fc7a0 0x7f364c149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f363006f430 0x7f3630071940 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 --2- 172.21.15.83:0/913496569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3638004810 0x7f364c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 >> 172.21.15.83:0/913496569 conn(0x7f364c06cd90 msgr2=0x7f364c0cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 shutdown_connections 2024-05-17T22:06:20.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.563+0000 7f3653977640 1 -- 172.21.15.83:0/913496569 wait complete. 2024-05-17T22:06:20.566 INFO:tasks.ceph.mds.a.smithi083.stderr: -5921> 2024-05-17T22:05:33.352+0000 7fee20eba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:06:20.575 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-17T22:06:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.643+0000 7fba14dc6400 1 -- start start 2024-05-17T22:06:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.643+0000 7fba14dc6400 1 Processor -- start 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 0x5587bca50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3ed80 0x5587bca51b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587bca66480 con 0x5587bbd41180 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587bca66600 con 0x5587bbd3ed80 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5587bca66780 con 0x5587bbd3f200 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38158/0 (socket says 172.21.15.83:38158) 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 -- 172.21.15.83:0/3673010963 learned_addr learned my addr 172.21.15.83:0/3673010963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13d86640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3ed80 0x5587bca51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba12d84640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 0x5587bca50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 -- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 msgr2=0x5587bca50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 0x5587bca50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 -- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3ed80 msgr2=0x5587bca51b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3ed80 0x5587bca51b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 -- 172.21.15.83:0/3673010963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bca4eb40 con 0x5587bbd41180 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba12d84640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 0x5587bca50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba13585640 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x5587bcae7320 tx=0x5587bbd94a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b561485938aae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba10d80640 1 -- 172.21.15.83:0/3673010963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb6b60 con 0x5587bbd41180 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba10d80640 1 -- 172.21.15.83:0/3673010963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5587bbcca1c0 con 0x5587bbd41180 2024-05-17T22:06:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba10d80640 1 -- 172.21.15.83:0/3673010963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb61a0 con 0x5587bbd41180 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bca4f0e0 con 0x5587bbd41180 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5587bca4f2c0 con 0x5587bbd41180 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 4 waiting for MDS map... 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba10d80640 1 -- 172.21.15.83:0/3673010963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x5587bca58a00 con 0x5587bbd41180 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba10d80640 1 -- 172.21.15.83:0/3673010963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 82) ==== 2885+0+0 (secure 0 0 0) 0x5587bbcd7180 con 0x5587bbd41180 2024-05-17T22:06:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 4 Got MDS map 82 2024-05-17T22:06:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 10 main: JournalTool::main 2024-05-17T22:06:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.647+0000 7fba14dc6400 4 main: JournalTool: connecting to RADOS... 2024-05-17T22:06:20.643 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 Processor -- start 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- start start 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587bbce1980 con 0x5587bbd40880 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587bca66f00 con 0x5587bbd3e000 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5587bca67080 con 0x5587bbd3e480 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38166/0 (socket says 172.21.15.83:38166) 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 -- 172.21.15.83:0/120775356 learned_addr learned my addr 172.21.15.83:0/120775356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba13d86640 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba13585640 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 -- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 msgr2=0x5587bca51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca51600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 -- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 msgr2=0x5587bca50000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca50000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 -- 172.21.15.83:0/120775356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bca4e960 con 0x5587bbd40880 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba12d84640 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x5587bcbbcc30 tx=0x5587bcbc8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba756bb1fc36fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba0ed7c640 1 -- 172.21.15.83:0/120775356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb6b60 con 0x5587bbd40880 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba0ed7c640 1 -- 172.21.15.83:0/120775356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5587bbccac40 con 0x5587bbd40880 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba0ed7c640 1 -- 172.21.15.83:0/120775356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb61a0 con 0x5587bbd40880 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- 172.21.15.83:0/120775356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 msgr2=0x5587bca52100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x5587bcbbcc30 tx=0x5587bcbc8cc0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- 172.21.15.83:0/120775356 shutdown_connections 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52100 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca50000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 --2- 172.21.15.83:0/120775356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- 172.21.15.83:0/120775356 >> 172.21.15.83:0/120775356 conn(0x5587bbd3e900 msgr2=0x5587bcb9c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- 172.21.15.83:0/120775356 shutdown_connections 2024-05-17T22:06:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.651+0000 7fba14dc6400 1 -- 172.21.15.83:0/120775356 wait complete. 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 Processor -- start 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- start start 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca53180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca52c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5587bca67380 con 0x5587bbd40880 2024-05-17T22:06:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5587bca67200 con 0x5587bbd3e000 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5587bca67500 con 0x5587bbd3e480 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38178/0 (socket says 172.21.15.83:38178) 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 -- 172.21.15.83:0/440503299 learned_addr learned my addr 172.21.15.83:0/440503299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba13d86640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca52c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 msgr2=0x5587bca52c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca52c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 msgr2=0x5587bca53180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3e000 0x5587bca53180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bca4fe00 con 0x5587bbd40880 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3e480 0x5587bca52c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba12d84640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x5587bcbbdef0 tx=0x5587bcbc8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba0d579640 1 -- 172.21.15.83:0/440503299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb6b60 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba0d579640 1 -- 172.21.15.83:0/440503299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5587bbccaa80 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5587bca4ed20 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba0d579640 1 -- 172.21.15.83:0/440503299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5587bbcb61a0 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bca4e960 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.655+0000 7fba14dc6400 4 main: JournalTool: resolving pool 14 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba0d579640 1 -- 172.21.15.83:0/440503299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x5587bbfcec00 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba0d579640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 0x5587bca53700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba0d579640 1 -- 172.21.15.83:0/440503299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x5587bbfd2600 con 0x5587bbd40880 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba13d86640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 0x5587bca53700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba14dc6400 4 main: JournalTool: creating IoCtx.. 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba14dc6400 4 main: Executing for rank 0 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba14dc6400 4 Filtering by inode '1099511627777' 2024-05-17T22:06:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5587bbd40d00 0x5587bca52100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x5587bc0ca000 con 0x5587bbd40d00 2024-05-17T22:06:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5587bbd40d00 0x5587bca52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba13d86640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 0x5587bca53700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x5587bbd84210 tx=0x5587bcbc8f60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.659+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5587bbd40d00 0x5587bca52100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba13585640 1 -- 172.21.15.83:0/440503299 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5587bbdb4000 con 0x5587bbd40d00 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba14dc6400 4 Pointer 400.00000000 is readable 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba14dc6400 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5587bbd41600 0x5587bca51600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x5587bc0ca400 con 0x5587bbd41600 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba13d86640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5587bbd41600 0x5587bca51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.663+0000 7fba13d86640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5587bbd41600 0x5587bca51600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba13d86640 1 -- 172.21.15.83:0/440503299 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5587bbdb4500 con 0x5587bbd41600 2024-05-17T22:06:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Header 0x400000 0x405f0e 0x40628a 2024-05-17T22:06:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Starting journal scan from offset 0x405f0e 2024-05-17T22:06:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x5587bc0cb400 con 0x5587bbd41600 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba13d86640 1 -- 172.21.15.83:0/440503299 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5587bbdb4500 con 0x5587bbd41600 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Parsing data, 0x37c bytes available 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Attempting decode at 0x405f0e 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Consumed 0x37c bytes 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 10 Valid entry at 0x405f0e 2024-05-17T22:06:20.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5587bbd40400 0x5587bca50000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x5587bc0cb800 con 0x5587bbd40400 2024-05-17T22:06:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5587bbd40400 0x5587bca50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.667+0000 7fba13585640 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5587bbd40400 0x5587bca50000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba13585640 1 -- 172.21.15.83:0/440503299 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5587bbdb4000 con 0x5587bbd40400 2024-05-17T22:06:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 Reached end of journal objects 2024-05-17T22:06:20.663 INFO:teuthology.orchestra.run.smithi083.stdout:Events by type: 2024-05-17T22:06:20.663 INFO:teuthology.orchestra.run.smithi083.stdout:Errors: 0 2024-05-17T22:06:20.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 Scanned objects, 0 missing, 1 valid 2024-05-17T22:06:20.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 Events scanned, 0 gaps 2024-05-17T22:06:20.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 Found 1 valid events 2024-05-17T22:06:20.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 Selected 0 events events for processing 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 4 0 events 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5587bbd40400 msgr2=0x5587bca50000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5587bbd40400 0x5587bca50000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5587bbd41600 msgr2=0x5587bca51600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5587bbd41600 0x5587bca51600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5587bbd40d00 msgr2=0x5587bca52100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5587bbd40d00 0x5587bca52100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba12d84640 1 -- 172.21.15.83:0/440503299 reap_dead start 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 msgr2=0x5587bca53700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 0x5587bca53700 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x5587bbd84210 tx=0x5587bcbc8f60 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 msgr2=0x5587bca52680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x5587bcbbdef0 tx=0x5587bcbc8c00 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 shutdown_connections 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd40880 0x5587bca52680 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/440503299 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5587bbd41a80 0x5587bca53700 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 >> 172.21.15.83:0/440503299 conn(0x5587bbd3e900 msgr2=0x5587bcb9c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 shutdown_connections 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/440503299 wait complete. 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 msgr2=0x5587bca51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x5587bcae7320 tx=0x5587bbd94a80 comp rx=0 tx=0).stop 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 shutdown_connections 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5587bbd3ed80 0x5587bca51b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5587bbd3f200 0x5587bca50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 --2- 172.21.15.83:0/3673010963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5587bbd41180 0x5587bca51080 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 >> 172.21.15.83:0/3673010963 conn(0x5587bbd3ff80 msgr2=0x5587bcb9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 shutdown_connections 2024-05-17T22:06:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.671+0000 7fba14dc6400 1 -- 172.21.15.83:0/3673010963 wait complete. 2024-05-17T22:06:20.670 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.095499, result: Events by type: Errors: 0 2024-05-17T22:06:20.671 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-17T22:06:20.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 -- start start 2024-05-17T22:06:20.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 Processor -- start 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0955680 0x55b5c1666580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0956400 0x55b5c1666b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1682180 con 0x55b5c0955b00 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1682300 con 0x55b5c0955680 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b5c1682480 con 0x55b5c0956400 2024-05-17T22:06:20.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff6236d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff6236d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38182/0 (socket says 172.21.15.83:38182) 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff6236d640 1 -- 172.21.15.83:0/1953579300 learned_addr learned my addr 172.21.15.83:0/1953579300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0955680 0x55b5c1666580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.739+0000 7eff6336f640 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0956400 0x55b5c1666b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0956400 msgr2=0x55b5c1666b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0956400 0x55b5c1666b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0955680 msgr2=0x55b5c1666580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0955680 0x55b5c1666580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/1953579300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c1664b40 con 0x55b5c0955b00 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55b5c16faf90 tx=0x55b5c09aaae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e84fd2a9ffff974 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff60369640 1 -- 172.21.15.83:0/1953579300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08cc9c0 con 0x55b5c0955b00 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff60369640 1 -- 172.21.15.83:0/1953579300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5c08e0e00 con 0x55b5c0955b00 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff60369640 1 -- 172.21.15.83:0/1953579300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08cc4e0 con 0x55b5c0955b00 2024-05-17T22:06:20.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c1664d20 con 0x55b5c0955b00 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b5c1664f00 con 0x55b5c0955b00 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 4 waiting for MDS map... 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff60369640 1 -- 172.21.15.83:0/1953579300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x55b5c166ea00 con 0x55b5c0955b00 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff60369640 1 -- 172.21.15.83:0/1953579300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 82) ==== 2885+0+0 (secure 0 0 0) 0x55b5c08ed180 con 0x55b5c0955b00 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 4 Got MDS map 82 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 10 main: JournalTool::main 2024-05-17T22:06:20.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 4 main: JournalTool: connecting to RADOS... 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 Processor -- start 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- start start 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1682600 con 0x55b5c0954480 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1682780 con 0x55b5c0954900 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b5c1682900 con 0x55b5c0954d80 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38198/0 (socket says 172.21.15.83:38198) 2024-05-17T22:06:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/3933272998 learned_addr learned my addr 172.21.15.83:0/3933272998 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6336f640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff62b6e640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 msgr2=0x55b5c1667080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 msgr2=0x55b5c1667600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 -- 172.21.15.83:0/3933272998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c16641e0 con 0x55b5c0954480 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6336f640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff62b6e640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff6236d640 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55b5c17b2b10 tx=0x55b5c09aa840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c43686529ea89ac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff5e365640 1 -- 172.21.15.83:0/3933272998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08ccb60 con 0x55b5c0954480 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff5e365640 1 -- 172.21.15.83:0/3933272998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5c08e0c40 con 0x55b5c0954480 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.743+0000 7eff5e365640 1 -- 172.21.15.83:0/3933272998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08cc340 con 0x55b5c0954480 2024-05-17T22:06:20.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/3933272998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 msgr2=0x55b5c1667b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55b5c17b2b10 tx=0x55b5c09aa840 comp rx=0 tx=0).stop 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/3933272998 shutdown_connections 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- 172.21.15.83:0/3933272998 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/3933272998 >> 172.21.15.83:0/3933272998 conn(0x55b5c0955200 msgr2=0x55b5c0bb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/3933272998 shutdown_connections 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/3933272998 wait complete. 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 Processor -- start 2024-05-17T22:06:20.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- start start 2024-05-17T22:06:20.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1683500 con 0x55b5c0954900 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5c1682a80 con 0x55b5c0954480 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b5c1683680 con 0x55b5c0954d80 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38200/0 (socket says 172.21.15.83:38200) 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 learned_addr learned my addr 172.21.15.83:0/1911199692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 msgr2=0x55b5c1667080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff6236d640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0954d80 0x55b5c1667080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954480 msgr2=0x55b5c1667b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c1665a40 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0954480 0x55b5c1667b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x55b5c17b3890 tx=0x55b5c17a8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff5cb62640 1 -- 172.21.15.83:0/1911199692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08ccb60 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff5cb62640 1 -- 172.21.15.83:0/1911199692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5c08e0a80 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5c1665c20 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff5cb62640 1 -- 172.21.15.83:0/1911199692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5c08cc340 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c1665a40 con 0x55b5c0954900 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.747+0000 7eff643af400 4 main: JournalTool: resolving pool 14 2024-05-17T22:06:20.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff5cb62640 1 -- 172.21.15.83:0/1911199692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55b5c0bc6c00 con 0x55b5c0954900 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff5cb62640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 0x55b5c1668100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff5cb62640 1 -- 172.21.15.83:0/1911199692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x55b5c17f5000 con 0x55b5c0954900 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 0x55b5c1668100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 4 main: JournalTool: creating IoCtx.. 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 4 main: Executing for rank 0 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 4 Filtering by inode '1099511627778' 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55b5c0954000 0x55b5c1668680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5c0cd6000 con 0x55b5c0954000 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 0x55b5c1668100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55b5c09e0e10 tx=0x55b5c17a8ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6236d640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55b5c0954000 0x55b5c1668680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6236d640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55b5c0954000 0x55b5c1668680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6236d640 1 -- 172.21.15.83:0/1911199692 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55b5c099a000 con 0x55b5c0954000 2024-05-17T22:06:20.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 4 Pointer 400.00000000 is readable 2024-05-17T22:06:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-17T22:06:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b5c0956880 0x55b5c1668c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5c0cd6400 con 0x55b5c0956880 2024-05-17T22:06:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.751+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b5c0956880 0x55b5c1668c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff6336f640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b5c0956880 0x55b5c1668c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff6336f640 1 -- 172.21.15.83:0/1911199692 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55b5c099a500 con 0x55b5c0956880 2024-05-17T22:06:20.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Header 0x400000 0x405f0e 0x40628a 2024-05-17T22:06:20.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Starting journal scan from offset 0x405f0e 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5c0cd7400 con 0x55b5c0956880 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff6336f640 1 -- 172.21.15.83:0/1911199692 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55b5c099a500 con 0x55b5c0956880 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Parsing data, 0x37c bytes available 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Attempting decode at 0x405f0e 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Consumed 0x37c bytes 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 10 Valid entry at 0x405f0e 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55b5c0956d00 0x55b5c1669180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5c0cd7800 con 0x55b5c0956d00 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55b5c0956d00 0x55b5c1669180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff62b6e640 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55b5c0956d00 0x55b5c1669180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b5c099a780 con 0x55b5c0956d00 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Reached end of journal objects 2024-05-17T22:06:20.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Scanned objects, 0 missing, 1 valid 2024-05-17T22:06:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Events scanned, 0 gaps 2024-05-17T22:06:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Found 1 valid events 2024-05-17T22:06:20.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 Selected 0 events events for processing 2024-05-17T22:06:20.753 INFO:teuthology.orchestra.run.smithi083.stdout:Events by type: 2024-05-17T22:06:20.753 INFO:teuthology.orchestra.run.smithi083.stdout:Errors: 0 2024-05-17T22:06:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 4 0 events 2024-05-17T22:06:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55b5c0956d00 msgr2=0x55b5c1669180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55b5c0956d00 0x55b5c1669180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b5c0956880 msgr2=0x55b5c1668c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55b5c0956880 0x55b5c1668c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55b5c0954000 msgr2=0x55b5c1668680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55b5c0954000 0x55b5c1668680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff62b6e640 1 -- 172.21.15.83:0/1911199692 reap_dead start 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 msgr2=0x55b5c1668100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 0x55b5c1668100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55b5c09e0e10 tx=0x55b5c17a8ea0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 msgr2=0x55b5c1667600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x55b5c17b3890 tx=0x55b5c17a8c00 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 shutdown_connections 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0954900 0x55b5c1667600 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1911199692 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55b5c0957180 0x55b5c1668100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 >> 172.21.15.83:0/1911199692 conn(0x55b5c0955200 msgr2=0x55b5c0bb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 shutdown_connections 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1911199692 wait complete. 2024-05-17T22:06:20.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 msgr2=0x55b5c1666000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55b5c16faf90 tx=0x55b5c09aaae0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 shutdown_connections 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b5c0956400 0x55b5c1666b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5c0955b00 0x55b5c1666000 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 --2- 172.21.15.83:0/1953579300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5c0955680 0x55b5c1666580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 >> 172.21.15.83:0/1953579300 conn(0x55b5c0955f80 msgr2=0x55b5bc895800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 shutdown_connections 2024-05-17T22:06:20.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.755+0000 7eff643af400 1 -- 172.21.15.83:0/1953579300 wait complete. 2024-05-17T22:06:20.758 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.087332, result: Events by type: Errors: 0 2024-05-17T22:06:20.758 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-17T22:06:20.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 Processor -- start 2024-05-17T22:06:20.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- start start 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb510850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5abfd0 0x5560cb5a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5560cb4201b0 con 0x5560cb5adbf0 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560cb47bef0 con 0x5560cb5abfd0 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5560cb4d4240 con 0x5560cb4d44a0 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38216/0 (socket says 172.21.15.83:38216) 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 -- 172.21.15.83:0/1735314363 learned_addr learned my addr 172.21.15.83:0/1735314363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858197d640 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5abfd0 0x5560cb5a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858217e640 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb510850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 -- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 msgr2=0x5560cb510850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb510850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 -- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5abfd0 msgr2=0x5560cb5a7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5abfd0 0x5560cb5a7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 -- 172.21.15.83:0/1735314363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cb3fac10 con 0x5560cb5adbf0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858297f640 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8574001dd0 tx=0x7f857400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fabaf72026abded server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858097b640 1 -- 172.21.15.83:0/1735314363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574012020 con 0x5560cb5adbf0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858097b640 1 -- 172.21.15.83:0/1735314363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857400e690 con 0x5560cb5adbf0 2024-05-17T22:06:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858097b640 1 -- 172.21.15.83:0/1735314363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400ea80 con 0x5560cb5adbf0 2024-05-17T22:06:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- 172.21.15.83:0/1735314363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 msgr2=0x5560cb5a82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8574001dd0 tx=0x7f857400b6d0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- 172.21.15.83:0/1735314363 shutdown_connections 2024-05-17T22:06:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5adbf0 0x5560cb5a82c0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5abfd0 0x5560cb5a7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 --2- 172.21.15.83:0/1735314363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb510850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- 172.21.15.83:0/1735314363 >> 172.21.15.83:0/1735314363 conn(0x5560cb4bce40 msgr2=0x5560cb51ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.831+0000 7f858869c080 1 -- 172.21.15.83:0/1735314363 shutdown_connections 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- 172.21.15.83:0/1735314363 wait complete. 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 Processor -- start 2024-05-17T22:06:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- start start 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb517000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 0x5560cb5a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5adbf0 0x5560cb590ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5560cb510d90 con 0x5560cb5abfd0 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5560cb5a7680 con 0x5560cb5adbf0 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5560cb5aa0b0 con 0x5560cb4d44a0 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 0x5560cb5a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 0x5560cb5a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38222/0 (socket says 172.21.15.83:38222) 2024-05-17T22:06:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 -- 172.21.15.83:0/2212346279 learned_addr learned my addr 172.21.15.83:0/2212346279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858297f640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5adbf0 0x5560cb590ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858217e640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb517000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 -- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 msgr2=0x5560cb517000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb517000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 -- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5adbf0 msgr2=0x5560cb590ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560cb5adbf0 0x5560cb590ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 -- 172.21.15.83:0/2212346279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cb591690 con 0x5560cb5abfd0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858217e640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5560cb4d44a0 0x5560cb517000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858197d640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 0x5560cb5a5b10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f856c01b850 tx=0x7f856c018110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f857b7fe640 1 -- 172.21.15.83:0/2212346279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c0bb020 con 0x5560cb5abfd0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f857b7fe640 1 -- 172.21.15.83:0/2212346279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856c0b1040 con 0x5560cb5abfd0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560cb591930 con 0x5560cb5abfd0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f857b7fe640 1 -- 172.21.15.83:0/2212346279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c0b0070 con 0x5560cb5abfd0 2024-05-17T22:06:20.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.835+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cb591e70 con 0x5560cb5abfd0 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f857b7fe640 1 -- 172.21.15.83:0/2212346279 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f856c0be020 con 0x5560cb5abfd0 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f857b7fe640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f856406dde0 0x7f85640702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858217e640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f856406dde0 0x7f85640702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f857b7fe640 1 -- 172.21.15.83:0/2212346279 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f856c128d40 con 0x5560cb5abfd0 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858869c080 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5560cb593680 0x5560cb593a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5560cb593fd0 con 0x5560cb593680 2024-05-17T22:06:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858297f640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5560cb593680 0x5560cb593a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:20.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858217e640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f856406dde0 0x7f85640702f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f857c00e6e0 tx=0x7f857c007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.839+0000 7f858297f640 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5560cb593680 0x5560cb593a90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:20.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858297f640 1 -- 172.21.15.83:0/2212346279 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v97'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8574008030 con 0x5560cb593680 2024-05-17T22:06:20.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5560cb593680 msgr2=0x5560cb593a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5560cb593680 0x5560cb593a90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f856406dde0 msgr2=0x7f85640702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f856406dde0 0x7f85640702f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f857c00e6e0 tx=0x7f857c007040 comp rx=0 tx=0).stop 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 msgr2=0x5560cb5a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 --2- 172.21.15.83:0/2212346279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5560cb5abfd0 0x5560cb5a5b10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f856c01b850 tx=0x7f856c018110 comp rx=0 tx=0).stop 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858297f640 1 -- 172.21.15.83:0/2212346279 reap_dead start 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 shutdown_connections 2024-05-17T22:06:20.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 >> 172.21.15.83:0/2212346279 conn(0x5560cb4bce40 msgr2=0x5560cb51a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:20.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 shutdown_connections 2024-05-17T22:06:20.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:20.843+0000 7f858869c080 1 -- 172.21.15.83:0/2212346279 wait complete. 2024-05-17T22:06:20.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:06:21.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.051+0000 7f36f6d49640 1 Processor -- start 2024-05-17T22:06:21.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.051+0000 7f36f6d49640 1 -- start start 2024-05-17T22:06:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f6d49640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f36f00b3d00 0x7f36f00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f6d49640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f00d10d0 con 0x7f36f0143f20 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f6d49640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f0070120 con 0x7f36f0073590 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f6d49640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f36f00b4110 con 0x7f36f00b3d00 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36effff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f36f0143f20 0x7f36f0072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41014/0 (socket says 172.21.15.83:41014) 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36effff640 1 -- 172.21.15.83:0/3377201923 learned_addr learned my addr 172.21.15.83:0/3377201923 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3377201923 >> v2:172.21.15.155:3300/0 conn(0x7f36f00b3d00 0x7f36f00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942913403 0 0) 0x7f36f00d10d0 con 0x7f36f0143f20 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36d8003530 con 0x7f36f0143f20 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1483749979 0 0) 0x7f36d8003530 con 0x7f36f0143f20 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 >> v2:172.21.15.155:3300/0 conn(0x7f36f00b3d00 msgr2=0x7f36f00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 --2- 172.21.15.83:0/3377201923 >> v2:172.21.15.155:3300/0 conn(0x7f36f00b3d00 0x7f36f00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 >> v1:172.21.15.155:6789/0 conn(0x7f36f0073590 legacy=0x7f36f01447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f0070400 con 0x7f36f0143f20 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f36e4001700 con 0x7f36f0143f20 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 >> v1:172.21.15.83:6789/0 conn(0x7f36f0143f20 legacy=0x7f36f0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 0x7f36d8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36d8004810 0x7f36d8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8005260 0x7f36d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36f00b4110 con 0x7f36d8003a60 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36d800c1f0 con 0x7f36d8004810 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f36d800c3f0 con 0x7f36d8005260 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 0x7f36d8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36effff640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36d8004810 0x7f36d8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 -- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8005260 msgr2=0x7f36d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f52e6640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8005260 0x7f36d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8005260 0x7f36d800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 -- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36d8004810 msgr2=0x7f36d8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36d8004810 0x7f36d8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f4ae5640 1 -- 172.21.15.83:0/3377201923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d8003530 con 0x7f36d8003a60 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.055+0000 7f36f52e6640 1 -- 172.21.15.83:0/3377201923 reap_dead start 2024-05-17T22:06:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 0x7f36d8003ed0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009ae0 tx=0x7f36e001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78895c80220c4288 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00b2070 con 0x7f36d8003a60 2024-05-17T22:06:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f0145800 con 0x7f36d8003a60 2024-05-17T22:06:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36ef7fe640 1 -- 172.21.15.83:0/3377201923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00096b0 con 0x7f36d8003a60 2024-05-17T22:06:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 -- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 msgr2=0x7f36d8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 0x7f36d8003ed0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009ae0 tx=0x7f36e001ca70 comp rx=0 tx=0).stop 2024-05-17T22:06:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 -- 172.21.15.83:0/3377201923 shutdown_connections 2024-05-17T22:06:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3377201923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8003a60 0x7f36d8003ed0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 -- 172.21.15.83:0/3377201923 >> 172.21.15.83:0/3377201923 conn(0x7f36f006cd90 msgr2=0x7f36f00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 -- 172.21.15.83:0/3377201923 shutdown_connections 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.059+0000 7f36f6d49640 1 -- 172.21.15.83:0/3377201923 wait complete. 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 Processor -- start 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- start start 2024-05-17T22:06:21.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8003a60 0x7f36f015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f013ea90 0x7f36f01618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0073410 con 0x7f36f0161e00 2024-05-17T22:06:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0073290 con 0x7f36f013ea90 2024-05-17T22:06:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f36f0070c00 con 0x7f36d8003a60 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38242/0 (socket says 172.21.15.83:38242) 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 -- 172.21.15.83:0/3228037316 learned_addr learned my addr 172.21.15.83:0/3228037316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8003a60 0x7f36f015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36effff640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f013ea90 0x7f36f01618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 -- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8003a60 msgr2=0x7f36f015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8003a60 0x7f36f015fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 -- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f013ea90 msgr2=0x7f36f01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f013ea90 0x7f36f01618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 -- 172.21.15.83:0/3228037316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f013da30 con 0x7f36f0161e00 2024-05-17T22:06:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f52e6640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f36f00cfce0 tx=0x7f36e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8018070 con 0x7f36f0161e00 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e8003b80 con 0x7f36f0161e00 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f01602a0 con 0x7f36f0161e00 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8005070 con 0x7f36f0161e00 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f013dfb0 con 0x7f36f0161e00 2024-05-17T22:06:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.063+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f00d10d0 con 0x7f36f0161e00 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.067+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f36e801b070 con 0x7f36f0161e00 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.067+0000 7f36ed7fa640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 0x7f36dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.067+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f36e808e9d0 con 0x7f36f0161e00 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.067+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 0x7f36dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.067+0000 7f36f4ae5640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 0x7f36dc071940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36e00af000 tx=0x7f36e00046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.071+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f36e805b8f0 con 0x7f36f0161e00 2024-05-17T22:06:21.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.271+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f36f01e9e90 con 0x7f36f0161e00 2024-05-17T22:06:21.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.559+0000 7f36ed7fa640 1 -- 172.21.15.83:0/3228037316 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v83) ==== 162+0+0 (secure 0 0 0) 0x7f36e805b1f0 con 0x7f36f0161e00 2024-05-17T22:06:21.553 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:06:21.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 msgr2=0x7f36dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 0x7f36dc071940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f36e00af000 tx=0x7f36e00046d0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 msgr2=0x7f36f01eb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f36f00cfce0 tx=0x7f36e800dc00 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 shutdown_connections 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36f0161e00 0x7f36f01eb9a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f36dc06f430 0x7f36dc071940 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f013ea90 0x7f36f01618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 --2- 172.21.15.83:0/3228037316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36d8003a60 0x7f36f015fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 >> 172.21.15.83:0/3228037316 conn(0x7f36f006cd90 msgr2=0x7f36f0146a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 shutdown_connections 2024-05-17T22:06:21.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.563+0000 7f36f6d49640 1 -- 172.21.15.83:0/3228037316 wait complete. 2024-05-17T22:06:21.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 Processor -- start 2024-05-17T22:06:21.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 -- start start 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efeb015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efeb00af3c0 0x7efeb00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00b16f0 con 0x7efeb00afd10 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7efeb00b0120 con 0x7efeb00af3c0 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb86fc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efeb015eec0 con 0x7efeb015df90 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb6c99640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efeb00afd10 0x7efeb0161580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41020/0 (socket says 172.21.15.83:41020) 2024-05-17T22:06:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb6c99640 1 -- 172.21.15.83:0/3936861214 learned_addr learned my addr 172.21.15.83:0/3936861214 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb6498640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efeb015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb5c97640 1 --2- 172.21.15.83:0/3936861214 >> v2:172.21.15.155:3300/0 conn(0x7efeb00af3c0 0x7efeb00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2514219327 0 0) 0x7efeb00b16f0 con 0x7efeb00afd10 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.739+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe9c003530 con 0x7efeb00afd10 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1180936149 0 0) 0x7efe9c003530 con 0x7efeb00afd10 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 msgr2=0x7efeb015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efeb015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 >> v2:172.21.15.155:3300/0 conn(0x7efeb00af3c0 msgr2=0x7efeb00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 --2- 172.21.15.83:0/3936861214 >> v2:172.21.15.155:3300/0 conn(0x7efeb00af3c0 0x7efeb00af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb00b6190 con 0x7efeb00afd10 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7efea8001a80 con 0x7efeb00afd10 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 >> v1:172.21.15.83:6789/0 conn(0x7efeb00afd10 legacy=0x7efeb0161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c003a10 0x7efe9c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efe9c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efeb015eec0 con 0x7efe9c004810 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c00ba80 con 0x7efe9c003a10 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efe9c00bc00 con 0x7efeb015df90 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb6c99640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efe9c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb6498640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c003a10 0x7efe9c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 -- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 msgr2=0x7efe9c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb015df90 0x7efe9c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 -- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c003a10 msgr2=0x7efe9c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c003a10 0x7efe9c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 -- 172.21.15.83:0/3936861214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c003530 con 0x7efe9c004810 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5c97640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efeb006e2c0 tx=0x7efea001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b67f809204759 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00b2070 con 0x7efe9c004810 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea00a5530 con 0x7efe9c004810 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.743+0000 7efeb5496640 1 -- 172.21.15.83:0/3936861214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00096b0 con 0x7efe9c004810 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- 172.21.15.83:0/3936861214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe9c004810 msgr2=0x7efe9c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 --2- 172.21.15.83:0/3936861214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe9c004810 0x7efe9c004c20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efeb006e2c0 tx=0x7efea001ca50 comp rx=0 tx=0).stop 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb6c99640 1 -- 172.21.15.83:0/3936861214 reap_dead start 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- 172.21.15.83:0/3936861214 shutdown_connections 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- 172.21.15.83:0/3936861214 >> 172.21.15.83:0/3936861214 conn(0x7efeb00d20b0 msgr2=0x7efeb00d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- 172.21.15.83:0/3936861214 shutdown_connections 2024-05-17T22:06:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- 172.21.15.83:0/3936861214 wait complete. 2024-05-17T22:06:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 Processor -- start 2024-05-17T22:06:21.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.747+0000 7efeb86fc640 1 -- start start 2024-05-17T22:06:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c004810 0x7efeb01f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 0x7efeb009e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b6a30 con 0x7efeb0151b70 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b6750 con 0x7efe9c004810 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efeb00b6d10 con 0x7efeb01e8d30 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38262/0 (socket says 172.21.15.83:38262) 2024-05-17T22:06:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 -- 172.21.15.83:0/2140519535 learned_addr learned my addr 172.21.15.83:0/2140519535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb6498640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c004810 0x7efeb01f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb6c99640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 0x7efeb009e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 -- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 msgr2=0x7efeb009e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 0x7efeb009e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 -- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c004810 msgr2=0x7efeb01f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c004810 0x7efeb01f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 -- 172.21.15.83:0/2140519535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01e9140 con 0x7efeb0151b70 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb6c99640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 0x7efeb009e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb5c97640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efea001c880 tx=0x7efe9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00b2040 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea000f000 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb01f48a0 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00ad680 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb009ddb0 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb01f4dc0 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efea00be030 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 0x7efe94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efeb6498640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 0x7efe94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.751+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7efea0128950 con 0x7efeb0151b70 2024-05-17T22:06:21.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.755+0000 7efeb6498640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 0x7efe94071910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efeac009160 tx=0x7efeac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:21.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.755+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efeb01f4dc0 con 0x7efeb0151b70 2024-05-17T22:06:21.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.951+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb00b5e20 con 0x7efeb0151b70 2024-05-17T22:06:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.951+0000 7efea6ffd640 1 -- 172.21.15.83:0/2140519535 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5180 (secure 0 0 0) 0x7efeb01e9140 con 0x7efeb0151b70 2024-05-17T22:06:21.945 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:21.945 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:21.569956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7704":{"gid":7704,"name":"a","rank":0,"incarnation":84,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6836/563832419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":563832419},{"type":"v1","addr":"172.21.15.83:6836","nonce":563832419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 84 2024-05-17T22:06:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 msgr2=0x7efe94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 0x7efe94071910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efeac009160 tx=0x7efeac008040 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 msgr2=0x7efeb01e87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efea001c880 tx=0x7efe9c005840 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 shutdown_connections 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efeb01e8d30 0x7efeb009e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb0151b70 0x7efeb01e87f0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efe9406f400 0x7efe94071910 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 --2- 172.21.15.83:0/2140519535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe9c004810 0x7efeb01f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 >> 172.21.15.83:0/2140519535 conn(0x7efeb00d20b0 msgr2=0x7efeb0160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 shutdown_connections 2024-05-17T22:06:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:21.955+0000 7efeb86fc640 1 -- 172.21.15.83:0/2140519535 wait complete. 2024-05-17T22:06:22.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:05:26.778012+0000', 'modified': '2024-05-17T22:06:21.569956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7704': {'gid': 7704, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6836/563832419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 563832419}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 563832419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:06:22.370 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7704:up:replay 2024-05-17T22:06:23.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.539+0000 7f9d0a080640 1 Processor -- start 2024-05-17T22:06:23.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.539+0000 7f9d0a080640 1 -- start start 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d040d1040 0x7f9d04144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9d0415e080 0x7f9d04072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d0415e490 0x7f9d04073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9d0415ea40 con 0x7f9d0415e490 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9d0415ed20 con 0x7f9d0415e080 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d0a080640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0415f000 con 0x7f9d040d1040 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d0415e490 0x7f9d04073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d0415e490 0x7f9d04073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38266/0 (socket says 172.21.15.83:38266) 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 -- 172.21.15.83:0/3184775275 learned_addr learned my addr 172.21.15.83:0/3184775275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9cfbfff640 1 --2- 172.21.15.83:0/3184775275 >> v2:172.21.15.155:3300/0 conn(0x7f9d0415e080 0x7f9d04072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d037fe640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d040d1040 0x7f9d04144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 -- 172.21.15.83:0/3184775275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d040d1040 msgr2=0x7f9d04144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d040d1040 0x7f9d04144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 -- 172.21.15.83:0/3184775275 >> v2:172.21.15.155:3300/0 conn(0x7f9d0415e080 msgr2=0x7f9d04072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 --2- 172.21.15.83:0/3184775275 >> v2:172.21.15.155:3300/0 conn(0x7f9d0415e080 0x7f9d04072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.543+0000 7f9d03fff640 1 -- 172.21.15.83:0/3184775275 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0406ef30 con 0x7f9d0415e490 2024-05-17T22:06:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d03fff640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e490 0x7f9d04073370 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9d040b6c80 tx=0x7f9cfc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ddd574be5d5eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d02ffd640 1 -- 172.21.15.83:0/3184775275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0a5780 con 0x7f9d0415e490 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d02ffd640 1 -- 172.21.15.83:0/3184775275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc0a5920 con 0x7f9d0415e490 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d02ffd640 1 -- 172.21.15.83:0/3184775275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0a5cf0 con 0x7f9d0415e490 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- 172.21.15.83:0/3184775275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e490 msgr2=0x7f9d04073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e490 0x7f9d04073370 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9d040b6c80 tx=0x7f9cfc01c9f0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- 172.21.15.83:0/3184775275 shutdown_connections 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e490 0x7f9d04073370 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 --2- 172.21.15.83:0/3184775275 >> v2:172.21.15.155:3300/0 conn(0x7f9d0415e080 0x7f9d04072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 --2- 172.21.15.83:0/3184775275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d040d1040 0x7f9d04144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- 172.21.15.83:0/3184775275 >> 172.21.15.83:0/3184775275 conn(0x7f9d0406cd90 msgr2=0x7f9d040b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- 172.21.15.83:0/3184775275 shutdown_connections 2024-05-17T22:06:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- 172.21.15.83:0/3184775275 wait complete. 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 Processor -- start 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.547+0000 7f9d0a080640 1 -- start start 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040d1040 0x7f9d0415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d0415e490 0x7f9d0413b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041622a0 con 0x7f9d0415e080 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04162120 con 0x7f9d040d1040 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d04162420 con 0x7f9d0415e490 2024-05-17T22:06:23.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38280/0 (socket says 172.21.15.83:38280) 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 -- 172.21.15.83:0/1541649924 learned_addr learned my addr 172.21.15.83:0/1541649924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d03fff640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d0415e490 0x7f9d0413b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d037fe640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040d1040 0x7f9d0415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 -- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d0415e490 msgr2=0x7f9d0413b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d0415e490 0x7f9d0413b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 -- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040d1040 msgr2=0x7f9d0415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040d1040 0x7f9d0415d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 -- 172.21.15.83:0/1541649924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d04146f80 con 0x7f9d0415e080 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9cfbfff640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00d800 tx=0x7f9cec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00b6c0 con 0x7f9d0415e080 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cec019070 con 0x7f9d0415e080 2024-05-17T22:06:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d04147260 con 0x7f9d0415e080 2024-05-17T22:06:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00ba80 con 0x7f9d0415e080 2024-05-17T22:06:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d040b5ff0 con 0x7f9d0415e080 2024-05-17T22:06:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d04144080 con 0x7f9d0415e080 2024-05-17T22:06:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cec00bc20 con 0x7f9d0415e080 2024-05-17T22:06:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 0x7f9ce4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f9cec0900b0 con 0x7f9d0415e080 2024-05-17T22:06:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.551+0000 7f9d037fe640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 0x7f9ce4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:23.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.555+0000 7f9d037fe640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 0x7f9ce4071940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b990 tx=0x7f9cf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:23.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.555+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9cec05d0b0 con 0x7f9d0415e080 2024-05-17T22:06:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.751+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d0415f2e0 con 0x7f9d0415e080 2024-05-17T22:06:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.751+0000 7f9d00ff9640 1 -- 172.21.15.83:0/1541649924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5180 (secure 0 0 0) 0x7f9cec05c9b0 con 0x7f9d0415e080 2024-05-17T22:06:23.745 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:23.746 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:22.581621+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7704":{"gid":7704,"name":"a","rank":0,"incarnation":84,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6836/563832419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":563832419},{"type":"v1","addr":"172.21.15.83:6836","nonce":563832419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:23.746 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 86 2024-05-17T22:06:23.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 msgr2=0x7f9ce4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 0x7f9ce4071940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b990 tx=0x7f9cf4008040 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 msgr2=0x7f9d04167e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00d800 tx=0x7f9cec00dcc0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 shutdown_connections 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d0415e490 0x7f9d0413b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0415e080 0x7f9d04167e30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9ce406f430 0x7f9ce4071940 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 --2- 172.21.15.83:0/1541649924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d040d1040 0x7f9d0415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 >> 172.21.15.83:0/1541649924 conn(0x7f9d0406cd90 msgr2=0x7f9d04160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 shutdown_connections 2024-05-17T22:06:23.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:23.755+0000 7f9d0a080640 1 -- 172.21.15.83:0/1541649924 wait complete. 2024-05-17T22:06:24.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:05:26.778012+0000', 'modified': '2024-05-17T22:06:22.581621+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7704': {'gid': 7704, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6836/563832419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 563832419}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 563832419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:06:24.172 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7704:up:rejoin 2024-05-17T22:06:25.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 Processor -- start 2024-05-17T22:06:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 -- start start 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe2fc1461c0 0x7fe2fc15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2fc06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc1465d0 con 0x7fe2fc070510 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe2fc06ef30 con 0x7fe2fc1461c0 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe300817640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc06f210 con 0x7fe2fc15e6b0 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe2fc070510 0x7fe2fc1605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41034/0 (socket says 172.21.15.83:41034) 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 -- 172.21.15.83:0/1735861324 learned_addr learned my addr 172.21.15.83:0/1735861324 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1735861324 >> v2:172.21.15.155:3300/0 conn(0x7fe2fc1461c0 0x7fe2fc15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2fc06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3807359896 0 0) 0x7fe2fc1465d0 con 0x7fe2fc070510 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e4003530 con 0x7fe2fc070510 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1081439881 0 0) 0x7fe2e4003530 con 0x7fe2fc070510 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 msgr2=0x7fe2fc06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2fc06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 >> v2:172.21.15.155:3300/0 conn(0x7fe2fc1461c0 msgr2=0x7fe2fc15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 --2- 172.21.15.83:0/1735861324 >> v2:172.21.15.155:3300/0 conn(0x7fe2fc1461c0 0x7fe2fc15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc070920 con 0x7fe2fc070510 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe2f4001a80 con 0x7fe2fc070510 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 >> v1:172.21.15.83:6789/0 conn(0x7fe2fc070510 legacy=0x7fe2fc1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2e4003a10 0x7fe2e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e4004d50 0x7fe2e4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc06ef30 con 0x7fe2e4004d50 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc06f210 con 0x7fe2e4003a10 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e400dcc0 con 0x7fe2fc15e6b0 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e4004d50 0x7fe2e4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2e4003a10 0x7fe2e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 -- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 msgr2=0x7fe2e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 -- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2e4003a10 msgr2=0x7fe2e4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2e4003a10 0x7fe2e4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.343+0000 7fe2fad9d640 1 -- 172.21.15.83:0/1735861324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e4003530 con 0x7fe2e4004d50 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc15e6b0 0x7fe2e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2e4003a10 0x7fe2e4003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2fad9d640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e4004d50 0x7fe2e4005160 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400a8f0 tx=0x7fe2f401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c42268765134cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400a4c0 con 0x7fe2e4004d50 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f40ad650 con 0x7fe2e4004d50 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2f959a640 1 -- 172.21.15.83:0/1735861324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40ada40 con 0x7fe2e4004d50 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 -- 172.21.15.83:0/1735861324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e4004d50 msgr2=0x7fe2e4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 --2- 172.21.15.83:0/1735861324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2e4004d50 0x7fe2e4005160 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400a8f0 tx=0x7fe2f401d380 comp rx=0 tx=0).stop 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe2fad9d640 1 -- 172.21.15.83:0/1735861324 reap_dead start 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 -- 172.21.15.83:0/1735861324 shutdown_connections 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 -- 172.21.15.83:0/1735861324 >> 172.21.15.83:0/1735861324 conn(0x7fe2fc06cd90 msgr2=0x7fe2fc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 -- 172.21.15.83:0/1735861324 shutdown_connections 2024-05-17T22:06:25.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.347+0000 7fe300817640 1 -- 172.21.15.83:0/1735861324 wait complete. 2024-05-17T22:06:25.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 Processor -- start 2024-05-17T22:06:25.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- start start 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2fc0afd10 0x7fe2fc13dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc140eb0 0x7fe2fc1412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc06fab0 con 0x7fe2fc15c100 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc06f7d0 con 0x7fe2fc0afd10 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc160cb0 con 0x7fe2fc140eb0 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38294/0 (socket says 172.21.15.83:38294) 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 -- 172.21.15.83:0/1016371873 learned_addr learned my addr 172.21.15.83:0/1016371873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2fad9d640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc140eb0 0x7fe2fc1412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2fc0afd10 0x7fe2fc13dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 -- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc140eb0 msgr2=0x7fe2fc1412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc140eb0 0x7fe2fc1412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 -- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2fc0afd10 msgr2=0x7fe2fc13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2fc0afd10 0x7fe2fc13dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 -- 172.21.15.83:0/1016371873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc15c510 con 0x7fe2fc15c100 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2f9d9b640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00108d0 tx=0x7fe2f0010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000f040 con 0x7fe2fc15c100 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0004220 con 0x7fe2fc15c100 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc141800 con 0x7fe2fc15c100 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000e960 con 0x7fe2fc15c100 2024-05-17T22:06:25.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.351+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc1f6520 con 0x7fe2fc15c100 2024-05-17T22:06:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc15e170 con 0x7fe2fc15c100 2024-05-17T22:06:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2f000eb00 con 0x7fe2fc15c100 2024-05-17T22:06:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe2eaffd640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7fe2f008f390 con 0x7fe2fc15c100 2024-05-17T22:06:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.355+0000 7fe2fa59c640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 0x7fe2dc071910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc144290 tx=0x7fe2ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.359+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe2fc141800 con 0x7fe2fc15c100 2024-05-17T22:06:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.555+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2fc0b41a0 con 0x7fe2fc15c100 2024-05-17T22:06:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.555+0000 7fe2eaffd640 1 -- 172.21.15.83:0/1016371873 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5187 (secure 0 0 0) 0x7fe2fc15c510 con 0x7fe2fc15c100 2024-05-17T22:06:25.549 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:25.550 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:24.585879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7704":{"gid":7704,"name":"a","rank":0,"incarnation":84,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6836/563832419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":563832419},{"type":"v1","addr":"172.21.15.83:6836","nonce":563832419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7704,"qdb_cluster":[7704]},"id":7}]} 2024-05-17T22:06:25.550 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 87 2024-05-17T22:06:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 msgr2=0x7fe2dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 0x7fe2dc071910 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc144290 tx=0x7fe2ec008040 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 msgr2=0x7fe2fc0b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00108d0 tx=0x7fe2f0010d90 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 shutdown_connections 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2fc140eb0 0x7fe2fc1412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2fc15c100 0x7fe2fc0b6630 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 --2- 172.21.15.83:0/1016371873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2fc0afd10 0x7fe2fc13dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 >> 172.21.15.83:0/1016371873 conn(0x7fe2fc06cd90 msgr2=0x7fe2fc15f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 shutdown_connections 2024-05-17T22:06:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:25.559+0000 7fe300817640 1 -- 172.21.15.83:0/1016371873 wait complete. 2024-05-17T22:06:25.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:05:26.778012+0000', 'modified': '2024-05-17T22:06:24.585879+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7704': {'gid': 7704, 'name': 'a', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6836/563832419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 563832419}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 563832419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7704, 'qdb_cluster': [7704]} 2024-05-17T22:06:25.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:06:25.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-17T22:06:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 Processor -- start 2024-05-17T22:06:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 -- start start 2024-05-17T22:06:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84080d17d0 0x7f84080d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84080cf380 0x7f840806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84080b5020 con 0x7f84080d2120 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f84080cdee0 con 0x7f84080d17d0 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f8410058640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f840809cde0 con 0x7f84080cf380 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84080d17d0 0x7f84080d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84080d17d0 0x7f84080d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38306/0 (socket says 172.21.15.83:38306) 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 -- 172.21.15.83:0/1704405260 learned_addr learned my addr 172.21.15.83:0/1704405260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840e5f5640 1 --2- 172.21.15.83:0/1704405260 >> v2:172.21.15.155:3300/0 conn(0x7f84080cf380 0x7f840806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 -- 172.21.15.83:0/1704405260 >> v2:172.21.15.155:3300/0 conn(0x7f84080cf380 msgr2=0x7f840806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 --2- 172.21.15.83:0/1704405260 >> v2:172.21.15.155:3300/0 conn(0x7f84080cf380 0x7f840806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 -- 172.21.15.83:0/1704405260 >> v1:172.21.15.155:6789/0 conn(0x7f84080d2120 legacy=0x7f840806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.147+0000 7f840ddf4640 1 -- 172.21.15.83:0/1704405260 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840809c540 con 0x7f84080d17d0 2024-05-17T22:06:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f840ddf4640 1 --2- 172.21.15.83:0/1704405260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080d1be0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f84080cdcb0 tx=0x7f83f801c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87660745d9390104 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f840cdf2640 1 -- 172.21.15.83:0/1704405260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80b0070 con 0x7f84080d17d0 2024-05-17T22:06:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f840cdf2640 1 -- 172.21.15.83:0/1704405260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f80ab600 con 0x7f84080d17d0 2024-05-17T22:06:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f840cdf2640 1 -- 172.21.15.83:0/1704405260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80ab9d0 con 0x7f84080d17d0 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 -- 172.21.15.83:0/1704405260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 msgr2=0x7f84080d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 --2- 172.21.15.83:0/1704405260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080d1be0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f84080cdcb0 tx=0x7f83f801c1c0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 -- 172.21.15.83:0/1704405260 shutdown_connections 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 --2- 172.21.15.83:0/1704405260 >> v2:172.21.15.155:3300/0 conn(0x7f84080cf380 0x7f840806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 --2- 172.21.15.83:0/1704405260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080d1be0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 -- 172.21.15.83:0/1704405260 >> 172.21.15.83:0/1704405260 conn(0x7f84080d36d0 msgr2=0x7f84080d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 -- 172.21.15.83:0/1704405260 shutdown_connections 2024-05-17T22:06:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.151+0000 7f8410058640 1 -- 172.21.15.83:0/1704405260 wait complete. 2024-05-17T22:06:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 Processor -- start 2024-05-17T22:06:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- start start 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 0x7f8408159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080d2120 0x7f8408158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84080d26b0 con 0x7f84080d17d0 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84080d2530 con 0x7f84080cf380 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f840816c980 con 0x7f84080d2120 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38310/0 (socket says 172.21.15.83:38310) 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 -- 172.21.15.83:0/2789859461 learned_addr learned my addr 172.21.15.83:0/2789859461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840ddf4640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 0x7f8408159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840e5f5640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080d2120 0x7f8408158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 -- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080d2120 msgr2=0x7f8408158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080d2120 0x7f8408158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 -- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 msgr2=0x7f8408159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 0x7f8408159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 -- 172.21.15.83:0/2789859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8408157780 con 0x7f84080d17d0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840ddf4640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 0x7f8408159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f840d5f3640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f840400b410 tx=0x7f840400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f83fe7fc640 1 -- 172.21.15.83:0/2789859461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404015030 con 0x7f84080d17d0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f83fe7fc640 1 -- 172.21.15.83:0/2789859461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404004020 con 0x7f84080d17d0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f83fe7fc640 1 -- 172.21.15.83:0/2789859461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004450 con 0x7f84080d17d0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8408157a60 con 0x7f84080d17d0 2024-05-17T22:06:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.155+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8408158020 con 0x7f84080d17d0 2024-05-17T22:06:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f83fe7fc640 1 -- 172.21.15.83:0/2789859461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8404018070 con 0x7f84080d17d0 2024-05-17T22:06:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f83fe7fc640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 0x7f83ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f83fe7fc640 1 -- 172.21.15.83:0/2789859461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f840408efa0 con 0x7f84080d17d0 2024-05-17T22:06:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f840ddf4640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 0x7f83ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f840ddf4640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 0x7f83ec071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ad000 tx=0x7f83f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f84172d2000 1 Processor -- start 2024-05-17T22:06:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.159+0000 7f84172d2000 1 -- start start 2024-05-17T22:06:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c2123600 0x5634c2273df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634c2111ca0 con 0x5634c210cd30 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c2084110 con 0x5634c212dfa0 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5634c2073cb0 con 0x5634c2123600 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38318/0 (socket says 172.21.15.83:38318) 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 -- 172.21.15.83:0/1001674216 learned_addr learned my addr 172.21.15.83:0/1001674216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 -- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c2123600 msgr2=0x5634c2273df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c2123600 0x5634c2273df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 -- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 msgr2=0x5634c22738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840ddf4640 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 -- 172.21.15.83:0/1001674216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c2038370 con 0x5634c210cd30 2024-05-17T22:06:26.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f840e5f5640 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ef0 tx=0x7f8400007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427ec495973e9e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f83eb7fe640 1 -- 172.21.15.83:0/1001674216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400013780 con 0x5634c210cd30 2024-05-17T22:06:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f83eb7fe640 1 -- 172.21.15.83:0/1001674216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400019070 con 0x5634c210cd30 2024-05-17T22:06:26.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f83eb7fe640 1 -- 172.21.15.83:0/1001674216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400013b60 con 0x5634c210cd30 2024-05-17T22:06:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- 172.21.15.83:0/1001674216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 msgr2=0x5634c2274330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ef0 tx=0x7f8400007700 comp rx=0 tx=0).stop 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- 172.21.15.83:0/1001674216 shutdown_connections 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2274330 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c2123600 0x5634c2273df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 --2- 172.21.15.83:0/1001674216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- 172.21.15.83:0/1001674216 >> 172.21.15.83:0/1001674216 conn(0x5634c1d35460 msgr2=0x5634c2228e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- 172.21.15.83:0/1001674216 shutdown_connections 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- 172.21.15.83:0/1001674216 wait complete. 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 -- start start 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.163+0000 7f84172d2000 1 Processor -- start 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c212dfa0 0x5634c2235460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634c202fd40 con 0x5634c210cd30 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c2077cf0 con 0x5634c2123600 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5634c201ce90 con 0x5634c212dfa0 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 10 client.0 _ll_register_callbacks cb 0x5634c2051fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38322/0 (socket says 172.21.15.83:38322) 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 -- 172.21.15.83:0/914613407 learned_addr learned my addr 172.21.15.83:0/914613407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840d5f3640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840e5f5640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c212dfa0 0x5634c2235460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 -- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c212dfa0 msgr2=0x5634c2235460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c212dfa0 0x5634c2235460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 -- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 msgr2=0x5634c22307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22307d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 -- 172.21.15.83:0/914613407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c20426e0 con 0x5634c210cd30 2024-05-17T22:06:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840d5f3640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840ddf4640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ae040 tx=0x7f83f80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f456cd4b0c2e6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80b0040 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f83f80b6620 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400019070 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f83f80cc050 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 10 client.7767 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x5634c2032690 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400019070 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f83f80cf020 con 0x5634c210cd30 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 10 client.7767 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 10 client.7767 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 20 client.7767 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d1a7d0 0x5634c227a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 4 client.7767 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c200fa00 con 0x5634c1d1a7d0 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840d5f3640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d1a7d0 0x5634c227a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 10 client.7767 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f840d5f3640 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d1a7d0 0x5634c227a920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ee01e91fca896a server_cookie=34abc34e1e9817df in_seq=0 out_seq=0 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 1 -- 172.21.15.83:0/914613407 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634c200fa00 con 0x5634c1d1a7d0 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 10 client.7767 handle_command_reply: tid=0 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 client.7767 shutdown 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 msgr2=0x5634c2230290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ae040 tx=0x7f83f80046d0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 shutdown_connections 2024-05-17T22:06:26.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d1a7d0 0x5634c227a920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c212dfa0 0x5634c2235460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f83e8ff9640 0 client.7767 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 --2- 172.21.15.83:0/914613407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c210cd30 0x5634c2230290 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 >> 172.21.15.83:0/914613407 conn(0x5634c1d35460 msgr2=0x5634c2228e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 shutdown_connections 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 1 -- 172.21.15.83:0/914613407 wait complete. 2024-05-17T22:06:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.167+0000 7f84172d2000 20 client.7767 trim_cache size 0 max 16384 2024-05-17T22:06:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 Processor -- start 2024-05-17T22:06:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 -- start start 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84082046c0 con 0x5634c212dfa0 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f840815b840 con 0x5634c2123600 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8408204070 con 0x5634c210cd30 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f840e5f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f840e5f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38324/0 (socket says 172.21.15.83:38324) 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.187+0000 7f840e5f5640 1 -- 172.21.15.83:0/616504647 learned_addr learned my addr 172.21.15.83:0/616504647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840d5f3640 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 -- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 msgr2=0x5634c22e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 -- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 msgr2=0x5634c22e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 -- 172.21.15.83:0/616504647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c20facb0 con 0x5634c212dfa0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840d5f3640 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840e5f5640 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8400015000 tx=0x7f8400013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28ee3f11bf4ea99 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f83eaffd640 1 -- 172.21.15.83:0/616504647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004040 con 0x5634c212dfa0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f83eaffd640 1 -- 172.21.15.83:0/616504647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400019040 con 0x5634c212dfa0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f83eaffd640 1 -- 172.21.15.83:0/616504647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004420 con 0x5634c212dfa0 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- 172.21.15.83:0/616504647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 msgr2=0x5634c22e2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8400015000 tx=0x7f8400013000 comp rx=0 tx=0).stop 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- 172.21.15.83:0/616504647 shutdown_connections 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c212dfa0 0x5634c22e2880 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c2123600 0x5634c22e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- 172.21.15.83:0/616504647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- 172.21.15.83:0/616504647 >> 172.21.15.83:0/616504647 conn(0x5634c1d35460 msgr2=0x5634c2289ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- 172.21.15.83:0/616504647 shutdown_connections 2024-05-17T22:06:26.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- 172.21.15.83:0/616504647 wait complete. 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- start start 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 Processor -- start 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634c2077000 con 0x5634c2123600 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c2233c20 con 0x5634c212dfa0 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5634c2082ee0 con 0x5634c210cd30 2024-05-17T22:06:26.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f84172d2000 10 client.0 _ll_register_callbacks cb 0x7f83f80ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840d5f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840d5f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38338/0 (socket says 172.21.15.83:38338) 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.191+0000 7f840d5f3640 1 -- 172.21.15.83:0/256649382 learned_addr learned my addr 172.21.15.83:0/256649382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840ddf4640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840e5f5640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 -- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 msgr2=0x5634c22e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 -- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 msgr2=0x5634c22e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22e68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 -- 172.21.15.83:0/256649382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c20334f0 con 0x5634c2123600 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840e5f5640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840ddf4640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840d5f3640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8404057710 tx=0x7f840400b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2e2504824acf04 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040644f0 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f840405a8f0 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400019040 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f840407b050 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 10 client.7773 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x5634c203d4f0 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400019040 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f8404022060 con 0x5634c2123600 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 10 client.7773 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 10 client.7773 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 20 client.7773 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d08830 0x5634c222a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 4 client.7773 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "status"}) -- 0x5634c2086060 con 0x5634c1d08830 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840ddf4640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d08830 0x5634c222a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 10 client.7773 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f840ddf4640 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d08830 0x5634c222a8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774f19ad7702f747 server_cookie=2055c23d3b3107c in_seq=0 out_seq=0 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 1 -- 172.21.15.83:0/256649382 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5634c2086060 con 0x5634c1d08830 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 10 client.7773 handle_command_reply: tid=0 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 client.7773 shutdown 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 msgr2=0x5634c22e6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8404057710 tx=0x7f840400b360 comp rx=0 tx=0).stop 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 shutdown_connections 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x5634c1d08830 0x5634c222a8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c212dfa0 0x5634c22e68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634c2123600 0x5634c22e6380 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f83e8ff9640 0 client.7773 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 --2- 172.21.15.83:0/256649382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5634c210cd30 0x5634c22e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 >> 172.21.15.83:0/256649382 conn(0x5634c1d35460 msgr2=0x5634c2273180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 shutdown_connections 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 1 -- 172.21.15.83:0/256649382 wait complete. 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f84172d2000 20 client.7773 trim_cache size 0 max 16384 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7704, 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:06:26.193 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.622653626, 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 87, 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 97, 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 97, 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 5.5446245440000004 2024-05-17T22:06:26.194 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 msgr2=0x7f83ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 0x7f83ec071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83f80ad000 tx=0x7f83f8008040 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 msgr2=0x7f84080cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f840400b410 tx=0x7f840400b8d0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 shutdown_connections 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080d2120 0x7f8408158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84080d17d0 0x7f84080cb540 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f83ec06f400 0x7f83ec071910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 --2- 172.21.15.83:0/2789859461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84080cf380 0x7f8408159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 >> 172.21.15.83:0/2789859461 conn(0x7f84080d36d0 msgr2=0x7f840806d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 shutdown_connections 2024-05-17T22:06:26.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.195+0000 7f8410058640 1 -- 172.21.15.83:0/2789859461 wait complete. 2024-05-17T22:06:26.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-17T22:06:26.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:06:26.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-17T22:06:26.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 Processor -- start 2024-05-17T22:06:26.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 -- start start 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe2cc15cb20 0x7fe2cc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc0af4c0 0x7fe2cc06cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2cc06d300 con 0x7fe2cc06c790 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe2cc06d0e0 con 0x7fe2cc15cb20 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2d0e16640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc071630 con 0x7fe2cc0af4c0 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe2cc06c790 0x7fe2cc0aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41046/0 (socket says 172.21.15.83:41046) 2024-05-17T22:06:26.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/378218760 learned_addr learned my addr 172.21.15.83:0/378218760 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc0af4c0 0x7fe2cc06cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/378218760 >> v2:172.21.15.155:3300/0 conn(0x7fe2cc15cb20 0x7fe2cc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375295764 0 0) 0x7fe2cc06d300 con 0x7fe2cc06c790 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2b8003530 con 0x7fe2cc06c790 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1513071177 0 0) 0x7fe2b8003530 con 0x7fe2cc06c790 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc0af4c0 msgr2=0x7fe2cc06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc0af4c0 0x7fe2cc06cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 >> v2:172.21.15.155:3300/0 conn(0x7fe2cc15cb20 msgr2=0x7fe2cc0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 --2- 172.21.15.83:0/378218760 >> v2:172.21.15.155:3300/0 conn(0x7fe2cc15cb20 0x7fe2cc0710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc06d5e0 con 0x7fe2cc06c790 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe2b4001700 con 0x7fe2cc06c790 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 >> v1:172.21.15.83:6789/0 conn(0x7fe2cc06c790 legacy=0x7fe2cc0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2b8003a10 0x7fe2b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8004810 0x7fe2b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8005160 0x7fe2b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc06d0e0 con 0x7fe2b8004810 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc071630 con 0x7fe2b8005160 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b800bf10 con 0x7fe2b8003a10 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8004810 0x7fe2b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2b8003a10 0x7fe2b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8005160 0x7fe2b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2b8003a10 msgr2=0x7fe2b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2b8003a10 0x7fe2b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8005160 msgr2=0x7fe2b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8005160 0x7fe2b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/378218760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b8003530 con 0x7fe2b8004810 2024-05-17T22:06:26.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.371+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8005160 0x7fe2b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8004810 0x7fe2b8004c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c290 tx=0x7fe2b401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f6511fbcb72754 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4007d60 con 0x7fe2b8004810 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b4005ae0 con 0x7fe2b8004810 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2c9d9b640 1 -- 172.21.15.83:0/378218760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4007070 con 0x7fe2b8004810 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 -- 172.21.15.83:0/378218760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8004810 msgr2=0x7fe2b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/378218760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8004810 0x7fe2b8004c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c290 tx=0x7fe2b401f3d0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2cad9d640 1 -- 172.21.15.83:0/378218760 reap_dead start 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 -- 172.21.15.83:0/378218760 shutdown_connections 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 -- 172.21.15.83:0/378218760 >> 172.21.15.83:0/378218760 conn(0x7fe2cc0b3ad0 msgr2=0x7fe2cc0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 -- 172.21.15.83:0/378218760 shutdown_connections 2024-05-17T22:06:26.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.375+0000 7fe2d0e16640 1 -- 172.21.15.83:0/378218760 wait complete. 2024-05-17T22:06:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 Processor -- start 2024-05-17T22:06:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- start start 2024-05-17T22:06:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8004810 0x7fe2cc155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc09ede0 0x7fe2cc151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc15cf30 con 0x7fe2cc152660 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc070bd0 con 0x7fe2b8004810 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc06dbd0 con 0x7fe2cc09ede0 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38354/0 (socket says 172.21.15.83:38354) 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/1023005403 learned_addr learned my addr 172.21.15.83:0/1023005403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8004810 0x7fe2cc155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc09ede0 0x7fe2cc151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc09ede0 msgr2=0x7fe2cc151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc09ede0 0x7fe2cc151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8004810 msgr2=0x7fe2cc155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8004810 0x7fe2cc155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/1023005403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc156330 con 0x7fe2cc152660 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c290 tx=0x7fe2b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c37fe640 1 -- 172.21.15.83:0/1023005403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40047f0 con 0x7fe2cc152660 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c37fe640 1 -- 172.21.15.83:0/1023005403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b4004990 con 0x7fe2cc152660 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2c37fe640 1 -- 172.21.15.83:0/1023005403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004d40 con 0x7fe2cc152660 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc156610 con 0x7fe2cc152660 2024-05-17T22:06:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.379+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc09c2e0 con 0x7fe2cc152660 2024-05-17T22:06:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2c37fe640 1 -- 172.21.15.83:0/1023005403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2b4007070 con 0x7fe2cc152660 2024-05-17T22:06:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2c37fe640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 0x7fe2ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 0x7fe2ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2c37fe640 1 -- 172.21.15.83:0/1023005403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7fe2b412c860 con 0x7fe2cc152660 2024-05-17T22:06:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 0x7fe2ac071910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009fd0 tx=0x7fe2c400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2d8090000 1 Processor -- start 2024-05-17T22:06:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.383+0000 7fe2d8090000 1 -- start start 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 Processor -- start 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604588db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c6045892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c604396220 con 0x55c60441cdf0 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c604385dc0 con 0x55c604432f60 2024-05-17T22:06:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c604389e00 con 0x55c604441240 2024-05-17T22:06:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 -- start start 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38358/0 (socket says 172.21.15.83:38358) 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 -- 172.21.15.83:0/3159308692 learned_addr learned my addr 172.21.15.83:0/3159308692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604588db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c6045892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 -- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 msgr2=0x55c604588db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604588db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 -- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 msgr2=0x55c6045892f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c6045892f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 -- 172.21.15.83:0/3159308692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c604361bf0 con 0x55c60441cdf0 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604588db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c6045892f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0b48c0 tx=0x7fe2bc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b6fdc22cc2542c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2abfff640 1 -- 172.21.15.83:0/3159308692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc019020 con 0x55c60441cdf0 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2abfff640 1 -- 172.21.15.83:0/3159308692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc009780 con 0x55c60441cdf0 2024-05-17T22:06:26.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2abfff640 1 -- 172.21.15.83:0/3159308692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00f040 con 0x55c60441cdf0 2024-05-17T22:06:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 0x7f53d40b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53d40d2070 0x7f53d406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d40d3350 con 0x7f53d40d1c60 2024-05-17T22:06:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f53d409c8c0 con 0x7f53d40d2070 2024-05-17T22:06:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53db5ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53d409cba0 con 0x7f53d40d0bc0 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- 172.21.15.83:0/3159308692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 msgr2=0x55c604589830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0b48c0 tx=0x7fe2bc009190 comp rx=0 tx=0).stop 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- 172.21.15.83:0/3159308692 shutdown_connections 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c604589830 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c6045892f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 --2- 172.21.15.83:0/3159308692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604588db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- 172.21.15.83:0/3159308692 >> 172.21.15.83:0/3159308692 conn(0x55c604047810 msgr2=0x55c6045390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- 172.21.15.83:0/3159308692 shutdown_connections 2024-05-17T22:06:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9349640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 0x7f53d40b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- 172.21.15.83:0/3159308692 wait complete. 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d8b48640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f53d40d1c60 0x7f53d406d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:51592/0 (socket says 172.21.15.83:51592) 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d8b48640 1 -- 172.21.15.83:0/2871680562 learned_addr learned my addr 172.21.15.83:0/2871680562 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9b4a640 1 --2- 172.21.15.83:0/2871680562 >> v2:172.21.15.155:3300/0 conn(0x7f53d40d2070 0x7f53d406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9349640 1 -- 172.21.15.83:0/2871680562 >> v2:172.21.15.155:3300/0 conn(0x7f53d40d2070 msgr2=0x7f53d406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9349640 1 --2- 172.21.15.83:0/2871680562 >> v2:172.21.15.155:3300/0 conn(0x7f53d40d2070 0x7f53d406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9349640 1 -- 172.21.15.83:0/2871680562 >> v1:172.21.15.155:6789/0 conn(0x7f53d40d1c60 legacy=0x7f53d406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53cbfff640 1 -- 172.21.15.83:0/2871680562 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779503836 0 0) 0x7f53c4001120 con 0x7f53d40d1c60 2024-05-17T22:06:26.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7f53d9349640 1 -- 172.21.15.83:0/2871680562 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d406eb50 con 0x7f53d40d0bc0 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 -- start start 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.387+0000 7fe2d8090000 1 Processor -- start 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 0x55c60458f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 0x55c60458faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c60430ef30 con 0x55c604432f60 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c604382740 con 0x55c604441240 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38368/0 (socket says 172.21.15.83:38368) 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/187660906 learned_addr learned my addr 172.21.15.83:0/187660906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c60430ccc0 con 0x55c60441cdf0 2024-05-17T22:06:26.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 0x55c60458f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 0x55c60458faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 msgr2=0x55c60458f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 0x55c60458f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 msgr2=0x55c60458faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 0x55c60458faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/187660906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60433f460 con 0x55c604432f60 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 0x55c60458f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 0x55c60458faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:26.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53d9349640 1 --2- 172.21.15.83:0/2871680562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 0x7f53d40b8590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097a0 tx=0x7f53d001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804574519e9ba32d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40f61d0 tx=0x7fe2b40f8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256951c2354891c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 10 client.0 _ll_register_callbacks cb 0x55c6045266a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2bc009780 con 0x55c604432f60 2024-05-17T22:06:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 1 -- 172.21.15.83:0/187660906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40f8ff0 con 0x55c604432f60 2024-05-17T22:06:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 1 -- 172.21.15.83:0/187660906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7fe2b4007360 con 0x55c604432f60 2024-05-17T22:06:26.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53cbfff640 1 -- 172.21.15.83:0/2871680562 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00b0070 con 0x7f53d40d0bc0 2024-05-17T22:06:26.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53cbfff640 1 -- 172.21.15.83:0/2871680562 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d00ab600 con 0x7f53d40d0bc0 2024-05-17T22:06:26.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53cbfff640 1 -- 172.21.15.83:0/2871680562 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00ab9d0 con 0x7f53d40d0bc0 2024-05-17T22:06:26.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 1 -- 172.21.15.83:0/187660906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe2b40fd3a0 con 0x55c604432f60 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 10 client.7785 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=87}) -- 0x55c6043449e0 con 0x55c604432f60 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 1 -- 172.21.15.83:0/187660906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fe2b40f8250 con 0x55c604432f60 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 10 client.7785 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 10 client.7785 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 20 client.7785 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c60402cb80 0x55c604599400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 4 client.7785 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2bc009780 con 0x55c60402cb80 2024-05-17T22:06:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c60402cb80 0x55c604599400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 -- 172.21.15.83:0/2871680562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 msgr2=0x7f53d40b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 --2- 172.21.15.83:0/2871680562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 0x7f53d40b8590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097a0 tx=0x7f53d001c3f0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 -- 172.21.15.83:0/2871680562 shutdown_connections 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 --2- 172.21.15.83:0/2871680562 >> v2:172.21.15.155:3300/0 conn(0x7f53d40d2070 0x7f53d406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 --2- 172.21.15.83:0/2871680562 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d0bc0 0x7f53d40b8590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 -- 172.21.15.83:0/2871680562 >> 172.21.15.83:0/2871680562 conn(0x7f53d40d00a0 msgr2=0x7f53d40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 10 client.7785 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c60402cb80 0x55c604599400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7770bc7f370d7e1e server_cookie=afb635fe084f56d9 in_seq=0 out_seq=0 2024-05-17T22:06:26.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 -- 172.21.15.83:0/2871680562 shutdown_connections 2024-05-17T22:06:26.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7f53db5ad640 1 -- 172.21.15.83:0/2871680562 wait complete. 2024-05-17T22:06:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 1 -- 172.21.15.83:0/187660906 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fe2bc009780 con 0x55c60402cb80 2024-05-17T22:06:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2a97fa640 10 client.7785 handle_command_reply: tid=0 2024-05-17T22:06:26.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.391+0000 7fe2d8090000 1 client.7785 shutdown 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 msgr2=0x55c60458f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40f61d0 tx=0x7fe2b40f8760 comp rx=0 tx=0).stop 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 shutdown_connections 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c60402cb80 0x55c604599400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604441240 0x55c60458faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c60458f560 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 --2- 172.21.15.83:0/187660906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c60441cdf0 0x55c60458f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 >> 172.21.15.83:0/187660906 conn(0x55c604047810 msgr2=0x55c60453f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2a97fa640 0 client.7785 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 shutdown_connections 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 1 -- 172.21.15.83:0/187660906 wait complete. 2024-05-17T22:06:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7fe2d8090000 20 client.7785 trim_cache size 0 max 16384 2024-05-17T22:06:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 Processor -- start 2024-05-17T22:06:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- start start 2024-05-17T22:06:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40d0bc0 0x7f53d415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d1c60 0x7f53d4168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d40d2070 0x7f53d41693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53d40b7480 con 0x7f53d40d0bc0 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53d406e900 con 0x7f53d40d2070 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53d40b7600 con 0x7f53d40d1c60 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40d0bc0 0x7f53d415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40d0bc0 0x7f53d415c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38378/0 (socket says 172.21.15.83:38378) 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 -- 172.21.15.83:0/3822549779 learned_addr learned my addr 172.21.15.83:0/3822549779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9b4a640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d40d2070 0x7f53d41693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d8b48640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d1c60 0x7f53d4168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 -- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d1c60 msgr2=0x7f53d4168ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53d40d1c60 0x7f53d4168ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 -- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d40d2070 msgr2=0x7f53d41693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53d40d2070 0x7f53d41693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 -- 172.21.15.83:0/3822549779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4161240 con 0x7f53d40d0bc0 2024-05-17T22:06:26.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53d9349640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d40d0bc0 0x7f53d415c780 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097a0 tx=0x7f53d001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00bb070 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d00059a0 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d00b0040 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- 172.21.15.83:0/3822549779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d4161520 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.395+0000 7f53db5ad640 1 -- 172.21.15.83:0/3822549779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d415ed60 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53db5ad640 1 -- 172.21.15.83:0/3822549779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f53d4166490 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f53d0006030 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53c9ffb640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f53b806f400 0x7f53b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f53d01297d0 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53d8b48640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f53b806f400 0x7f53b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f53d0016070 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53db5ad640 1 -- 172.21.15.83:0/3822549779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f53d40b8590 con 0x7f53d40d0bc0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53d8b48640 1 --2- 172.21.15.83:0/3822549779 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f53b806f400 0x7f53b8071910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f53d4164e00 tx=0x7f53c4007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.399+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 7 ==== log(2 entries from seq 894 at 2024-05-17T22:06:24.586035+0000) ==== 389+0+0 (secure 0 0 0) 0x7f53d0127db0 con 0x7f53d40d0bc0 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: cluster: 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: id: 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: health: HEALTH_OK 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: services: 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: mgr: y(active, since 9m), standbys: x 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: mds: 1/1 daemons up, 3 standby 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: data: 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: volumes: 1/1 healthy 2024-05-17T22:06:26.402 INFO:teuthology.orchestra.run.smithi083.stdout: pools: 3 pools, 97 pgs 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: objects: 30 objects, 19 MiB 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: pgs: 97 active+clean 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: io: 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:24.586035+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-17T22:06:26.403 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:24.586091+0000 mon.a [INF] Cluster is now healthy 2024-05-17T22:06:26.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.411+0000 7fe2d8090000 1 Processor -- start 2024-05-17T22:06:26.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.411+0000 7fe2d8090000 1 -- start start 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c604588630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604592040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc15b280 con 0x55c60441cdf0 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc09d880 con 0x55c604432f60 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc09d350 con 0x55c604441240 2024-05-17T22:06:26.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38392/0 (socket says 172.21.15.83:38392) 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 -- 172.21.15.83:0/1734973332 learned_addr learned my addr 172.21.15.83:0/1734973332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 -- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 msgr2=0x55c604592040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604592040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c604588630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 -- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 msgr2=0x55c604588630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c604588630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 -- 172.21.15.83:0/1734973332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60440aa10 con 0x55c60441cdf0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004160 tx=0x7fe2c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcd0837687b238a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ab7fe640 1 -- 172.21.15.83:0/1734973332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4019950 con 0x55c60441cdf0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ab7fe640 1 -- 172.21.15.83:0/1734973332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c402d070 con 0x55c60441cdf0 2024-05-17T22:06:26.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2ab7fe640 1 -- 172.21.15.83:0/1734973332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4019d30 con 0x55c60441cdf0 2024-05-17T22:06:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- 172.21.15.83:0/1734973332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 msgr2=0x55c6045880f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4004160 tx=0x7fe2c4015000 comp rx=0 tx=0).stop 2024-05-17T22:06:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- 172.21.15.83:0/1734973332 shutdown_connections 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c604592040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c604432f60 0x55c604588630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 --2- 172.21.15.83:0/1734973332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c60441cdf0 0x55c6045880f0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- 172.21.15.83:0/1734973332 >> 172.21.15.83:0/1734973332 conn(0x55c604047810 msgr2=0x55c604538700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- 172.21.15.83:0/1734973332 shutdown_connections 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- 172.21.15.83:0/1734973332 wait complete. 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 -- start start 2024-05-17T22:06:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.415+0000 7fe2d8090000 1 Processor -- start 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 0x55c6045fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c6045fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c604354dd0 con 0x55c604432f60 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c604394ff0 con 0x55c60441cdf0 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55c604389110 con 0x55c604441240 2024-05-17T22:06:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 10 client.0 _ll_register_callbacks cb 0x7fe2b4007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38404/0 (socket says 172.21.15.83:38404) 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/2082193988 learned_addr learned my addr 172.21.15.83:0/2082193988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 0x55c6045fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 msgr2=0x55c6045fcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2cad9d640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c6045fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c6045fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 msgr2=0x55c6045fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 0x55c6045fc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 -- 172.21.15.83:0/2082193988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c604342dc0 con 0x55c604432f60 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 0x55c6045fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2c1d9b640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c380 tx=0x7fe2b410f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aaf9190ee5880a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40f8ff0 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7fe2b4007360 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c402d070 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe2b40f8dc0 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 10 client.7794 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55c60432d3e0 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c402d070 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7fe2b40f48d0 con 0x55c604432f60 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 10 client.7794 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 10 client.7794 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 20 client.7794 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c604046230 0x55c6045949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 4 client.7794 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-17T22:06:26.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55c604321d40 con 0x55c604046230 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c604046230 0x55c6045949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 10 client.7794 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2ca59c640 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c604046230 0x55c6045949e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a5deca9af5b271 server_cookie=7defebf8ad1c6a67 in_seq=0 out_seq=0 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 1 -- 172.21.15.83:0/2082193988 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c604321d40 con 0x55c604046230 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 10 client.7794 handle_command_reply: tid=0 2024-05-17T22:06:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 client.7794 shutdown 2024-05-17T22:06:26.416 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-17T22:06:26.417 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-17T22:06:26.417 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-17T22:06:26.417 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 msgr2=0x55c6045fcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c380 tx=0x7fe2b410f000 comp rx=0 tx=0).stop 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 shutdown_connections 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55c604046230 0x55c6045949e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c604441240 0x55c6045fcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c604432f60 0x55c6045fcab0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 --2- 172.21.15.83:0/2082193988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60441cdf0 0x55c6045fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 >> 172.21.15.83:0/2082193988 conn(0x55c604047810 msgr2=0x55c604546190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2a97fa640 0 client.7794 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 shutdown_connections 2024-05-17T22:06:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.419+0000 7fe2d8090000 1 -- 172.21.15.83:0/2082193988 wait complete. 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d8090000 20 client.7794 trim_cache size 0 max 16384 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "f74796ba-290c-4461-8187-108f948430af", 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-05-17T22:06:26.419 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 msgr2=0x7fe2ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 0x7fe2ac071910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009fd0 tx=0x7fe2c400b040 comp rx=0 tx=0).stop 2024-05-17T22:06:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 msgr2=0x7fe2cc09e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400c290 tx=0x7fe2b8005840 comp rx=0 tx=0).stop 2024-05-17T22:06:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 shutdown_connections 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe2cc09ede0 0x7fe2cc151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2cc152660 0x7fe2cc09e8a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe2ac06f400 0x7fe2ac071910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 --2- 172.21.15.83:0/1023005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2b8004810 0x7fe2cc155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 >> 172.21.15.83:0/1023005403 conn(0x7fe2cc0b3ad0 msgr2=0x7fe2cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 shutdown_connections 2024-05-17T22:06:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.423+0000 7fe2d0e16640 1 -- 172.21.15.83:0/1023005403 wait complete. 2024-05-17T22:06:26.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:06:26.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-17T22:06:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 Processor -- start 2024-05-17T22:06:26.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 -- start start 2024-05-17T22:06:26.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f523c073590 0x7f523c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523c0d10d0 con 0x7f523c0b3d00 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f523c0b4110 con 0x7f523c073590 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5243ea0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f523c070770 con 0x7f523c143f20 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524243d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524143b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f523c073590 0x7f523c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524243d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:60074/0 (socket says 172.21.15.83:60074) 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f5241c3c640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f523c0b3d00 0x7f523c072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:51604/0 (socket says 172.21.15.83:51604) 2024-05-17T22:06:26.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524243d640 1 -- 172.21.15.83:0/763852716 learned_addr learned my addr 172.21.15.83:0/763852716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524143b640 1 -- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 msgr2=0x7f523c1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524143b640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524143b640 1 -- 172.21.15.83:0/763852716 >> v1:172.21.15.155:6789/0 conn(0x7f523c0b3d00 legacy=0x7f523c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524143b640 1 -- 172.21.15.83:0/763852716 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c070120 con 0x7f523c073590 2024-05-17T22:06:26.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.587+0000 7f524243d640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:06:26.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f524143b640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c073590 0x7f523c0d0a10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f522c0097c0 tx=0x7f522c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301aa63b8ac6fff server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5240c3a640 1 -- 172.21.15.83:0/763852716 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0b0070 con 0x7f523c073590 2024-05-17T22:06:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5240c3a640 1 -- 172.21.15.83:0/763852716 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c0ab600 con 0x7f523c073590 2024-05-17T22:06:26.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5240c3a640 1 -- 172.21.15.83:0/763852716 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0ab9d0 con 0x7f523c073590 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 -- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c073590 msgr2=0x7f523c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c073590 0x7f523c0d0a10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f522c0097c0 tx=0x7f522c01c4a0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 -- 172.21.15.83:0/763852716 shutdown_connections 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c143f20 0x7f523c1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 --2- 172.21.15.83:0/763852716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c073590 0x7f523c0d0a10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 -- 172.21.15.83:0/763852716 >> 172.21.15.83:0/763852716 conn(0x7f523c06cd90 msgr2=0x7f523c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 -- 172.21.15.83:0/763852716 shutdown_connections 2024-05-17T22:06:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.591+0000 7f5243ea0640 1 -- 172.21.15.83:0/763852716 wait complete. 2024-05-17T22:06:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 Processor -- start 2024-05-17T22:06:26.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 -- start start 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c073590 0x7f523c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 0x7f523c13e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c070c00 con 0x7f523c0b3d00 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f523c145540 con 0x7f523c143f20 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5243ea0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f523c0b61f0 con 0x7f523c073590 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38420/0 (socket says 172.21.15.83:38420) 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 -- 172.21.15.83:0/582124131 learned_addr learned my addr 172.21.15.83:0/582124131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f5241c3c640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c073590 0x7f523c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524243d640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 0x7f523c13e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 -- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c073590 msgr2=0x7f523c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c073590 0x7f523c15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 -- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 msgr2=0x7f523c13e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 0x7f523c13e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 -- 172.21.15.83:0/582124131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c0af550 con 0x7f523c0b3d00 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524243d640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 0x7f523c13e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:26.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.595+0000 7f524143b640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f522c01a0d0 tx=0x7f522c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.599+0000 7f52327fc640 1 -- 172.21.15.83:0/582124131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0ba070 con 0x7f523c0b3d00 2024-05-17T22:06:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.599+0000 7f52327fc640 1 -- 172.21.15.83:0/582124131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c0b43f0 con 0x7f523c0b3d00 2024-05-17T22:06:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.599+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c0af830 con 0x7f523c0b3d00 2024-05-17T22:06:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.599+0000 7f52327fc640 1 -- 172.21.15.83:0/582124131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0b0040 con 0x7f523c0b3d00 2024-05-17T22:06:26.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.599+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c0afdf0 con 0x7f523c0b3d00 2024-05-17T22:06:26.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f52327fc640 1 -- 172.21.15.83:0/582124131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f522c0bd070 con 0x7f523c0b3d00 2024-05-17T22:06:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f52327fc640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f52327fc640 1 -- 172.21.15.83:0/582124131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f522c128140 con 0x7f523c0b3d00 2024-05-17T22:06:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f5241c3c640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f5241c3c640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5234000f70 tx=0x7f5234007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 Processor -- start 2024-05-17T22:06:26.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 -- start start 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fe6ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe6f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e88fd0da90 con 0x55e88fcf6090 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e88fc81120 con 0x55e88fd08f80 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e88fc70cc0 con 0x55e88fcf5880 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38424/0 (socket says 172.21.15.83:38424) 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 -- 172.21.15.83:0/755409180 learned_addr learned my addr 172.21.15.83:0/755409180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f5241c3c640 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fe6ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524143b640 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe6f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 -- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 msgr2=0x55e88fe6f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe6f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 -- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 msgr2=0x55e88fe6ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fe6ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 -- 172.21.15.83:0/755409180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88fc2d110 con 0x55e88fcf6090 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f524243d640 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e80 tx=0x7f5238009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d75c1682603761d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f521f7fe640 1 -- 172.21.15.83:0/755409180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238007e60 con 0x55e88fcf6090 2024-05-17T22:06:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f521f7fe640 1 -- 172.21.15.83:0/755409180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523801a070 con 0x55e88fcf6090 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.603+0000 7f521f7fe640 1 -- 172.21.15.83:0/755409180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380149f0 con 0x55e88fcf6090 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/755409180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 msgr2=0x55e88fe6f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e80 tx=0x7f5238009810 comp rx=0 tx=0).stop 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/755409180 shutdown_connections 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fe6f8a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe6f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- 172.21.15.83:0/755409180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fe6ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/755409180 >> 172.21.15.83:0/755409180 conn(0x55e88fd44010 msgr2=0x55e88fe245c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/755409180 shutdown_connections 2024-05-17T22:06:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/755409180 wait complete. 2024-05-17T22:06:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- start start 2024-05-17T22:06:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 Processor -- start 2024-05-17T22:06:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88fe2bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e88fc2cec0 con 0x55e88fd08f80 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55e88fc74d00 con 0x55e88fcf6090 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e88f93a680 con 0x55e88fcf5880 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 10 client.0 _ll_register_callbacks cb 0x55e88fd281c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38434/0 (socket says 172.21.15.83:38434) 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 -- 172.21.15.83:0/2763666790 learned_addr learned my addr 172.21.15.83:0/2763666790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524143b640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88fe2bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f5241c3c640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 -- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 msgr2=0x55e88fe2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe2ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 -- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 msgr2=0x55e88fe2bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88fe2bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 -- 172.21.15.83:0/2763666790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88fc2f890 con 0x55e88fd08f80 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524243d640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5238000f90 tx=0x7f5238004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2547047620a2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f521cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238023070 con 0x55e88fd08f80 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f5238003db0 con 0x55e88fd08f80 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.607+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523801a070 con 0x55e88fd08f80 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f523802d050 con 0x55e88fd08f80 2024-05-17T22:06:26.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 10 client.7803 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55e88fc31650 con 0x55e88fd08f80 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523801a070 con 0x55e88fd08f80 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f5238030020 con 0x55e88fd08f80 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 10 client.7803 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 10 client.7803 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 20 client.7803 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f917930 0x55e88fe75e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 4 client.7803 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e88fc0cc40 con 0x55e88f917930 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f5241c3c640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f917930 0x55e88fe75e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 10 client.7803 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f5241c3c640 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f917930 0x55e88fe75e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a9af49cfc363e server_cookie=5e2ad42aa2f4bc9f in_seq=0 out_seq=0 2024-05-17T22:06:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 1 -- 172.21.15.83:0/2763666790 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e88fc0cc40 con 0x55e88f917930 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 10 client.7803 handle_command_reply: tid=0 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 client.7803 shutdown 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 msgr2=0x55e88fe30be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5238000f90 tx=0x7f5238004ac0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 shutdown_connections 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f917930 0x55e88fe75e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fe30be0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88fe2bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f521cff9640 0 client.7803 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 --2- 172.21.15.83:0/2763666790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fe2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 >> 172.21.15.83:0/2763666790 conn(0x55e88fd44010 msgr2=0x55e88fe245c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 shutdown_connections 2024-05-17T22:06:26.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 1 -- 172.21.15.83:0/2763666790 wait complete. 2024-05-17T22:06:26.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.611+0000 7f524b11a000 20 client.7803 trim_cache size 0 max 16384 2024-05-17T22:06:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.627+0000 7f524b11a000 1 Processor -- start 2024-05-17T22:06:26.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.627+0000 7f524b11a000 1 -- start start 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.627+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fedd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88feddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c205510 con 0x55e88fd08f80 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f523c204530 con 0x55e88fcf6090 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f523c0b5c20 con 0x55e88fcf5880 2024-05-17T22:06:26.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38446/0 (socket says 172.21.15.83:38446) 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 -- 172.21.15.83:0/2594113182 learned_addr learned my addr 172.21.15.83:0/2594113182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524143b640 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88feddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 -- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 msgr2=0x55e88fedd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fedd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 -- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 msgr2=0x55e88feddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88feddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 -- 172.21.15.83:0/2594113182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88fc108c0 con 0x55e88fd08f80 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524243d640 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f523802b040 tx=0x7f5238020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd01f67b23b647dc server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f521effd640 1 -- 172.21.15.83:0/2594113182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238007e60 con 0x55e88fd08f80 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f521effd640 1 -- 172.21.15.83:0/2594113182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523801a070 con 0x55e88fd08f80 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f521effd640 1 -- 172.21.15.83:0/2594113182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380041c0 con 0x55e88fd08f80 2024-05-17T22:06:26.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- 172.21.15.83:0/2594113182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 msgr2=0x55e88fede010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f523802b040 tx=0x7f5238020000 comp rx=0 tx=0).stop 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- 172.21.15.83:0/2594113182 shutdown_connections 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fd08f80 0x55e88fede010 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fcf6090 0x55e88feddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- 172.21.15.83:0/2594113182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fedd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- 172.21.15.83:0/2594113182 >> 172.21.15.83:0/2594113182 conn(0x55e88fd44010 msgr2=0x55e88fe85410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- 172.21.15.83:0/2594113182 shutdown_connections 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- 172.21.15.83:0/2594113182 wait complete. 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- start start 2024-05-17T22:06:26.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 Processor -- start 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fee1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fee23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e88fc180f0 con 0x55e88fcf6090 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f523c142920 con 0x55e88fd08f80 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.631+0000 7f524b11a000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55e88f93a680 con 0x55e88fcf5880 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 10 client.0 _ll_register_callbacks cb 0x7f5238009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:26.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38456/0 (socket says 172.21.15.83:38456) 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 -- 172.21.15.83:0/251295018 learned_addr learned my addr 172.21.15.83:0/251295018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5241c3c640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fee1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524243d640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fee23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 -- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 msgr2=0x55e88fee1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fee1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 -- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 msgr2=0x55e88fee23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fee23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 -- 172.21.15.83:0/251295018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88fc2bfa0 con 0x55e88fcf6090 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524243d640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fee23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524143b640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c10 tx=0x7f522c0fb360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaec8e96f5749c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0f42b0 con 0x55e88fcf6090 2024-05-17T22:06:26.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7f522c0f3bc0 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523801a070 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f522c0f40d0 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 10 client.7809 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55e88f92b220 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523801a070 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7f522c0b3070 con 0x55e88fcf6090 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 10 client.7809 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 10 client.7809 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 20 client.7809 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f905990 0x55e88fe9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 4 client.7809 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e88fc83070 con 0x55e88f905990 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5241c3c640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f905990 0x55e88fe9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 10 client.7809 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5241c3c640 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f905990 0x55e88fe9d680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f24845e1050945 server_cookie=6c2e55a87b3bb36b in_seq=0 out_seq=0 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 1 -- 172.21.15.83:0/251295018 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55e88fc83070 con 0x55e88f905990 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 10 client.7809 handle_command_reply: tid=0 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 client.7809 shutdown 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 msgr2=0x55e88fee1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c10 tx=0x7f522c0fb360 comp rx=0 tx=0).stop 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 shutdown_connections 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55e88f905990 0x55e88fe9d680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88fd08f80 0x55e88fee23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e88fcf6090 0x55e88fee1e70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f521cff9640 0 client.7809 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 --2- 172.21.15.83:0/251295018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e88fcf5880 0x55e88fee1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 >> 172.21.15.83:0/251295018 conn(0x55e88fd44010 msgr2=0x55e88fe2c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 shutdown_connections 2024-05-17T22:06:26.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 1 -- 172.21.15.83:0/251295018 wait complete. 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f524b11a000 20 client.7809 trim_cache size 0 max 16384 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "f74796ba-290c-4461-8187-108f948430af": { 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "f74796ba-290c-4461-8187-108f948430af", 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,repair" 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:26.634 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:26.635 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:26.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 msgr2=0x7f5220071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5234000f70 tx=0x7f5234007c90 comp rx=0 tx=0).stop 2024-05-17T22:06:26.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 msgr2=0x7f523c13ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f522c01a0d0 tx=0x7f522c019240 comp rx=0 tx=0).stop 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 shutdown_connections 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f522006f400 0x7f5220071910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f523c143f20 0x7f523c13e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c0b3d00 0x7f523c13ee10 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 --2- 172.21.15.83:0/582124131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f523c073590 0x7f523c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 >> 172.21.15.83:0/582124131 conn(0x7f523c06cd90 msgr2=0x7f523c0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 shutdown_connections 2024-05-17T22:06:26.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:26.635+0000 7f5243ea0640 1 -- 172.21.15.83:0/582124131 wait complete. 2024-05-17T22:06:26.648 INFO:tasks.cephfs.filesystem:scrub status for tag:f74796ba-290c-4461-8187-108f948430af - {'path': '/', 'tag': 'f74796ba-290c-4461-8187-108f948430af', 'options': 'recursive,repair'} 2024-05-17T22:06:27.564 INFO:tasks.ceph.mds.a.smithi083.stderr:2024-05-17T22:06:27.571+0000 7fc3e8b54640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:06:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:27.599+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-17T22:06:26.425217+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f53d0005d50 con 0x7f53d40d0bc0 2024-05-17T22:06:27.593 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.425217+0000 mds.a [INF] scrub queued for path: / 2024-05-17T22:06:27.593 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.425221+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.425225+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.425941+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.425946+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.426001+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.426008+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-05-17T22:06:27.594 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:26.427075+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-17T22:06:31.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:06:31.681 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:06:32.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:32.227+0000 7f53c9ffb640 1 -- 172.21.15.83:0/3822549779 <== mon.0 v2:172.21.15.83:3300/0 9 ==== log(1 entries from seq 9 at 2024-05-17T22:06:30.658794+0000) ==== 189+0+0 (secure 0 0 0) 0x7f53d00f2280 con 0x7f53d40d0bc0 2024-05-17T22:06:32.220 INFO:teuthology.orchestra.run.smithi083.stdout:2024-05-17T22:06:30.658794+0000 mds.a [INF] scrub summary: idle 2024-05-17T22:06:56.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:06:56.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-17T22:06:56.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.819+0000 7ff82d664640 1 Processor -- start 2024-05-17T22:06:56.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.819+0000 7ff82d664640 1 -- start start 2024-05-17T22:06:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff82d664640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 0x7ff8280d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff82d664640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff828143f20 0x7ff828072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff82d664640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8280737d0 con 0x7ff8280733c0 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff82d664640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff8280b4110 con 0x7ff828143f20 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff82d664640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff828070770 con 0x7ff8280b3d00 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff828143f20 0x7ff828072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff828143f20 0x7ff828072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48758/0 (socket says 172.21.15.83:48758) 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff8277fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff8280733c0 0x7ff82806eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46806/0 (socket says 172.21.15.83:46806) 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 -- 172.21.15.83:0/4163395766 learned_addr learned my addr 172.21.15.83:0/4163395766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff826ffd640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 0x7ff8280d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 -- 172.21.15.83:0/4163395766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 msgr2=0x7ff8280d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 0x7ff8280d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 -- 172.21.15.83:0/4163395766 >> v1:172.21.15.83:6789/0 conn(0x7ff8280733c0 legacy=0x7ff82806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff81e7fc640 1 -- 172.21.15.83:0/4163395766 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8280d10d0 con 0x7ff828143f20 2024-05-17T22:06:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.823+0000 7ff826ffd640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 0x7ff8280d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff828072e80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff810009870 tx=0x7ff81001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407321a08b7e229a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff8267fc640 1 -- 172.21.15.83:0/4163395766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100ab650 con 0x7ff828143f20 2024-05-17T22:06:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff8267fc640 1 -- 172.21.15.83:0/4163395766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8100b0070 con 0x7ff828143f20 2024-05-17T22:06:56.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff8267fc640 1 -- 172.21.15.83:0/4163395766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100aba10 con 0x7ff828143f20 2024-05-17T22:06:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 -- 172.21.15.83:0/4163395766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 msgr2=0x7ff828072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff828072e80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff810009870 tx=0x7ff81001c560 comp rx=0 tx=0).stop 2024-05-17T22:06:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 -- 172.21.15.83:0/4163395766 shutdown_connections 2024-05-17T22:06:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff828072e80 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 --2- 172.21.15.83:0/4163395766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280b3d00 0x7ff8280d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 -- 172.21.15.83:0/4163395766 >> 172.21.15.83:0/4163395766 conn(0x7ff82806cd90 msgr2=0x7ff8280b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 -- 172.21.15.83:0/4163395766 shutdown_connections 2024-05-17T22:06:56.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.827+0000 7ff82d664640 1 -- 172.21.15.83:0/4163395766 wait complete. 2024-05-17T22:06:56.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 Processor -- start 2024-05-17T22:06:56.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- start start 2024-05-17T22:06:56.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280733c0 0x7ff828139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8280b3d00 0x7ff8281381d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff828070c00 con 0x7ff828143f20 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff828144330 con 0x7ff8280b3d00 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff8281f7880 con 0x7ff8280733c0 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48768/0 (socket says 172.21.15.83:48768) 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 -- 172.21.15.83:0/4042012244 learned_addr learned my addr 172.21.15.83:0/4042012244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8280b3d00 0x7ff8281381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff826ffd640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280733c0 0x7ff828139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 -- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280733c0 msgr2=0x7ff828139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280733c0 0x7ff828139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 -- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8280b3d00 msgr2=0x7ff8281381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8280b3d00 0x7ff8281381d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 -- 172.21.15.83:0/4042012244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff828138ef0 con 0x7ff828143f20 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff8277fe640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c700 tx=0x7ff82000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff81ffff640 1 -- 172.21.15.83:0/4042012244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820018070 con 0x7ff828143f20 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff81ffff640 1 -- 172.21.15.83:0/4042012244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820004d70 con 0x7ff828143f20 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281391d0 con 0x7ff828143f20 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff81ffff640 1 -- 172.21.15.83:0/4042012244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82000a820 con 0x7ff828143f20 2024-05-17T22:06:56.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.831+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff828139790 con 0x7ff828143f20 2024-05-17T22:06:56.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81ffff640 1 -- 172.21.15.83:0/4042012244 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff82001b070 con 0x7ff828143f20 2024-05-17T22:06:56.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81ffff640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 0x7ff808071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81ffff640 1 -- 172.21.15.83:0/4042012244 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7ff82008e6e0 con 0x7ff828143f20 2024-05-17T22:06:56.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 0x7ff808071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 0x7ff808071940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b930 tx=0x7ff818008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 Processor -- start 2024-05-17T22:06:56.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 -- start start 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70165880 0x55cf702df360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf702df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf7017da90 con 0x55cf70178f80 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf700f1120 con 0x55cf70166090 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cf700e0cc0 con 0x55cf70165880 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48770/0 (socket says 172.21.15.83:48770) 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 -- 172.21.15.83:0/1318770651 learned_addr learned my addr 172.21.15.83:0/1318770651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70165880 0x55cf702df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff8277fe640 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf702df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 -- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70165880 msgr2=0x55cf702df360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70165880 0x55cf702df360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 -- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 msgr2=0x55cf702df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf702df8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 -- 172.21.15.83:0/1318770651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7009d110 con 0x55cf70178f80 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff81800f9f0 tx=0x7ff818012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b923315c89e86c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81cff9640 1 -- 172.21.15.83:0/1318770651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81801b050 con 0x55cf70178f80 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81cff9640 1 -- 172.21.15.83:0/1318770651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818025070 con 0x55cf70178f80 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.835+0000 7ff81cff9640 1 -- 172.21.15.83:0/1318770651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818014de0 con 0x55cf70178f80 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- 172.21.15.83:0/1318770651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 msgr2=0x55cf702dee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff81800f9f0 tx=0x7ff818012000 comp rx=0 tx=0).stop 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- 172.21.15.83:0/1318770651 shutdown_connections 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf702df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70165880 0x55cf702df360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- 172.21.15.83:0/1318770651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70178f80 0x55cf702dee20 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- 172.21.15.83:0/1318770651 >> 172.21.15.83:0/1318770651 conn(0x55cf701b4010 msgr2=0x55cf702945c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- 172.21.15.83:0/1318770651 shutdown_connections 2024-05-17T22:06:56.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- 172.21.15.83:0/1318770651 wait complete. 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- start start 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 Processor -- start 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf7029bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf702a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf7009cec0 con 0x55cf70165880 2024-05-17T22:06:56.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cf700e4d00 con 0x55cf70166090 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cf6fdaa680 con 0x55cf70178f80 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8348de000 10 client.0 _ll_register_callbacks cb 0x55cf701981c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48786/0 (socket says 172.21.15.83:48786) 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 -- 172.21.15.83:0/2922663966 learned_addr learned my addr 172.21.15.83:0/2922663966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff8277fe640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf702a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf7029bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 -- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 msgr2=0x55cf702a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf702a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 -- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 msgr2=0x55cf7029bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf7029bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 -- 172.21.15.83:0/2922663966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7009f890 con 0x55cf70165880 2024-05-17T22:06:56.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf7029bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff826ffd640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff818022040 tx=0x7ff818004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50e03cd82030f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.839+0000 7ff805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818020c80 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7ff818034470 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff818025070 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff81801b020 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 10 client.7824 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55cf700a1650 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818025070 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7ff818034670 con 0x55cf70165880 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 10 client.7824 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 10 client.7824 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 20 client.7824 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:56.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf6fd87930 0x55cf702e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 4 client.7824 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7007cc40 con 0x55cf6fd87930 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf6fd87930 0x55cf702e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf6fd87930 0x55cf702e5e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680d0597bda1d744 server_cookie=c3d07e81cb042f47 in_seq=0 out_seq=0 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 10 client.7824 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 1 -- 172.21.15.83:0/2922663966 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf7007cc40 con 0x55cf6fd87930 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 10 client.7824 handle_command_reply: tid=0 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 client.7824 shutdown 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 msgr2=0x55cf7029ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ff818022040 tx=0x7ff818004740 comp rx=0 tx=0).stop 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 shutdown_connections 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf6fd87930 0x55cf702e5e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf702a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70166090 0x55cf7029bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 --2- 172.21.15.83:0/2922663966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70165880 0x55cf7029ba70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff805ffb640 0 client.7824 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 >> 172.21.15.83:0/2922663966 conn(0x55cf701b4010 msgr2=0x55cf702945c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 shutdown_connections 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 1 -- 172.21.15.83:0/2922663966 wait complete. 2024-05-17T22:06:56.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.843+0000 7ff8348de000 20 client.7824 trim_cache size 0 max 16384 2024-05-17T22:06:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 Processor -- start 2024-05-17T22:06:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- start start 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf70352300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6fda0fe0 0x55cf70352840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff828203750 con 0x55cf70166090 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8282027d0 con 0x55cf6fda0fe0 2024-05-17T22:06:56.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff82813c6a0 con 0x55cf70178f80 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48794/0 (socket says 172.21.15.83:48794) 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 -- 172.21.15.83:0/351353464 learned_addr learned my addr 172.21.15.83:0/351353464 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 -- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 msgr2=0x55cf70352300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf70352300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 -- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6fda0fe0 msgr2=0x55cf70352840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6fda0fe0 0x55cf70352840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 -- 172.21.15.83:0/351353464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf700808c0 con 0x55cf70166090 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff826ffd640 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c010 tx=0x7ff81802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96251f1c65a8f94b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff807fff640 1 -- 172.21.15.83:0/351353464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818044070 con 0x55cf70166090 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff807fff640 1 -- 172.21.15.83:0/351353464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818025070 con 0x55cf70166090 2024-05-17T22:06:56.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff807fff640 1 -- 172.21.15.83:0/351353464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818034db0 con 0x55cf70166090 2024-05-17T22:06:56.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- 172.21.15.83:0/351353464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 msgr2=0x55cf70351dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c010 tx=0x7ff81802d070 comp rx=0 tx=0).stop 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- 172.21.15.83:0/351353464 shutdown_connections 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf6fda0fe0 0x55cf70352840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf70178f80 0x55cf70352300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 --2- 172.21.15.83:0/351353464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70351dc0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- 172.21.15.83:0/351353464 >> 172.21.15.83:0/351353464 conn(0x55cf701b4010 msgr2=0x55cf702f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- 172.21.15.83:0/351353464 shutdown_connections 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.863+0000 7ff8348de000 1 -- 172.21.15.83:0/351353464 wait complete. 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- start start 2024-05-17T22:06:56.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 Processor -- start 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf6fda0fe0 0x55cf70356130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 0x55cf70311180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf700880f0 con 0x55cf70166090 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8282022d0 con 0x55cf70178f80 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55cf6fdaa680 con 0x55cf6fda0fe0 2024-05-17T22:06:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 10 client.0 _ll_register_callbacks cb 0x7ff81800f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48804/0 (socket says 172.21.15.83:48804) 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 -- 172.21.15.83:0/1271384195 learned_addr learned my addr 172.21.15.83:0/1271384195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8277fe640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 0x55cf70311180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf6fda0fe0 0x55cf70356130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 -- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf6fda0fe0 msgr2=0x55cf70356130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf6fda0fe0 0x55cf70356130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 -- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 msgr2=0x55cf70311180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 0x55cf70311180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 -- 172.21.15.83:0/1271384195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7009bfa0 con 0x55cf70166090 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8277fe640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 0x55cf70311180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff81e7fc640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff81000b0b0 tx=0x7ff81001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7155effb3ca6cdb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:06:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100c4060 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7ff8100be6a0 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff818025070 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff8100ca050 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 10 client.7830 fetch_fsmap learned FSMap version 87 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x55cf6fd9b220 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818025070 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 87) ==== 3494+0+0 (secure 0 0 0) 0x7ff8100be8a0 con 0x55cf70166090 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 10 client.7830 fetch_fsmap finished waiting for FSMap version 87 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 10 client.7830 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 20 client.7830 populate_metadata read hostname 'smithi083' 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf703116c0 0x55cf70311ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 4 client.7830 mds_command: new command op to 7704 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-17T22:06:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 --> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf700f3070 con 0x55cf703116c0 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf703116c0 0x55cf70311ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 10 client.7830 ms_handle_connect on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff826ffd640 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf703116c0 0x55cf70311ad0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b42d46ad42e317 server_cookie=86d8796540b41c0e in_seq=0 out_seq=0 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 1 -- 172.21.15.83:0/1271384195 <== mds.0 v2:172.21.15.83:6834/563832419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf700f3070 con 0x55cf703116c0 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 10 client.7830 handle_command_reply: tid=0 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 client.7830 shutdown 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 msgr2=0x55cf70356670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff81000b0b0 tx=0x7ff81001ada0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 shutdown_connections 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:6834/563832419,v1:172.21.15.83:6836/563832419] conn(0x55cf703116c0 0x55cf70311ad0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cf70178f80 0x55cf70311180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf70166090 0x55cf70356670 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 --2- 172.21.15.83:0/1271384195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55cf6fda0fe0 0x55cf70356130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff805ffb640 0 client.7830 ms_handle_reset on v2:172.21.15.83:6834/563832419 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 >> 172.21.15.83:0/1271384195 conn(0x55cf701b4010 msgr2=0x55cf702f5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 shutdown_connections 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.867+0000 7ff8348de000 1 -- 172.21.15.83:0/1271384195 wait complete. 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff8348de000 20 client.7830 trim_cache size 0 max 16384 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-05-17T22:06:56.871 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 msgr2=0x7ff808071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 0x7ff808071940 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b930 tx=0x7ff818008040 comp rx=0 tx=0).stop 2024-05-17T22:06:56.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 msgr2=0x7ff8281f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c700 tx=0x7ff82000cbc0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 shutdown_connections 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff828143f20 0x7ff8281f6af0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff80806f430 0x7ff808071940 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8280b3d00 0x7ff8281381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 --2- 172.21.15.83:0/4042012244 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8280733c0 0x7ff828139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 >> 172.21.15.83:0/4042012244 conn(0x7ff82806cd90 msgr2=0x7ff8280d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 shutdown_connections 2024-05-17T22:06:56.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:56.871+0000 7ff82d664640 1 -- 172.21.15.83:0/4042012244 wait complete. 2024-05-17T22:06:56.883 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-17T22:06:56.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:06:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 Processor -- start 2024-05-17T22:06:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 -- start start 2024-05-17T22:06:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd1500b3d00 0x7fd1500d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd150143f20 0x7fd150072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1500737d0 con 0x7fd1500733c0 2024-05-17T22:06:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd1500b4110 con 0x7fd150143f20 2024-05-17T22:06:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd15635b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd150070770 con 0x7fd1500b3d00 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd150143f20 0x7fd150072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd150143f20 0x7fd150072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53212/0 (socket says 172.21.15.83:53212) 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 -- 172.21.15.83:0/488264632 learned_addr learned my addr 172.21.15.83:0/488264632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14ffff640 1 --2- 172.21.15.83:0/488264632 >> v2:172.21.15.155:3300/0 conn(0x7fd1500b3d00 0x7fd1500d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 -- 172.21.15.83:0/488264632 >> v2:172.21.15.155:3300/0 conn(0x7fd1500b3d00 msgr2=0x7fd1500d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 --2- 172.21.15.83:0/488264632 >> v2:172.21.15.155:3300/0 conn(0x7fd1500b3d00 0x7fd1500d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 -- 172.21.15.83:0/488264632 >> v1:172.21.15.155:6789/0 conn(0x7fd1500733c0 legacy=0x7fd15006eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.147+0000 7fd14f7fe640 1 -- 172.21.15.83:0/488264632 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1500d10d0 con 0x7fd150143f20 2024-05-17T22:06:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd14f7fe640 1 --2- 172.21.15.83:0/488264632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150143f20 0x7fd150072e80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd15006d6e0 tx=0x7fd13801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3a044d16f9fd1f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd14effd640 1 -- 172.21.15.83:0/488264632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380b2070 con 0x7fd150143f20 2024-05-17T22:06:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd14effd640 1 -- 172.21.15.83:0/488264632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1380ad680 con 0x7fd150143f20 2024-05-17T22:06:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd14effd640 1 -- 172.21.15.83:0/488264632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380ada50 con 0x7fd150143f20 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 -- 172.21.15.83:0/488264632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150143f20 msgr2=0x7fd150072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 --2- 172.21.15.83:0/488264632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150143f20 0x7fd150072e80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd15006d6e0 tx=0x7fd13801c490 comp rx=0 tx=0).stop 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 -- 172.21.15.83:0/488264632 shutdown_connections 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 --2- 172.21.15.83:0/488264632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd150143f20 0x7fd150072e80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 --2- 172.21.15.83:0/488264632 >> v2:172.21.15.155:3300/0 conn(0x7fd1500b3d00 0x7fd1500d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 -- 172.21.15.83:0/488264632 >> 172.21.15.83:0/488264632 conn(0x7fd15006cd90 msgr2=0x7fd1500b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 -- 172.21.15.83:0/488264632 shutdown_connections 2024-05-17T22:06:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.151+0000 7fd15635b640 1 -- 172.21.15.83:0/488264632 wait complete. 2024-05-17T22:06:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 Processor -- start 2024-05-17T22:06:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- start start 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 0x7fd15013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd150143f20 0x7fd1500ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd150070c00 con 0x7fd1500b3d00 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd150144330 con 0x7fd1500733c0 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd1500af9c0 con 0x7fd150143f20 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53228/0 (socket says 172.21.15.83:53228) 2024-05-17T22:06:57.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 -- 172.21.15.83:0/48846899 learned_addr learned my addr 172.21.15.83:0/48846899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14ffff640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 0x7fd15013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd1548f8640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd150143f20 0x7fd1500ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 -- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd150143f20 msgr2=0x7fd1500ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd150143f20 0x7fd1500ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 -- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 msgr2=0x7fd15013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 0x7fd15013fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 -- 172.21.15.83:0/48846899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd150153e60 con 0x7fd1500b3d00 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14ffff640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 0x7fd15013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14f7fe640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd1380190a0 tx=0x7fd13801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380bd070 con 0x7fd1500b3d00 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1380adc30 con 0x7fd1500b3d00 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd150154140 con 0x7fd1500b3d00 2024-05-17T22:06:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1380b2040 con 0x7fd1500b3d00 2024-05-17T22:06:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd150154700 con 0x7fd1500b3d00 2024-05-17T22:06:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.155+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1500d0a10 con 0x7fd1500b3d00 2024-05-17T22:06:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.159+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd1380c0070 con 0x7fd1500b3d00 2024-05-17T22:06:57.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.159+0000 7fd14cff9640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 0x7fd130071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.159+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5795+0+0 (secure 0 0 0) 0x7fd138128bb0 con 0x7fd1500b3d00 2024-05-17T22:06:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.159+0000 7fd14ffff640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 0x7fd130071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.159+0000 7fd14ffff640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 0x7fd130071940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b9f0 tx=0x7fd140008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.163+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd1380c7080 con 0x7fd1500b3d00 2024-05-17T22:06:57.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.359+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd15006f3b0 con 0x7fd1500b3d00 2024-05-17T22:06:57.354 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:06:57.363+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:06:57.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.363+0000 7fd14cff9640 1 -- 172.21.15.83:0/48846899 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7fd138016110 con 0x7fd1500b3d00 2024-05-17T22:06:57.357 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 msgr2=0x7fd130071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 0x7fd130071940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b9f0 tx=0x7fd140008040 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 msgr2=0x7fd15013f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd1380190a0 tx=0x7fd13801acd0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 shutdown_connections 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd150143f20 0x7fd1500ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1500b3d00 0x7fd15013f4b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd13006f430 0x7fd130071940 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 --2- 172.21.15.83:0/48846899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1500733c0 0x7fd15013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 >> 172.21.15.83:0/48846899 conn(0x7fd15006cd90 msgr2=0x7fd15015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 shutdown_connections 2024-05-17T22:06:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.367+0000 7fd15635b640 1 -- 172.21.15.83:0/48846899 wait complete. 2024-05-17T22:06:57.366 INFO:tasks.ceph.mds.a.smithi083.stderr: -1151> 2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-17T22:06:57.366 INFO:tasks.ceph.mds.a.smithi083.stderr: -1148> 2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-17T22:06:57.366 INFO:tasks.ceph.mds.a.smithi083.stderr: -1132> 2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-17T22:06:57.366 INFO:tasks.ceph.mds.a.smithi083.stderr: -1129> 2024-05-17T22:06:26.419+0000 7fc3e4b4c640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-17T22:06:57.366 INFO:tasks.ceph.mds.a.smithi083.stderr: -963> 2024-05-17T22:06:27.571+0000 7fc3e8b54640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:06:57.384 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-17T22:06:57.425 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:06:57.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- start start 2024-05-17T22:06:57.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 Processor -- start 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7037180 0x5627b7d8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6000 0x5627b7d8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d90180 con 0x5627b7ed6480 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d90300 con 0x5627b7037180 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b7d90480 con 0x5627b7ed6000 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53266/0 (socket says 172.21.15.83:53266) 2024-05-17T22:06:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 -- 172.21.15.83:0/422000579 learned_addr learned my addr 172.21.15.83:0/422000579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a181b7640 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7037180 0x5627b7d8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a179b6640 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6000 0x5627b7d8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 -- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6000 msgr2=0x5627b7d8c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6000 0x5627b7d8c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 -- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7037180 msgr2=0x5627b7d8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7037180 0x5627b7d8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 -- 172.21.15.83:0/422000579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b7d84b40 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a189b8640 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x5627b7e40960 tx=0x5627b70d5f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46aa7b802a7b8215 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a159b2640 1 -- 172.21.15.83:0/422000579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa9c0 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a159b2640 1 -- 172.21.15.83:0/422000579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b700e1c0 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a159b2640 1 -- 172.21.15.83:0/422000579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa4e0 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b7d84d20 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5627b7d84f00 con 0x5627b7ed6480 2024-05-17T22:06:57.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 4 waiting for MDS map... 2024-05-17T22:06:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a159b2640 1 -- 172.21.15.83:0/422000579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x5627b7d86a00 con 0x5627b7ed6480 2024-05-17T22:06:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a159b2640 1 -- 172.21.15.83:0/422000579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 88) ==== 2885+0+0 (secure 0 0 0) 0x5627b701b180 con 0x5627b7ed6480 2024-05-17T22:06:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 4 Got MDS map 88 2024-05-17T22:06:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 10 main: main 2024-05-17T22:06:57.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 4 main: connecting to RADOS... 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 Processor -- start 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.455+0000 7f4a199f8400 1 -- start start 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d90600 con 0x5627b7ed7200 2024-05-17T22:06:57.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d90780 con 0x5627b7ed6d80 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b7d90900 con 0x5627b7ed7680 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53272/0 (socket says 172.21.15.83:53272) 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 -- 172.21.15.83:0/714024356 learned_addr learned my addr 172.21.15.83:0/714024356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a179b6640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a179b6640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 -- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 msgr2=0x5627b7d8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 -- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed6d80 msgr2=0x5627b7d8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 -- 172.21.15.83:0/714024356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b7d841e0 con 0x5627b7ed7200 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a179b6640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:57.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a181b7640 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x5627b7f026c0 tx=0x5627b7efec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9832e21c086c17c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a139ae640 1 -- 172.21.15.83:0/714024356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa9c0 con 0x5627b7ed7200 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a139ae640 1 -- 172.21.15.83:0/714024356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b700ee00 con 0x5627b7ed7200 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a139ae640 1 -- 172.21.15.83:0/714024356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa4e0 con 0x5627b7ed7200 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- 172.21.15.83:0/714024356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 msgr2=0x5627b7d8d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x5627b7f026c0 tx=0x5627b7efec00 comp rx=0 tx=0).stop 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- 172.21.15.83:0/714024356 shutdown_connections 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 --2- 172.21.15.83:0/714024356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- 172.21.15.83:0/714024356 >> 172.21.15.83:0/714024356 conn(0x5627b7ed6900 msgr2=0x5627b72de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- 172.21.15.83:0/714024356 shutdown_connections 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- 172.21.15.83:0/714024356 wait complete. 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 Processor -- start 2024-05-17T22:06:57.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.459+0000 7f4a199f8400 1 -- start start 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d91c80 con 0x5627b7ed7200 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b7d91b00 con 0x5627b7ed7680 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b7d91980 con 0x5627b7ed6d80 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53280/0 (socket says 172.21.15.83:53280) 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 -- 172.21.15.83:0/3849266731 learned_addr learned my addr 172.21.15.83:0/3849266731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a179b6640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 -- 172.21.15.83:0/3849266731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6d80 msgr2=0x5627b7d8d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6d80 0x5627b7d8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 -- 172.21.15.83:0/3849266731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed7680 msgr2=0x5627b7d8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7ed7680 0x5627b7d8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 -- 172.21.15.83:0/3849266731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b7d85a40 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a181b7640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x5627b7f03380 tx=0x5627b70d7320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 -- 172.21.15.83:0/3849266731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa000 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 -- 172.21.15.83:0/3849266731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b700ec40 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 -- 172.21.15.83:0/3849266731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b6ffa9c0 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627b7d85c20 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b7d85a40 con 0x5627b7ed7200 2024-05-17T22:06:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 4 main: resolving pool 14 2024-05-17T22:06:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 -- 172.21.15.83:0/3849266731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x5627b72fec00 con 0x5627b7ed7200 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7ed9600 0x5627b7d8e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a121ab640 1 -- 172.21.15.83:0/3849266731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x5627b7f22a00 con 0x5627b7ed7200 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a179b6640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7ed9600 0x5627b7d8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 4 main: creating IoCtx.. 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7ed7b00 0x5627b7d8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5627b7404000 con 0x5627b7ed7b00 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a179b6640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7ed7b00 0x5627b7d8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a179b6640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7ed9600 0x5627b7d8e100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5627b7114720 tx=0x5627b7efeea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.463+0000 7f4a179b6640 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7ed7b00 0x5627b7d8e680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a179b6640 1 -- 172.21.15.83:0/3849266731 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x5627b70c2000 con 0x5627b7ed7b00 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "0": { 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "data": { 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "version": 5, 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "inotable": { 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "projected_free": [ 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "start": 1099511627779, 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "len": 1099511627773 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-05-17T22:06:57.462 INFO:teuthology.orchestra.run.smithi083.stdout: "free": [ 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: "start": 1099511627779, 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: "len": 1099511627773 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: "result": 0 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:06:57.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7ed7b00 msgr2=0x5627b7d8e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7ed7b00 0x5627b7d8e680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7ed9600 msgr2=0x5627b7d8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7ed9600 0x5627b7d8e100 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5627b7114720 tx=0x5627b7efeea0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 msgr2=0x5627b7d8d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/3849266731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed7200 0x5627b7d8d600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x5627b7f03380 tx=0x5627b70d7320 comp rx=0 tx=0).stop 2024-05-17T22:06:57.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a181b7640 1 -- 172.21.15.83:0/3849266731 reap_dead start 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 shutdown_connections 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 >> 172.21.15.83:0/3849266731 conn(0x5627b7ed6900 msgr2=0x5627b72de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 shutdown_connections 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/3849266731 wait complete. 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 msgr2=0x5627b7d8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x5627b7e40960 tx=0x5627b70d5f20 comp rx=0 tx=0).stop 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 shutdown_connections 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7ed6480 0x5627b7d8cb00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7ed6000 0x5627b7d8c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 --2- 172.21.15.83:0/422000579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7037180 0x5627b7d8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 >> 172.21.15.83:0/422000579 conn(0x5627b7037a80 msgr2=0x5627b2fc3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 shutdown_connections 2024-05-17T22:06:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.467+0000 7f4a199f8400 1 -- 172.21.15.83:0/422000579 wait complete. 2024-05-17T22:06:57.467 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.083157, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-17T22:06:57.468 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:06:57.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:06:57.548 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:06:57.610 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:06:57.610 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:06:57.610 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:06:57.610 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:06:57.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:06:57.615 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:06:57.616 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:06:57.616 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:06:57.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:06:57.622 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:06:57.622 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:06:57.622 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:06:57.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:06:57.663 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:06:57.663 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:06:57.663 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:06:57.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:06:57.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:57.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.859+0000 7f68d3fe3640 1 Processor -- start 2024-05-17T22:06:57.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.859+0000 7f68d3fe3640 1 -- start start 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d3fe3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68cc073540 0x7f68cc168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d3fe3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc15e790 con 0x7f68cc144080 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d3fe3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc15ea40 con 0x7f68cc0d1040 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d3fe3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f68cc15ed20 con 0x7f68cc073540 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d157e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68cc144080 0x7f68cc072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49818/0 (socket says 172.21.15.83:49818) 2024-05-17T22:06:57.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d157e640 1 -- 172.21.15.83:0/351632109 learned_addr learned my addr 172.21.15.83:0/351632109 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d2580640 1 --2- 172.21.15.83:0/351632109 >> v2:172.21.15.155:3300/0 conn(0x7f68cc073540 0x7f68cc168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674342075 0 0) 0x7f68cc15e790 con 0x7f68cc144080 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b4003530 con 0x7f68cc144080 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598720684 0 0) 0x7f68b4003530 con 0x7f68cc144080 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 >> v2:172.21.15.155:3300/0 conn(0x7f68cc073540 msgr2=0x7f68cc168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 --2- 172.21.15.83:0/351632109 >> v2:172.21.15.155:3300/0 conn(0x7f68cc073540 0x7f68cc168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 >> v1:172.21.15.155:6789/0 conn(0x7f68cc0d1040 legacy=0x7f68cc15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc15f000 con 0x7f68cc144080 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f68c0001700 con 0x7f68cc144080 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 >> v1:172.21.15.83:6789/0 conn(0x7f68cc144080 legacy=0x7f68cc072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68b4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b4004810 0x7f68b4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b4005260 0x7f68b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc15ed20 con 0x7f68b4003a60 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68b400c1f0 con 0x7f68b4004810 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68b400c3f0 con 0x7f68b4005260 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68b4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d2580640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b4005260 0x7f68b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d157e640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b4004810 0x7f68b4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 -- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b4005260 msgr2=0x7f68b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68b4005260 0x7f68b400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 -- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b4004810 msgr2=0x7f68b4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68b4004810 0x7f68b4004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d2580640 1 -- 172.21.15.83:0/351632109 reap_dead start 2024-05-17T22:06:57.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.863+0000 7f68d1d7f640 1 -- 172.21.15.83:0/351632109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b4003530 con 0x7f68b4003a60 2024-05-17T22:06:57.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68b4003ed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f68bc001790 tx=0x7f68bc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86bfcdaf19de241 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0013e0 con 0x7f68b4003a60 2024-05-17T22:06:57.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0052d0 con 0x7f68b4003a60 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d0d7d640 1 -- 172.21.15.83:0/351632109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc003840 con 0x7f68b4003a60 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 msgr2=0x7f68b4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68b4003ed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f68bc001790 tx=0x7f68bc01c360 comp rx=0 tx=0).stop 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- 172.21.15.83:0/351632109 shutdown_connections 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/351632109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68b4003ed0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- 172.21.15.83:0/351632109 >> 172.21.15.83:0/351632109 conn(0x7f68cc06cd90 msgr2=0x7f68cc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- 172.21.15.83:0/351632109 shutdown_connections 2024-05-17T22:06:57.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- 172.21.15.83:0/351632109 wait complete. 2024-05-17T22:06:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 Processor -- start 2024-05-17T22:06:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.867+0000 7f68d3fe3640 1 -- start start 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68cc13dfa0 0x7f68cc13e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68cc146870 0x7f68cc146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc15e610 con 0x7f68b4003a60 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc15e490 con 0x7f68cc13dfa0 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc1616c0 con 0x7f68cc146870 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53296/0 (socket says 172.21.15.83:53296) 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 -- 172.21.15.83:0/3229500595 learned_addr learned my addr 172.21.15.83:0/3229500595 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:57.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d157e640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68cc13dfa0 0x7f68cc13e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d2580640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68cc146870 0x7f68cc146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 -- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68cc146870 msgr2=0x7f68cc146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68cc146870 0x7f68cc146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 -- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68cc13dfa0 msgr2=0x7f68cc13e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68cc13dfa0 0x7f68cc13e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 -- 172.21.15.83:0/3229500595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc1471c0 con 0x7f68b4003a60 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d1d7f640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f68bc01aa60 tx=0x7f68b4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0013c0 con 0x7f68b4003a60 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc005480 con 0x7f68b4003a60 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc13b700 con 0x7f68b4003a60 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b4430 con 0x7f68b4003a60 2024-05-17T22:06:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1f6fa0 con 0x7f68b4003a60 2024-05-17T22:06:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc1397d0 con 0x7f68b4003a60 2024-05-17T22:06:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f68bc0b45d0 con 0x7f68b4003a60 2024-05-17T22:06:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 0x7f68b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.871+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f68bc129740 con 0x7f68b4003a60 2024-05-17T22:06:57.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.875+0000 7f68d157e640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 0x7f68b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:57.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.875+0000 7f68d157e640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 0x7f68b8071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f68c0001870 tx=0x7f68c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:57.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:57.879+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f68cc1471c0 con 0x7f68b4003a60 2024-05-17T22:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.059+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68cc161a50 con 0x7f68b4003a60 2024-05-17T22:06:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.059+0000 7f68ca7fc640 1 -- 172.21.15.83:0/3229500595 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+4350 (secure 0 0 0) 0x7f68bc0b8030 con 0x7f68b4003a60 2024-05-17T22:06:58.052 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:58.053 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:57.366262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 88 2024-05-17T22:06:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 msgr2=0x7f68b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 0x7f68b8071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f68c0001870 tx=0x7f68c0009040 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 msgr2=0x7f68cc1406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f68bc01aa60 tx=0x7f68b4005840 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 shutdown_connections 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68cc146870 0x7f68cc146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f68b806f400 0x7f68b8071910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68cc13dfa0 0x7f68cc13e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 --2- 172.21.15.83:0/3229500595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b4003a60 0x7f68cc1406b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 >> 172.21.15.83:0/3229500595 conn(0x7f68cc06cd90 msgr2=0x7f68cc0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 shutdown_connections 2024-05-17T22:06:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.063+0000 7f68d3fe3640 1 -- 172.21.15.83:0/3229500595 wait complete. 2024-05-17T22:06:58.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:58.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 Processor -- start 2024-05-17T22:06:58.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 -- start start 2024-05-17T22:06:58.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d181602e0 0x7f9d1806c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1809bcc0 0x7f9d1806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 0x7f9d181638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9d18160850 con 0x7f9d181602e0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9d18160b30 con 0x7f9d1815d9d0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1e405640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d18160e10 con 0x7f9d1809bcc0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d181602e0 0x7f9d1806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d181602e0 0x7f9d1806c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53312/0 (socket says 172.21.15.83:53312) 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 -- 172.21.15.83:0/2177412515 learned_addr learned my addr 172.21.15.83:0/2177412515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1c9a2640 1 --2- 172.21.15.83:0/2177412515 >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 0x7f9d181638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d177fe640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1809bcc0 0x7f9d1806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 -- 172.21.15.83:0/2177412515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1809bcc0 msgr2=0x7f9d1806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1809bcc0 0x7f9d1806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 -- 172.21.15.83:0/2177412515 >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 msgr2=0x7f9d181638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 --2- 172.21.15.83:0/2177412515 >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 0x7f9d181638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d17fff640 1 -- 172.21.15.83:0/2177412515 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d181610f0 con 0x7f9d181602e0 2024-05-17T22:06:58.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.231+0000 7f9d1c9a2640 1 --2- 172.21.15.83:0/2177412515 >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 0x7f9d181638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:58.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.235+0000 7f9d17fff640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1806c790 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9d1806d840 tx=0x7f9d0801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e478451f962e879 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.235+0000 7f9d16ffd640 1 -- 172.21.15.83:0/2177412515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080b0070 con 0x7f9d181602e0 2024-05-17T22:06:58.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.235+0000 7f9d16ffd640 1 -- 172.21.15.83:0/2177412515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d080ab600 con 0x7f9d181602e0 2024-05-17T22:06:58.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.235+0000 7f9d16ffd640 1 -- 172.21.15.83:0/2177412515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080ab9d0 con 0x7f9d181602e0 2024-05-17T22:06:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- 172.21.15.83:0/2177412515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 msgr2=0x7f9d1806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1806c790 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9d1806d840 tx=0x7f9d0801c260 comp rx=0 tx=0).stop 2024-05-17T22:06:58.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- 172.21.15.83:0/2177412515 shutdown_connections 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- 172.21.15.83:0/2177412515 >> v2:172.21.15.155:3300/0 conn(0x7f9d1815d9d0 0x7f9d181638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1809bcc0 0x7f9d1806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- 172.21.15.83:0/2177412515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1806c790 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- 172.21.15.83:0/2177412515 >> 172.21.15.83:0/2177412515 conn(0x7f9d1809b8b0 msgr2=0x7f9d180b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- 172.21.15.83:0/2177412515 shutdown_connections 2024-05-17T22:06:58.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- 172.21.15.83:0/2177412515 wait complete. 2024-05-17T22:06:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 Processor -- start 2024-05-17T22:06:58.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- start start 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1809bcc0 0x7f9d181582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1815d9d0 0x7f9d1809c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180b0360 con 0x7f9d181602e0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180b01e0 con 0x7f9d1809bcc0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1e405640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d180b04e0 con 0x7f9d1815d9d0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1c9a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1c9a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53316/0 (socket says 172.21.15.83:53316) 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.239+0000 7f9d1c9a2640 1 -- 172.21.15.83:0/1462987532 learned_addr learned my addr 172.21.15.83:0/1462987532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d17fff640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1809bcc0 0x7f9d181582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 -- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1815d9d0 msgr2=0x7f9d1809c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d177fe640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1815d9d0 0x7f9d1809c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1815d9d0 0x7f9d1809c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 -- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1809bcc0 msgr2=0x7f9d181582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1809bcc0 0x7f9d181582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 -- 172.21.15.83:0/1462987532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1809d0f0 con 0x7f9d181602e0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1c9a2640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d10007d90 tx=0x7f9d1000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1001a070 con 0x7f9d181602e0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d100039a0 con 0x7f9d181602e0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1815a850 con 0x7f9d181602e0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10003d90 con 0x7f9d181602e0 2024-05-17T22:06:58.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1815add0 con 0x7f9d181602e0 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d180b0940 con 0x7f9d181602e0 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d1001d070 con 0x7f9d181602e0 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 0x7f9d04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d17fff640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 0x7f9d04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f9d1008faa0 con 0x7f9d181602e0 2024-05-17T22:06:58.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.243+0000 7f9d17fff640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 0x7f9d04071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9d080ad000 tx=0x7f9d08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.247+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9d10024080 con 0x7f9d181602e0 2024-05-17T22:06:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.447+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d18158830 con 0x7f9d181602e0 2024-05-17T22:06:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.447+0000 7f9d14ff9640 1 -- 172.21.15.83:0/1462987532 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5175 (secure 0 0 0) 0x7f9d10010070 con 0x7f9d181602e0 2024-05-17T22:06:58.440 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:58.440 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2307755165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2307755165},{"type":"v1","addr":"172.21.15.83:6836","nonce":2307755165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:57.366262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 89 2024-05-17T22:06:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 msgr2=0x7f9d04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 0x7f9d04071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9d080ad000 tx=0x7f9d08008040 comp rx=0 tx=0).stop 2024-05-17T22:06:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 msgr2=0x7f9d1809cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d10007d90 tx=0x7f9d1000d4a0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 shutdown_connections 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d181602e0 0x7f9d1809cac0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d1815d9d0 0x7f9d1809c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d0406f400 0x7f9d04071910 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 --2- 172.21.15.83:0/1462987532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1809bcc0 0x7f9d181582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 >> 172.21.15.83:0/1462987532 conn(0x7f9d1809b8b0 msgr2=0x7f9d180701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 shutdown_connections 2024-05-17T22:06:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.451+0000 7f9d1e405640 1 -- 172.21.15.83:0/1462987532 wait complete. 2024-05-17T22:06:58.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:06:58.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 Processor -- start 2024-05-17T22:06:58.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 -- start start 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c1461c0 0x7fea5c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea5c0706e0 0x7fea5c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c1465d0 con 0x7fea5c15e6b0 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c06ef30 con 0x7fea5c0706e0 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea6363e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c06f210 con 0x7fea5c1461c0 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea61bdb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea5c0706e0 0x7fea5c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea60bd9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fea5c15e6b0 0x7fea5c06ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49830/0 (socket says 172.21.15.83:49830) 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea60bd9640 1 -- 172.21.15.83:0/4165506760 learned_addr learned my addr 172.21.15.83:0/4165506760 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea613da640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c1461c0 0x7fea5c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea61bdb640 1 -- 172.21.15.83:0/4165506760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c1461c0 msgr2=0x7fea5c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea61bdb640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c1461c0 0x7fea5c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea61bdb640 1 -- 172.21.15.83:0/4165506760 >> v1:172.21.15.83:6789/0 conn(0x7fea5c15e6b0 legacy=0x7fea5c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea53fff640 1 -- 172.21.15.83:0/4165506760 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1864458150 0 0) 0x7fea5c1465d0 con 0x7fea5c15e6b0 2024-05-17T22:06:58.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.615+0000 7fea61bdb640 1 -- 172.21.15.83:0/4165506760 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c06f4f0 con 0x7fea5c0706e0 2024-05-17T22:06:58.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea61bdb640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c1605a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fea54009d90 tx=0x7fea5401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608781ec3ea8fd7a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea53fff640 1 -- 172.21.15.83:0/4165506760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540b2070 con 0x7fea5c0706e0 2024-05-17T22:06:58.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea53fff640 1 -- 172.21.15.83:0/4165506760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea540a5910 con 0x7fea5c0706e0 2024-05-17T22:06:58.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea53fff640 1 -- 172.21.15.83:0/4165506760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540a5ce0 con 0x7fea5c0706e0 2024-05-17T22:06:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 -- 172.21.15.83:0/4165506760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 msgr2=0x7fea5c1605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c1605a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fea54009d90 tx=0x7fea5401c940 comp rx=0 tx=0).stop 2024-05-17T22:06:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 -- 172.21.15.83:0/4165506760 shutdown_connections 2024-05-17T22:06:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c1605a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 --2- 172.21.15.83:0/4165506760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c1461c0 0x7fea5c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 -- 172.21.15.83:0/4165506760 >> 172.21.15.83:0/4165506760 conn(0x7fea5c06cd90 msgr2=0x7fea5c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 -- 172.21.15.83:0/4165506760 shutdown_connections 2024-05-17T22:06:58.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.619+0000 7fea6363e640 1 -- 172.21.15.83:0/4165506760 wait complete. 2024-05-17T22:06:58.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 Processor -- start 2024-05-17T22:06:58.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- start start 2024-05-17T22:06:58.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 0x7fea5c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c15e6b0 0x7fea5c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c072e30 con 0x7fea5c0706e0 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0d12b0 con 0x7fea5c1461c0 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c072fb0 con 0x7fea5c15e6b0 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53338/0 (socket says 172.21.15.83:53338) 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 -- 172.21.15.83:0/1015690317 learned_addr learned my addr 172.21.15.83:0/1015690317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea60bd9640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 0x7fea5c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 -- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c15e6b0 msgr2=0x7fea5c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c15e6b0 0x7fea5c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 -- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 msgr2=0x7fea5c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 0x7fea5c159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 -- 172.21.15.83:0/1015690317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1596a0 con 0x7fea5c0706e0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea60bd9640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 0x7fea5c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea613da640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d5d0 tx=0x7fea5800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58004240 con 0x7fea5c0706e0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea580043e0 con 0x7fea5c0706e0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c0b1220 con 0x7fea5c0706e0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58003840 con 0x7fea5c0706e0 2024-05-17T22:06:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c0b6020 con 0x7fea5c0706e0 2024-05-17T22:06:58.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.623+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c1605a0 con 0x7fea5c0706e0 2024-05-17T22:06:58.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.627+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fea580039e0 con 0x7fea5c0706e0 2024-05-17T22:06:58.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.627+0000 7fea51ffb640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 0x7fea40071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.627+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7fea5808f620 con 0x7fea5c0706e0 2024-05-17T22:06:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.627+0000 7fea60bd9640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 0x7fea40071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.627+0000 7fea60bd9640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 0x7fea40071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fea5c139840 tx=0x7fea4c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.631+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fea5805c4d0 con 0x7fea5c0706e0 2024-05-17T22:06:58.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.787+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea5c0b41a0 con 0x7fea5c0706e0 2024-05-17T22:06:58.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea51ffb640 1 -- 172.21.15.83:0/1015690317 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16841 (secure 0 0 0) 0x7fea5805bdd0 con 0x7fea5c0706e0 2024-05-17T22:06:58.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:58.783 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":98,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:06:57.365899+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:05:24.506575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-17T22:05:25.749261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6834/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6836/1996242614":"2024-05-18T22:06:20.560861+0000","172.21.15.83:6834/1996242614":"2024-05-18T22:06:20.560861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 msgr2=0x7fea40071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 0x7fea40071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fea5c139840 tx=0x7fea4c00b040 comp rx=0 tx=0).stop 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 msgr2=0x7fea5c0b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d5d0 tx=0x7fea5800da90 comp rx=0 tx=0).stop 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 shutdown_connections 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea5c15e6b0 0x7fea5c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fea4006f430 0x7fea40071940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea5c1461c0 0x7fea5c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 --2- 172.21.15.83:0/1015690317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea5c0706e0 0x7fea5c0b54c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 >> 172.21.15.83:0/1015690317 conn(0x7fea5c06cd90 msgr2=0x7fea5c1441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 shutdown_connections 2024-05-17T22:06:58.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.791+0000 7fea6363e640 1 -- 172.21.15.83:0/1015690317 wait complete. 2024-05-17T22:06:58.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:06:58.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 Processor -- start 2024-05-17T22:06:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 -- start start 2024-05-17T22:06:58.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd0bc16cde0 0x7fd0bc16f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0bc164960 con 0x7fd0bc072e30 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0bc164c40 con 0x7fd0bc161d50 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c2b93640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd0bc164f20 con 0x7fd0bc16cde0 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bbfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd0bc072e30 0x7fd0bc073240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49836/0 (socket says 172.21.15.83:49836) 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bbfff640 1 -- 172.21.15.83:0/3259552252 learned_addr learned my addr 172.21.15.83:0/3259552252 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820678598 0 0) 0x7fd0bc164960 con 0x7fd0bc072e30 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0c1130640 1 --2- 172.21.15.83:0/3259552252 >> v2:172.21.15.155:3300/0 conn(0x7fd0bc16cde0 0x7fd0bc16f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0a4003530 con 0x7fd0bc072e30 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 834155557 0 0) 0x7fd0a4003530 con 0x7fd0bc072e30 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 >> v2:172.21.15.155:3300/0 conn(0x7fd0bc16cde0 msgr2=0x7fd0bc16f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 --2- 172.21.15.83:0/3259552252 >> v2:172.21.15.155:3300/0 conn(0x7fd0bc16cde0 0x7fd0bc16f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 >> v1:172.21.15.155:6789/0 conn(0x7fd0bc161d50 legacy=0x7fd0bc0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc165200 con 0x7fd0bc072e30 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fd0b0001700 con 0x7fd0bc072e30 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 >> v1:172.21.15.83:6789/0 conn(0x7fd0bc072e30 legacy=0x7fd0bc073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a4005310 0x7fd0a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc164f20 con 0x7fd0a4004810 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a400c2b0 con 0x7fd0a4003a60 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a400c4b0 con 0x7fd0a4005310 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.955+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c092f640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 -- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a4005310 msgr2=0x7fd0a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0a4005310 0x7fd0a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 -- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a4003a60 msgr2=0x7fd0a4003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:58.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0a4003a60 0x7fd0a4003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c1130640 1 -- 172.21.15.83:0/3259552252 reap_dead start 2024-05-17T22:06:58.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 -- 172.21.15.83:0/3259552252 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a4003530 con 0x7fd0a4004810 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c240 tx=0x7fd0b001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f5284560798ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0007cf0 con 0x7fd0a4004810 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0005ae0 con 0x7fd0a4004810 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0bb7fe640 1 -- 172.21.15.83:0/3259552252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0007070 con 0x7fd0a4004810 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 -- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 msgr2=0x7fd0a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c240 tx=0x7fd0b001f330 comp rx=0 tx=0).stop 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 -- 172.21.15.83:0/3259552252 shutdown_connections 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/3259552252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0a4004c20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 -- 172.21.15.83:0/3259552252 >> 172.21.15.83:0/3259552252 conn(0x7fd0bc06cd90 msgr2=0x7fd0bc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 -- 172.21.15.83:0/3259552252 shutdown_connections 2024-05-17T22:06:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.959+0000 7fd0c2b93640 1 -- 172.21.15.83:0/3259552252 wait complete. 2024-05-17T22:06:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 Processor -- start 2024-05-17T22:06:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- start start 2024-05-17T22:06:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0bc0aa310 0x7fd0bc1fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0bc16b8a0 0x7fd0bc16bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc0b4750 con 0x7fd0a4004810 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd0bc0b45d0 con 0x7fd0bc0aa310 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd0bc073240 con 0x7fd0bc16b8a0 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53350/0 (socket says 172.21.15.83:53350) 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 -- 172.21.15.83:0/2732345660 learned_addr learned my addr 172.21.15.83:0/2732345660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0bc0aa310 0x7fd0bc1fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 -- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0bc16b8a0 msgr2=0x7fd0bc16bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c1130640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0bc16b8a0 0x7fd0bc16bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0bc16b8a0 0x7fd0bc16bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 -- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0bc0aa310 msgr2=0x7fd0bc1fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0bc0aa310 0x7fd0bc1fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 -- 172.21.15.83:0/2732345660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0bc0b23c0 con 0x7fd0a4004810 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c092f640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400e840 tx=0x7fd0b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4003b90 con 0x7fd0a4004810 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b4018070 con 0x7fd0a4004810 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0bc16a800 con 0x7fd0a4004810 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40050d0 con 0x7fd0a4004810 2024-05-17T22:06:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.963+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0bc16adc0 con 0x7fd0a4004810 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0bc073570 con 0x7fd0a4004810 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd0b4005270 con 0x7fd0a4004810 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0b97fa640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 0x7fd0a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 0x7fd0a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7fd0b408f440 con 0x7fd0a4004810 2024-05-17T22:06:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.967+0000 7fd0bbfff640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 0x7fd0a8071910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00ae000 tx=0x7fd0b0007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:58.971+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd0b405c1e0 con 0x7fd0a4004810 2024-05-17T22:06:59.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0bc070150 con 0x7fd0a4004810 2024-05-17T22:06:59.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0b97fa640 1 -- 172.21.15.83:0/2732345660 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5175 (secure 0 0 0) 0x7fd0b405bae0 con 0x7fd0a4004810 2024-05-17T22:06:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:59.148 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2307755165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2307755165},{"type":"v1","addr":"172.21.15.83:6836","nonce":2307755165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:57.366262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:59.148 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 89 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 msgr2=0x7fd0a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 0x7fd0a8071910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00ae000 tx=0x7fd0b0007040 comp rx=0 tx=0).stop 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 msgr2=0x7fd0bc161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400e840 tx=0x7fd0b400ed00 comp rx=0 tx=0).stop 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 shutdown_connections 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd0bc16b8a0 0x7fd0bc16bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd0a806f400 0x7fd0a8071910 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0bc0aa310 0x7fd0bc1fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 --2- 172.21.15.83:0/2732345660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0a4004810 0x7fd0bc161040 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.155+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 >> 172.21.15.83:0/2732345660 conn(0x7fd0bc06cd90 msgr2=0x7fd0bc167460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.159+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 shutdown_connections 2024-05-17T22:06:59.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.159+0000 7fd0c2b93640 1 -- 172.21.15.83:0/2732345660 wait complete. 2024-05-17T22:06:59.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:06:59.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 Processor -- start 2024-05-17T22:06:59.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 -- start start 2024-05-17T22:06:59.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdbb00d1040 0x7fdbb0144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdbb015e080 0x7fdbb0072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb015ea40 con 0x7fdbb015e490 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb015ed20 con 0x7fdbb00d1040 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbb55e1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb015f000 con 0x7fdbb015e080 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdbb00d1040 0x7fdbb0144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdbb00d1040 0x7fdbb0144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53354/0 (socket says 172.21.15.83:53354) 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 -- 172.21.15.83:0/3813338561 learned_addr learned my addr 172.21.15.83:0/3813338561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 -- 172.21.15.83:0/3813338561 >> v2:172.21.15.155:3300/0 conn(0x7fdbb015e080 msgr2=0x7fdbb0072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbae7fc640 1 --2- 172.21.15.83:0/3813338561 >> v2:172.21.15.155:3300/0 conn(0x7fdbb015e080 0x7fdbb0072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 --2- 172.21.15.83:0/3813338561 >> v2:172.21.15.155:3300/0 conn(0x7fdbb015e080 0x7fdbb0072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 -- 172.21.15.83:0/3813338561 >> v1:172.21.15.155:6789/0 conn(0x7fdbb015e490 legacy=0x7fdbb01616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:59.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.319+0000 7fdbaeffd640 1 -- 172.21.15.83:0/3813338561 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb006ef30 con 0x7fdbb00d1040 2024-05-17T22:06:59.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbaeffd640 1 --2- 172.21.15.83:0/3813338561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb00d1040 0x7fdbb0144080 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00b6b10 tx=0x7fdba801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ba7a398876d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbadffb640 1 -- 172.21.15.83:0/3813338561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80ab5f0 con 0x7fdbb00d1040 2024-05-17T22:06:59.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbadffb640 1 -- 172.21.15.83:0/3813338561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba80b0070 con 0x7fdbb00d1040 2024-05-17T22:06:59.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbadffb640 1 -- 172.21.15.83:0/3813338561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80ab9b0 con 0x7fdbb00d1040 2024-05-17T22:06:59.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3813338561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb00d1040 msgr2=0x7fdbb0144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3813338561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb00d1040 0x7fdbb0144080 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00b6b10 tx=0x7fdba801c520 comp rx=0 tx=0).stop 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3813338561 shutdown_connections 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3813338561 >> v2:172.21.15.155:3300/0 conn(0x7fdbb015e080 0x7fdbb0072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3813338561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb00d1040 0x7fdbb0144080 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3813338561 >> 172.21.15.83:0/3813338561 conn(0x7fdbb006cd90 msgr2=0x7fdbb00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3813338561 shutdown_connections 2024-05-17T22:06:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.323+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3813338561 wait complete. 2024-05-17T22:06:59.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 Processor -- start 2024-05-17T22:06:59.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 -- start start 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00d1040 0x7fdbb0142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 0x7fdbb00b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb016abe0 con 0x7fdbb015e080 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb016aa60 con 0x7fdbb00d1040 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbb55e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb016ad60 con 0x7fdbb015e490 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.327+0000 7fdbae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53370/0 (socket says 172.21.15.83:53370) 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 -- 172.21.15.83:0/3627249217 learned_addr learned my addr 172.21.15.83:0/3627249217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbaeffd640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00d1040 0x7fdbb0142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbaf7fe640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 0x7fdbb00b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 -- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 msgr2=0x7fdbb00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 0x7fdbb00b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 -- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00d1040 msgr2=0x7fdbb0142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00d1040 0x7fdbb0142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 -- 172.21.15.83:0/3627249217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01439f0 con 0x7fdbb015e080 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbaf7fe640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 0x7fdbb00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbae7fc640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c7c0 tx=0x7fdb9800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98015070 con 0x7fdbb015e080 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb98004220 con 0x7fdbb015e080 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb0143150 con 0x7fdbb015e080 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003620 con 0x7fdbb015e080 2024-05-17T22:06:59.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb0143710 con 0x7fdbb015e080 2024-05-17T22:06:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb0144080 con 0x7fdbb015e080 2024-05-17T22:06:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb980037c0 con 0x7fdbb015e080 2024-05-17T22:06:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 0x7fdb90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbaeffd640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 0x7fdb90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7fdb9808f000 con 0x7fdbb015e080 2024-05-17T22:06:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.331+0000 7fdbaeffd640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 0x7fdb90071910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdba80af000 tx=0x7fdba80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.335+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fdb9805b7f0 con 0x7fdbb015e080 2024-05-17T22:06:59.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb013dec0 con 0x7fdbb015e080 2024-05-17T22:06:59.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdba77fe640 1 -- 172.21.15.83:0/3627249217 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5175 (secure 0 0 0) 0x7fdb9805faf0 con 0x7fdbb015e080 2024-05-17T22:06:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:59.516 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2307755165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2307755165},{"type":"v1","addr":"172.21.15.83:6836","nonce":2307755165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:05:26.778012+0000","modified":"2024-05-17T22:06:57.366262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-17T22:06:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 89 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 msgr2=0x7fdb90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 0x7fdb90071910 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdba80af000 tx=0x7fdba80a7000 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 msgr2=0x7fdbb015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c7c0 tx=0x7fdb9800cc80 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 shutdown_connections 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb015e490 0x7fdbb00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb015e080 0x7fdbb015d330 secure :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c7c0 tx=0x7fdb9800cc80 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb9006f400 0x7fdb90071910 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 --2- 172.21.15.83:0/3627249217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00d1040 0x7fdbb0142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 >> 172.21.15.83:0/3627249217 conn(0x7fdbb006cd90 msgr2=0x7fdbb0141050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 shutdown_connections 2024-05-17T22:06:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.523+0000 7fdbb55e1640 1 -- 172.21.15.83:0/3627249217 wait complete. 2024-05-17T22:06:59.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:06:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.675+0000 7f52afd42640 1 Processor -- start 2024-05-17T22:06:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.675+0000 7f52afd42640 1 -- start start 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52afd42640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52a81461c0 0x7f52a815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52afd42640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a81465d0 con 0x7f52a80706e0 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52afd42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a806ef30 con 0x7f52a815e6b0 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52afd42640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f52a806f210 con 0x7f52a81461c0 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52adade640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52a81461c0 0x7f52a815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52ae2df640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f52a80706e0 0x7f52a81605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49842/0 (socket says 172.21.15.83:49842) 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52ae2df640 1 -- 172.21.15.83:0/1493716237 learned_addr learned my addr 172.21.15.83:0/1493716237 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52adade640 1 -- 172.21.15.83:0/1493716237 >> v1:172.21.15.155:6789/0 conn(0x7f52a815e6b0 legacy=0x7f52a806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52adade640 1 -- 172.21.15.83:0/1493716237 >> v1:172.21.15.83:6789/0 conn(0x7f52a80706e0 legacy=0x7f52a81605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52acadc640 1 -- 172.21.15.83:0/1493716237 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807163818 0 0) 0x7f52a81465d0 con 0x7f52a80706e0 2024-05-17T22:06:59.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.679+0000 7f52adade640 1 -- 172.21.15.83:0/1493716237 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a806f4f0 con 0x7f52a81461c0 2024-05-17T22:06:59.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52adade640 1 --2- 172.21.15.83:0/1493716237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81461c0 0x7f52a815e170 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f52a40097c0 tx=0x7f52a401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f378ed8642844842 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52acadc640 1 -- 172.21.15.83:0/1493716237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40a7070 con 0x7f52a81461c0 2024-05-17T22:06:59.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52acadc640 1 -- 172.21.15.83:0/1493716237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a40b0070 con 0x7f52a81461c0 2024-05-17T22:06:59.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52acadc640 1 -- 172.21.15.83:0/1493716237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40ab890 con 0x7f52a81461c0 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- 172.21.15.83:0/1493716237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81461c0 msgr2=0x7f52a815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 --2- 172.21.15.83:0/1493716237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81461c0 0x7f52a815e170 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f52a40097c0 tx=0x7f52a401c470 comp rx=0 tx=0).stop 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- 172.21.15.83:0/1493716237 shutdown_connections 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 --2- 172.21.15.83:0/1493716237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a81461c0 0x7f52a815e170 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- 172.21.15.83:0/1493716237 >> 172.21.15.83:0/1493716237 conn(0x7f52a806cd90 msgr2=0x7f52a80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- 172.21.15.83:0/1493716237 shutdown_connections 2024-05-17T22:06:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- 172.21.15.83:0/1493716237 wait complete. 2024-05-17T22:06:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 Processor -- start 2024-05-17T22:06:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.683+0000 7f52afd42640 1 -- start start 2024-05-17T22:06:59.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52a80706e0 0x7f52a813d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52a81461c0 0x7f52a80b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80d10a0 con 0x7f52a815e6b0 2024-05-17T22:06:59.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8160ec0 con 0x7f52a81461c0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f52a80d1220 con 0x7f52a80706e0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53380/0 (socket says 172.21.15.83:53380) 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 -- 172.21.15.83:0/2144598628 learned_addr learned my addr 172.21.15.83:0/2144598628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ad2dd640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52a81461c0 0x7f52a80b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52adade640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52a80706e0 0x7f52a813d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 -- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52a80706e0 msgr2=0x7f52a813d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52a80706e0 0x7f52a813d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 -- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52a81461c0 msgr2=0x7f52a80b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52a81461c0 0x7f52a80b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 -- 172.21.15.83:0/2144598628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a81401e0 con 0x7f52a815e6b0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52ae2df640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c6e0 tx=0x7f52a000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0018070 con 0x7f52a815e6b0 2024-05-17T22:06:59.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0004d70 con 0x7f52a815e6b0 2024-05-17T22:06:59.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000a880 con 0x7f52a815e6b0 2024-05-17T22:06:59.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a8140480 con 0x7f52a815e6b0 2024-05-17T22:06:59.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a8140a40 con 0x7f52a815e6b0 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a815e170 con 0x7f52a815e6b0 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.687+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52a001b070 con 0x7f52a815e6b0 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.691+0000 7f529e7fc640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 0x7f528c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.691+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f52a008e8f0 con 0x7f52a815e6b0 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.691+0000 7f52adade640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 0x7f528c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:06:59.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.691+0000 7f52adade640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 0x7f528c071910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f52a40af000 tx=0x7f52a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:06:59.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.695+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f52a005b730 con 0x7f52a815e6b0 2024-05-17T22:06:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.851+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52a80b41a0 con 0x7f52a815e6b0 2024-05-17T22:06:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f529e7fc640 1 -- 172.21.15.83:0/2144598628 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16841 (secure 0 0 0) 0x7f52a005b030 con 0x7f52a815e6b0 2024-05-17T22:06:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:06:59.847 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":98,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:06:57.365899+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:05:24.506575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-17T22:05:25.749261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6834/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6836/1996242614":"2024-05-18T22:06:20.560861+0000","172.21.15.83:6834/1996242614":"2024-05-18T22:06:20.560861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 msgr2=0x7f528c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 0x7f528c071910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f52a40af000 tx=0x7f52a4008040 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 msgr2=0x7f52a80b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c6e0 tx=0x7f52a000cba0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 shutdown_connections 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52a815e6b0 0x7f52a80b5b10 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f528c06f400 0x7f528c071910 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52a81461c0 0x7f52a80b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 --2- 172.21.15.83:0/2144598628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52a80706e0 0x7f52a813d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 >> 172.21.15.83:0/2144598628 conn(0x7f52a806cd90 msgr2=0x7f52a81437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:06:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 shutdown_connections 2024-05-17T22:06:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:06:59.855+0000 7f52afd42640 1 -- 172.21.15.83:0/2144598628 wait complete. 2024-05-17T22:06:59.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:07:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.023+0000 7f6aa5ce2640 1 Processor -- start 2024-05-17T22:07:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.023+0000 7f6aa5ce2640 1 -- start start 2024-05-17T22:07:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6aa015dcb0 0x7f6aa00b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160650 0x7f6aa0160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6aa0160fa0 0x7f6aa015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa00b6110 con 0x7f6aa0160fa0 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa00b65a0 con 0x7f6aa015dcb0 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6aa5ce2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa00b5500 con 0x7f6aa0160650 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6aa0160fa0 0x7f6aa015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6aa0160fa0 0x7f6aa015cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53390/0 (socket says 172.21.15.83:53390) 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 -- 172.21.15.83:0/2425945263 learned_addr learned my addr 172.21.15.83:0/2425945263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9effd640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160650 0x7f6aa0160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9f7fe640 1 --2- 172.21.15.83:0/2425945263 >> v2:172.21.15.155:3300/0 conn(0x7f6aa015dcb0 0x7f6aa00b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 -- 172.21.15.83:0/2425945263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160650 msgr2=0x7f6aa0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160650 0x7f6aa0160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 -- 172.21.15.83:0/2425945263 >> v2:172.21.15.155:3300/0 conn(0x7f6aa015dcb0 msgr2=0x7f6aa00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 --2- 172.21.15.83:0/2425945263 >> v2:172.21.15.155:3300/0 conn(0x7f6aa015dcb0 0x7f6aa00b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 -- 172.21.15.83:0/2425945263 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa00b57e0 con 0x7f6aa0160fa0 2024-05-17T22:07:00.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.027+0000 7f6a9ffff640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa0160fa0 0x7f6aa015cb90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6aa006f220 tx=0x7f6a9801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a848ad9bd5f76ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2425945263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800b2d0 con 0x7f6aa0160fa0 2024-05-17T22:07:00.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2425945263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9800b470 con 0x7f6aa0160fa0 2024-05-17T22:07:00.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6a9e7fc640 1 -- 172.21.15.83:0/2425945263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a980a7b90 con 0x7f6aa0160fa0 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/2425945263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa0160fa0 msgr2=0x7f6aa015cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa0160fa0 0x7f6aa015cb90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6aa006f220 tx=0x7f6a9801ebf0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/2425945263 shutdown_connections 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa0160fa0 0x7f6aa015cb90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/2425945263 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160650 0x7f6aa0160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/2425945263 >> v2:172.21.15.155:3300/0 conn(0x7f6aa015dcb0 0x7f6aa00b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/2425945263 >> 172.21.15.83:0/2425945263 conn(0x7f6aa009b8b0 msgr2=0x7f6aa00b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/2425945263 shutdown_connections 2024-05-17T22:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/2425945263 wait complete. 2024-05-17T22:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 Processor -- start 2024-05-17T22:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.031+0000 7f6aa5ce2640 1 -- start start 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0160650 0x7f6aa015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160fa0 0x7f6aa015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa006c790 con 0x7f6aa015dcb0 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01613b0 con 0x7f6aa0160650 2024-05-17T22:07:00.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa006c910 con 0x7f6aa0160fa0 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53400/0 (socket says 172.21.15.83:53400) 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 -- 172.21.15.83:0/1690912237 learned_addr learned my addr 172.21.15.83:0/1690912237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9effd640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0160650 0x7f6aa015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9ffff640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160fa0 0x7f6aa015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 -- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160fa0 msgr2=0x7f6aa015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160fa0 0x7f6aa015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 -- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0160650 msgr2=0x7f6aa015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0160650 0x7f6aa015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 -- 172.21.15.83:0/1690912237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa00d0410 con 0x7f6aa015dcb0 2024-05-17T22:07:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9f7fe640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d620 tx=0x7f6a9000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90004240 con 0x7f6aa015dcb0 2024-05-17T22:07:00.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a900043e0 con 0x7f6aa015dcb0 2024-05-17T22:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa00d06f0 con 0x7f6aa015dcb0 2024-05-17T22:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90003620 con 0x7f6aa015dcb0 2024-05-17T22:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa00d0cb0 con 0x7f6aa015dcb0 2024-05-17T22:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa0160a60 con 0x7f6aa015dcb0 2024-05-17T22:07:00.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6a9001b020 con 0x7f6aa015dcb0 2024-05-17T22:07:00.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 0x7f6a8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a9effd640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 0x7f6a8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.035+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f6a9008f0d0 con 0x7f6aa015dcb0 2024-05-17T22:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.039+0000 7f6a9effd640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 0x7f6a8c071910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015f280 tx=0x7f6a94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.039+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6a9005bf10 con 0x7f6aa015dcb0 2024-05-17T22:07:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.231+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6aa014e940 con 0x7f6aa015dcb0 2024-05-17T22:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.375+0000 7f6a87fff640 1 -- 172.21.15.83:0/1690912237 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f6a9005b810 con 0x7f6aa015dcb0 2024-05-17T22:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:07:00.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 msgr2=0x7f6a8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 0x7f6a8c071910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015f280 tx=0x7f6a94008040 comp rx=0 tx=0).stop 2024-05-17T22:07:00.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 msgr2=0x7f6aa014e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d620 tx=0x7f6a9000dae0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 shutdown_connections 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa0160fa0 0x7f6aa015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a8c06f400 0x7f6a8c071910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0160650 0x7f6aa015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 --2- 172.21.15.83:0/1690912237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6aa015dcb0 0x7f6aa014e400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 >> 172.21.15.83:0/1690912237 conn(0x7f6aa009b8b0 msgr2=0x7f6aa00b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 shutdown_connections 2024-05-17T22:07:00.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.379+0000 7f6aa5ce2640 1 -- 172.21.15.83:0/1690912237 wait complete. 2024-05-17T22:07:00.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:07:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 Processor -- start 2024-05-17T22:07:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 -- start start 2024-05-17T22:07:00.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 0x7f890c161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890c0af9c0 con 0x7f890c156c00 2024-05-17T22:07:00.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f890c0afca0 con 0x7f890c157010 2024-05-17T22:07:00.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.547+0000 7f89119b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0aff80 con 0x7f890c06c790 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 0x7f890c161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:59508/0 (socket says 172.21.15.83:59508) 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f8903fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f890c156c00 0x7f890c0b0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:42092/0 (socket says 172.21.15.83:42092) 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 -- 172.21.15.83:0/245062390 learned_addr learned my addr 172.21.15.83:0/245062390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 -- 172.21.15.83:0/245062390 >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 msgr2=0x7f890c161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 --2- 172.21.15.83:0/245062390 >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 0x7f890c161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 -- 172.21.15.83:0/245062390 >> v1:172.21.15.155:6789/0 conn(0x7f890c156c00 legacy=0x7f890c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 -- 172.21.15.83:0/245062390 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c0b0260 con 0x7f890c06c790 2024-05-17T22:07:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890b7fe640 1 --2- 172.21.15.83:0/245062390 >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 0x7f890c161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:07:00.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890affd640 1 --2- 172.21.15.83:0/245062390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098a0 tx=0x7f88fc01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d60fd385fd99e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890a7fc640 1 -- 172.21.15.83:0/245062390 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0ab620 con 0x7f890c06c790 2024-05-17T22:07:00.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890a7fc640 1 -- 172.21.15.83:0/245062390 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88fc0b0070 con 0x7f890c06c790 2024-05-17T22:07:00.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.551+0000 7f890a7fc640 1 -- 172.21.15.83:0/245062390 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0ab9e0 con 0x7f890c06c790 2024-05-17T22:07:00.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- 172.21.15.83:0/245062390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 msgr2=0x7f890c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 --2- 172.21.15.83:0/245062390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0098a0 tx=0x7f88fc01c510 comp rx=0 tx=0).stop 2024-05-17T22:07:00.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- 172.21.15.83:0/245062390 shutdown_connections 2024-05-17T22:07:00.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 --2- 172.21.15.83:0/245062390 >> v2:172.21.15.155:3300/0 conn(0x7f890c157010 0x7f890c161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 --2- 172.21.15.83:0/245062390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0af3e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- 172.21.15.83:0/245062390 >> 172.21.15.83:0/245062390 conn(0x7f890c09b8b0 msgr2=0x7f890c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:00.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- 172.21.15.83:0/245062390 shutdown_connections 2024-05-17T22:07:00.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- 172.21.15.83:0/245062390 wait complete. 2024-05-17T22:07:00.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 Processor -- start 2024-05-17T22:07:00.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.555+0000 7f89119b4640 1 -- start start 2024-05-17T22:07:00.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f890c156c00 0x7f890c149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d19a0 con 0x7f890c157010 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f890c0d1820 con 0x7f890c156c00 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f890c0d1b20 con 0x7f890c06c790 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53414/0 (socket says 172.21.15.83:53414) 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 -- 172.21.15.83:0/3479359367 learned_addr learned my addr 172.21.15.83:0/3479359367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8903fff640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f890c156c00 0x7f890c149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890affd640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 -- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 msgr2=0x7f890c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 -- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f890c156c00 msgr2=0x7f890c149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f890c156c00 0x7f890c149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 -- 172.21.15.83:0/3479359367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c1495b0 con 0x7f890c157010 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890b7fe640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e00 tx=0x7f890400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904018020 con 0x7f890c157010 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904014480 con 0x7f890c157010 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c0b1a10 con 0x7f890c157010 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904014870 con 0x7f890c157010 2024-05-17T22:07:00.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c0b1fd0 con 0x7f890c157010 2024-05-17T22:07:00.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c0af3e0 con 0x7f890c157010 2024-05-17T22:07:00.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f890401b020 con 0x7f890c157010 2024-05-17T22:07:00.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f8908ff9640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 0x7f88ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:00.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.559+0000 7f890affd640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 0x7f88ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:00.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.563+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f890408ff40 con 0x7f890c157010 2024-05-17T22:07:00.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.563+0000 7f890affd640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 0x7f88ec071940 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0af000 tx=0x7f88fc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:00.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.567+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8904022030 con 0x7f890c157010 2024-05-17T22:07:00.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:00.747+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f890c16bb20 con 0x7f890c157010 2024-05-17T22:07:01.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.387+0000 7f8908ff9640 1 -- 172.21.15.83:0/3479359367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7f8904010040 con 0x7f890c157010 2024-05-17T22:07:01.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 msgr2=0x7f88ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 0x7f88ec071940 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0af000 tx=0x7f88fc0a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 msgr2=0x7f890c1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e00 tx=0x7f890400c490 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 shutdown_connections 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f890c157010 0x7f890c1515a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f88ec06f430 0x7f88ec071940 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f890c156c00 0x7f890c149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 --2- 172.21.15.83:0/3479359367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f890c06c790 0x7f890c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 >> 172.21.15.83:0/3479359367 conn(0x7f890c09b8b0 msgr2=0x7f890c0b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 shutdown_connections 2024-05-17T22:07:01.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.391+0000 7f89119b4640 1 -- 172.21.15.83:0/3479359367 wait complete. 2024-05-17T22:07:01.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:07:01.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.563+0000 7f25ee303640 1 Processor -- start 2024-05-17T22:07:01.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.563+0000 7f25ee303640 1 -- start start 2024-05-17T22:07:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 0x7f25e815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f25e8072e50 0x7f25e8073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25e806ee70 0x7f25e806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f25e80b0e90 con 0x7f25e8072e50 2024-05-17T22:07:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f25e80737a0 con 0x7f25e806ee70 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ee303640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25e80cf050 con 0x7f25e8159da0 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f25e8072e50 0x7f25e8073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f25e8072e50 0x7f25e8073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53420/0 (socket says 172.21.15.83:53420) 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 -- 172.21.15.83:0/1377008970 learned_addr learned my addr 172.21.15.83:0/1377008970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25ec8a0640 1 --2- 172.21.15.83:0/1377008970 >> v2:172.21.15.155:3300/0 conn(0x7f25e806ee70 0x7f25e806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25e7fff640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 0x7f25e815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 -- 172.21.15.83:0/1377008970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 msgr2=0x7f25e815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 0x7f25e815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 -- 172.21.15.83:0/1377008970 >> v2:172.21.15.155:3300/0 conn(0x7f25e806ee70 msgr2=0x7f25e806f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 --2- 172.21.15.83:0/1377008970 >> v2:172.21.15.155:3300/0 conn(0x7f25e806ee70 0x7f25e806f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25dffff640 1 -- 172.21.15.83:0/1377008970 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e80cf2f0 con 0x7f25e8072e50 2024-05-17T22:07:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.567+0000 7f25e7fff640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 0x7f25e815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25dffff640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8073260 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f25e8070910 tx=0x7f25d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3b59ef882ef2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25e77fe640 1 -- 172.21.15.83:0/1377008970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00aa620 con 0x7f25e8072e50 2024-05-17T22:07:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25e77fe640 1 -- 172.21.15.83:0/1377008970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d00af070 con 0x7f25e8072e50 2024-05-17T22:07:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25e77fe640 1 -- 172.21.15.83:0/1377008970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00aa9e0 con 0x7f25e8072e50 2024-05-17T22:07:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- 172.21.15.83:0/1377008970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 msgr2=0x7f25e8073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8073260 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f25e8070910 tx=0x7f25d001c500 comp rx=0 tx=0).stop 2024-05-17T22:07:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- 172.21.15.83:0/1377008970 shutdown_connections 2024-05-17T22:07:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 --2- 172.21.15.83:0/1377008970 >> v2:172.21.15.155:3300/0 conn(0x7f25e806ee70 0x7f25e806f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8073260 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 --2- 172.21.15.83:0/1377008970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e8159da0 0x7f25e815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- 172.21.15.83:0/1377008970 >> 172.21.15.83:0/1377008970 conn(0x7f25e806cd90 msgr2=0x7f25e80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- 172.21.15.83:0/1377008970 shutdown_connections 2024-05-17T22:07:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- 172.21.15.83:0/1377008970 wait complete. 2024-05-17T22:07:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 Processor -- start 2024-05-17T22:07:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.571+0000 7f25ee303640 1 -- start start 2024-05-17T22:07:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e806ee70 0x7f25e814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e8159da0 0x7f25e80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8070520 con 0x7f25e8072e50 2024-05-17T22:07:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f25e80cf870 con 0x7f25e8159da0 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f25e806fbd0 con 0x7f25e806ee70 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53430/0 (socket says 172.21.15.83:53430) 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 -- 172.21.15.83:0/1903202288 learned_addr learned my addr 172.21.15.83:0/1903202288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e7fff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e806ee70 0x7f25e814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ec8a0640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e8159da0 0x7f25e80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 -- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e806ee70 msgr2=0x7f25e814fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e806ee70 0x7f25e814fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 -- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e8159da0 msgr2=0x7f25e80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e8159da0 0x7f25e80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 -- 172.21.15.83:0/1903202288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e814bc80 con 0x7f25e8072e50 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25dffff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f25d001abb0 tx=0x7f25d001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00aad80 con 0x7f25e8072e50 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d00bd3f0 con 0x7f25e8072e50 2024-05-17T22:07:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00bd7e0 con 0x7f25e8072e50 2024-05-17T22:07:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e80b1140 con 0x7f25e8072e50 2024-05-17T22:07:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e80b1700 con 0x7f25e8072e50 2024-05-17T22:07:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e80b0810 con 0x7f25e8072e50 2024-05-17T22:07:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25d00b33f0 con 0x7f25e8072e50 2024-05-17T22:07:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 0x7f25c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5881+0+0 (secure 0 0 0) 0x7f25d01291d0 con 0x7f25e8072e50 2024-05-17T22:07:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.575+0000 7f25e7fff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 0x7f25c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:01.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.579+0000 7f25e7fff640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 0x7f25c8071910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006fd0 tx=0x7f25d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.583+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f25e814bc80 con 0x7f25e8072e50 2024-05-17T22:07:01.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:07:01.736 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:07:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:01.743+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25e814aa50 con 0x7f25e8072e50 2024-05-17T22:07:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.403+0000 7f25e57fa640 1 -- 172.21.15.83:0/1903202288 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7f25d00160f0 con 0x7f25e8072e50 2024-05-17T22:07:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:07:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.407+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 msgr2=0x7f25c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.407+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 0x7f25c8071910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006fd0 tx=0x7f25d8008040 comp rx=0 tx=0).stop 2024-05-17T22:07:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.407+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 msgr2=0x7f25e8150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.407+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f25d001abb0 tx=0x7f25d001ad40 comp rx=0 tx=0).stop 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 shutdown_connections 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f25c806f400 0x7f25c8071910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25e8159da0 0x7f25e80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25e8072e50 0x7f25e8150250 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 --2- 172.21.15.83:0/1903202288 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25e806ee70 0x7f25e814fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 >> 172.21.15.83:0/1903202288 conn(0x7f25e806cd90 msgr2=0x7f25e8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 shutdown_connections 2024-05-17T22:07:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.411+0000 7f25ee303640 1 -- 172.21.15.83:0/1903202288 wait complete. 2024-05-17T22:07:02.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:07:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 Processor -- start 2024-05-17T22:07:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 -- start start 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff600b66c0 0x7eff600b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff6015df30 0x7eff6015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6015ec60 con 0x7eff60072e50 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7eff600b6230 con 0x7eff600b66c0 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff6823b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff600b5be0 con 0x7eff6015df30 2024-05-17T22:07:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff600b66c0 0x7eff600b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff600b66c0 0x7eff600b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53444/0 (socket says 172.21.15.83:53444) 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 -- 172.21.15.83:0/4102699332 learned_addr learned my addr 172.21.15.83:0/4102699332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff667d8640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff6015df30 0x7eff6015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 -- 172.21.15.83:0/4102699332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff6015df30 msgr2=0x7eff6015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff6015df30 0x7eff6015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 -- 172.21.15.83:0/4102699332 >> v1:172.21.15.155:6789/0 conn(0x7eff60072e50 legacy=0x7eff60073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.595+0000 7eff65fd7640 1 -- 172.21.15.83:0/4102699332 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff600b5ec0 con 0x7eff600b66c0 2024-05-17T22:07:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff65fd7640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff600b66c0 0x7eff600b0bc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009870 tx=0x7eff5c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73942cbf3bbcaca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff64fd5640 1 -- 172.21.15.83:0/4102699332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0a7070 con 0x7eff600b66c0 2024-05-17T22:07:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff64fd5640 1 -- 172.21.15.83:0/4102699332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5c0b0070 con 0x7eff600b66c0 2024-05-17T22:07:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff64fd5640 1 -- 172.21.15.83:0/4102699332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0ab890 con 0x7eff600b66c0 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 -- 172.21.15.83:0/4102699332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff600b66c0 msgr2=0x7eff600b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff600b66c0 0x7eff600b0bc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009870 tx=0x7eff5c01c560 comp rx=0 tx=0).stop 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 -- 172.21.15.83:0/4102699332 shutdown_connections 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff6015df30 0x7eff6015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 --2- 172.21.15.83:0/4102699332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff600b66c0 0x7eff600b0bc0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 -- 172.21.15.83:0/4102699332 >> 172.21.15.83:0/4102699332 conn(0x7eff6006cd90 msgr2=0x7eff600b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 -- 172.21.15.83:0/4102699332 shutdown_connections 2024-05-17T22:07:02.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.599+0000 7eff6823b640 1 -- 172.21.15.83:0/4102699332 wait complete. 2024-05-17T22:07:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 Processor -- start 2024-05-17T22:07:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- start start 2024-05-17T22:07:02.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff60072e50 0x7eff600709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff600b66c0 0x7eff6013ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff600b5060 con 0x7eff6015df30 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff600b4ee0 con 0x7eff600b66c0 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7eff600b51e0 con 0x7eff60072e50 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53448/0 (socket says 172.21.15.83:53448) 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 -- 172.21.15.83:0/1957051500 learned_addr learned my addr 172.21.15.83:0/1957051500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff65fd7640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff60072e50 0x7eff600709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff657d6640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff600b66c0 0x7eff6013ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 -- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff60072e50 msgr2=0x7eff600709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff60072e50 0x7eff600709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 -- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff600b66c0 msgr2=0x7eff6013ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:02.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff600b66c0 0x7eff6013ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 -- 172.21.15.83:0/1957051500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6013e300 con 0x7eff6015df30 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff667d8640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7eff58007d80 tx=0x7eff5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58014420 con 0x7eff6015df30 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff58019070 con 0x7eff6015df30 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6006eff0 con 0x7eff6015df30 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58014800 con 0x7eff6015df30 2024-05-17T22:07:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601f9ce0 con 0x7eff6015df30 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.603+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff600b0bc0 con 0x7eff6015df30 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.607+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7eff580149a0 con 0x7eff6015df30 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.607+0000 7eff567fc640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 0x7eff44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.607+0000 7eff65fd7640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 0x7eff44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.607+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5443+0+0 (secure 0 0 0) 0x7eff5808faf0 con 0x7eff6015df30 2024-05-17T22:07:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.607+0000 7eff65fd7640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 0x7eff44071910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0af000 tx=0x7eff5c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.611+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7eff6013e300 con 0x7eff6015df30 2024-05-17T22:07:02.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:02.771+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7eff600b3dd0 con 0x7eff6015df30 2024-05-17T22:07:03.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.407+0000 7eff567fc640 1 -- 172.21.15.83:0/1957051500 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7eff5805ca40 con 0x7eff6015df30 2024-05-17T22:07:03.400 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T22:07:03.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 msgr2=0x7eff44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 0x7eff44071910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0af000 tx=0x7eff5c0a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 msgr2=0x7eff6015c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7eff58007d80 tx=0x7eff5800e4b0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 shutdown_connections 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff6015df30 0x7eff6015c970 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7eff4406f400 0x7eff44071910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff600b66c0 0x7eff6013ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 --2- 172.21.15.83:0/1957051500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff60072e50 0x7eff600709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 >> 172.21.15.83:0/1957051500 conn(0x7eff6006cd90 msgr2=0x7eff600b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 shutdown_connections 2024-05-17T22:07:03.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.411+0000 7eff6823b640 1 -- 172.21.15.83:0/1957051500 wait complete. 2024-05-17T22:07:03.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T22:07:03.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.595+0000 7fbd467dd640 1 Processor -- start 2024-05-17T22:07:03.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.595+0000 7fbd467dd640 1 -- start start 2024-05-17T22:07:03.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd467dd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd400b3d00 0x7fbd400d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd467dd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd40073590 0x7fbd401447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd467dd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd400d10d0 con 0x7fbd40143f20 2024-05-17T22:07:03.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd467dd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbd400b4110 con 0x7fbd40073590 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd467dd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd40070770 con 0x7fbd400b3d00 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd44d7a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbd40073590 0x7fbd401447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd3f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd40143f20 0x7fbd40072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49848/0 (socket says 172.21.15.83:49848) 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd3f7fe640 1 -- 172.21.15.83:0/828837372 learned_addr learned my addr 172.21.15.83:0/828837372 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd3ffff640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd400b3d00 0x7fbd400d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd44d7a640 1 -- 172.21.15.83:0/828837372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd400b3d00 msgr2=0x7fbd400d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd3effd640 1 -- 172.21.15.83:0/828837372 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358613545 0 0) 0x7fbd400d10d0 con 0x7fbd40143f20 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd44d7a640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd400b3d00 0x7fbd400d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd44d7a640 1 -- 172.21.15.83:0/828837372 >> v1:172.21.15.83:6789/0 conn(0x7fbd40143f20 legacy=0x7fbd40072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.599+0000 7fbd44d7a640 1 -- 172.21.15.83:0/828837372 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd40070120 con 0x7fbd40073590 2024-05-17T22:07:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.603+0000 7fbd44d7a640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd40073590 0x7fbd401447f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009d90 tx=0x7fbd3801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdbac6a9ebd0773 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.603+0000 7fbd3effd640 1 -- 172.21.15.83:0/828837372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b2070 con 0x7fbd40073590 2024-05-17T22:07:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.603+0000 7fbd3effd640 1 -- 172.21.15.83:0/828837372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd380a58c0 con 0x7fbd40073590 2024-05-17T22:07:03.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.603+0000 7fbd3effd640 1 -- 172.21.15.83:0/828837372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380a5c90 con 0x7fbd40073590 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- 172.21.15.83:0/828837372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd40073590 msgr2=0x7fbd401447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd40073590 0x7fbd401447f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009d90 tx=0x7fbd3801c970 comp rx=0 tx=0).stop 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- 172.21.15.83:0/828837372 shutdown_connections 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd40073590 0x7fbd401447f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- 172.21.15.83:0/828837372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd400b3d00 0x7fbd400d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- 172.21.15.83:0/828837372 >> 172.21.15.83:0/828837372 conn(0x7fbd4006cd90 msgr2=0x7fbd400b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- 172.21.15.83:0/828837372 shutdown_connections 2024-05-17T22:07:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- 172.21.15.83:0/828837372 wait complete. 2024-05-17T22:07:03.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 Processor -- start 2024-05-17T22:07:03.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- start start 2024-05-17T22:07:03.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 0x7fbd4015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd40143f20 0x7fbd4013d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40070c00 con 0x7fbd400b3d00 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40145540 con 0x7fbd40073590 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd467dd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd401f7e00 con 0x7fbd40143f20 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53464/0 (socket says 172.21.15.83:53464) 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd3f7fe640 1 -- 172.21.15.83:0/2016208195 learned_addr learned my addr 172.21.15.83:0/2016208195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd3ffff640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 0x7fbd4015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.607+0000 7fbd3f7fe640 1 -- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd40143f20 msgr2=0x7fbd4013d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd44d7a640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd40143f20 0x7fbd4013d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3f7fe640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd40143f20 0x7fbd4013d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3f7fe640 1 -- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 msgr2=0x7fbd4015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3f7fe640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 0x7fbd4015f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3f7fe640 1 -- 172.21.15.83:0/2016208195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd401601d0 con 0x7fbd400b3d00 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3ffff640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 0x7fbd4015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3f7fe640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c400 tx=0x7fbd3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300074e0 con 0x7fbd400b3d00 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd300047f0 con 0x7fbd400b3d00 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30007680 con 0x7fbd400b3d00 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd401604b0 con 0x7fbd400b3d00 2024-05-17T22:07:03.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd40160a70 con 0x7fbd400b3d00 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4015ff40 con 0x7fbd400b3d00 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd30011070 con 0x7fbd400b3d00 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 0x7fbd2c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7fbd3008f180 con 0x7fbd400b3d00 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3ffff640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 0x7fbd2c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.611+0000 7fbd3ffff640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 0x7fbd2c071940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbd340097a0 tx=0x7fbd34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.615+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbd3005c3f0 con 0x7fbd400b3d00 2024-05-17T22:07:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.775+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fbd40076800 con 0x7fbd400b3d00 2024-05-17T22:07:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.783+0000 7fbd3cff9640 1 -- 172.21.15.83:0/2016208195 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fbd3005bcf0 con 0x7fbd400b3d00 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 msgr2=0x7fbd2c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 0x7fbd2c071940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbd340097a0 tx=0x7fbd34008040 comp rx=0 tx=0).stop 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 msgr2=0x7fbd4015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c400 tx=0x7fbd3000c8c0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 shutdown_connections 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd40143f20 0x7fbd4013d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd400b3d00 0x7fbd4015c370 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd2c06f430 0x7fbd2c071940 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 --2- 172.21.15.83:0/2016208195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd40073590 0x7fbd4015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 >> 172.21.15.83:0/2016208195 conn(0x7fbd4006cd90 msgr2=0x7fbd400d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 shutdown_connections 2024-05-17T22:07:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.787+0000 7fbd467dd640 1 -- 172.21.15.83:0/2016208195 wait complete. 2024-05-17T22:07:03.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-17T22:07:03.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 Processor -- start 2024-05-17T22:07:03.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 -- start start 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce80144080 0x7fce80072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce80073370 0x7fce8006f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce80073780 con 0x7fce800d1040 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fce8015e790 con 0x7fce80073370 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce86fd3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fce8015ea40 con 0x7fce80144080 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce84d6f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fce800d1040 0x7fce8015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49856/0 (socket says 172.21.15.83:49856) 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce84d6f640 1 -- 172.21.15.83:0/614627938 learned_addr learned my addr 172.21.15.83:0/614627938 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce85570640 1 --2- 172.21.15.83:0/614627938 >> v2:172.21.15.83:3300/0 conn(0x7fce80073370 0x7fce8006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce7ffff640 1 --2- 172.21.15.83:0/614627938 >> v2:172.21.15.155:3300/0 conn(0x7fce80144080 0x7fce80072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce7f7fe640 1 -- 172.21.15.83:0/614627938 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819519255 0 0) 0x7fce70001120 con 0x7fce800d1040 2024-05-17T22:07:03.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce7f7fe640 1 -- 172.21.15.83:0/614627938 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce68003530 con 0x7fce800d1040 2024-05-17T22:07:03.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce85570640 1 -- 172.21.15.83:0/614627938 >> v2:172.21.15.155:3300/0 conn(0x7fce80144080 msgr2=0x7fce80072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:03.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce85570640 1 --2- 172.21.15.83:0/614627938 >> v2:172.21.15.155:3300/0 conn(0x7fce80144080 0x7fce80072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce85570640 1 -- 172.21.15.83:0/614627938 >> v1:172.21.15.83:6789/0 conn(0x7fce800d1040 legacy=0x7fce8015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.955+0000 7fce85570640 1 -- 172.21.15.83:0/614627938 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8015ed20 con 0x7fce80073370 2024-05-17T22:07:03.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce85570640 1 --2- 172.21.15.83:0/614627938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce80073370 0x7fce8006f730 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fce800b6b10 tx=0x7fce7801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e47b29fd1576b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce7f7fe640 1 -- 172.21.15.83:0/614627938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780b2070 con 0x7fce80073370 2024-05-17T22:07:03.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce7f7fe640 1 -- 172.21.15.83:0/614627938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce780a59c0 con 0x7fce80073370 2024-05-17T22:07:03.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce7f7fe640 1 -- 172.21.15.83:0/614627938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780a5d90 con 0x7fce80073370 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 -- 172.21.15.83:0/614627938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce80073370 msgr2=0x7fce8006f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 --2- 172.21.15.83:0/614627938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce80073370 0x7fce8006f730 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fce800b6b10 tx=0x7fce7801c900 comp rx=0 tx=0).stop 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 -- 172.21.15.83:0/614627938 shutdown_connections 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 --2- 172.21.15.83:0/614627938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce80073370 0x7fce8006f730 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 --2- 172.21.15.83:0/614627938 >> v2:172.21.15.155:3300/0 conn(0x7fce80144080 0x7fce80072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 -- 172.21.15.83:0/614627938 >> 172.21.15.83:0/614627938 conn(0x7fce8006cd90 msgr2=0x7fce800b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 -- 172.21.15.83:0/614627938 shutdown_connections 2024-05-17T22:07:03.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.959+0000 7fce86fd3640 1 -- 172.21.15.83:0/614627938 wait complete. 2024-05-17T22:07:03.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 Processor -- start 2024-05-17T22:07:03.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- start start 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce80073370 0x7fce800af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce80144080 0x7fce800ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce80070480 con 0x7fce800d1040 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce80144490 con 0x7fce80144080 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fce80070600 con 0x7fce80073370 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53478/0 (socket says 172.21.15.83:53478) 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 -- 172.21.15.83:0/788026823 learned_addr learned my addr 172.21.15.83:0/788026823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce85570640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce80144080 0x7fce800ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce84d6f640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce80073370 0x7fce800af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 -- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce80073370 msgr2=0x7fce800af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce80073370 0x7fce800af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 -- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce80144080 msgr2=0x7fce800ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce80144080 0x7fce800ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 -- 172.21.15.83:0/788026823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800ad5b0 con 0x7fce800d1040 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7ffff640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fce7400e870 tx=0x7fce7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74017070 con 0x7fce800d1040 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7400ee60 con 0x7fce800d1040 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce800ad890 con 0x7fce800d1040 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7400cb40 con 0x7fce800d1040 2024-05-17T22:07:03.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce800ade50 con 0x7fce800d1040 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.963+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80069c10 con 0x7fce800d1040 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.967+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce7401a070 con 0x7fce800d1040 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.967+0000 7fce7d7fa640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 0x7fce6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.967+0000 7fce84d6f640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 0x7fce6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.967+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7fce7408ee60 con 0x7fce800d1040 2024-05-17T22:07:03.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.967+0000 7fce84d6f640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 0x7fce6c071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:03.971+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fce7405c080 con 0x7fce800d1040 2024-05-17T22:07:04.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.207+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fce800ad340 con 0x7fce800d1040 2024-05-17T22:07:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.427+0000 7fce7d7fa640 1 -- 172.21.15.83:0/788026823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v428) ==== 146+0+0 (secure 0 0 0) 0x7fce7405b980 con 0x7fce800d1040 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 msgr2=0x7fce6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 0x7fce6c071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce70007fd0 tx=0x7fce70009040 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 msgr2=0x7fce800acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fce7400e870 tx=0x7fce7400ed30 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 shutdown_connections 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fce6c06f400 0x7fce6c071910 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce80144080 0x7fce800ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800d1040 0x7fce800acd80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 --2- 172.21.15.83:0/788026823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce80073370 0x7fce800af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 >> 172.21.15.83:0/788026823 conn(0x7fce8006cd90 msgr2=0x7fce801468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 shutdown_connections 2024-05-17T22:07:04.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.431+0000 7fce86fd3640 1 -- 172.21.15.83:0/788026823 wait complete. 2024-05-17T22:07:04.450 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-17T22:07:04.450 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-17T22:07:04.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:07:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 Processor -- start 2024-05-17T22:07:04.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 -- start start 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 0x7fda28072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2806f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda28073780 con 0x7fda280d1040 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fda2815e790 con 0x7fda28144080 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda302c1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda2815ea40 con 0x7fda28073370 2024-05-17T22:07:04.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda2e85e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda2d85c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 0x7fda28072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda2e05d640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fda280d1040 0x7fda2815e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:42108/0 (socket says 172.21.15.83:42108) 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda2d85c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 0x7fda28072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:34338/0 (socket says 172.21.15.83:34338) 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.607+0000 7fda2e05d640 1 -- 172.21.15.83:0/136175031 learned_addr learned my addr 172.21.15.83:0/136175031 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2e85e640 1 -- 172.21.15.83:0/136175031 >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 msgr2=0x7fda28072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:04.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2e85e640 1 --2- 172.21.15.83:0/136175031 >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 0x7fda28072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2e85e640 1 -- 172.21.15.83:0/136175031 >> v1:172.21.15.155:6789/0 conn(0x7fda280d1040 legacy=0x7fda2815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2e85e640 1 -- 172.21.15.83:0/136175031 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2815ed20 con 0x7fda28073370 2024-05-17T22:07:04.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2e85e640 1 --2- 172.21.15.83:0/136175031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2806f730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69faf2b50cbe7c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2d05b640 1 -- 172.21.15.83:0/136175031 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200b2070 con 0x7fda28073370 2024-05-17T22:07:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2d05b640 1 -- 172.21.15.83:0/136175031 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda200a5a00 con 0x7fda28073370 2024-05-17T22:07:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda2d05b640 1 -- 172.21.15.83:0/136175031 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200a5dd0 con 0x7fda28073370 2024-05-17T22:07:04.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 -- 172.21.15.83:0/136175031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 msgr2=0x7fda2806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 --2- 172.21.15.83:0/136175031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2806f730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda20009d90 tx=0x7fda2001c940 comp rx=0 tx=0).stop 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 -- 172.21.15.83:0/136175031 shutdown_connections 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 --2- 172.21.15.83:0/136175031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2806f730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 --2- 172.21.15.83:0/136175031 >> v2:172.21.15.155:3300/0 conn(0x7fda28144080 0x7fda28072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 -- 172.21.15.83:0/136175031 >> 172.21.15.83:0/136175031 conn(0x7fda2806cd90 msgr2=0x7fda280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 -- 172.21.15.83:0/136175031 shutdown_connections 2024-05-17T22:07:04.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.611+0000 7fda302c1640 1 -- 172.21.15.83:0/136175031 wait complete. 2024-05-17T22:07:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 Processor -- start 2024-05-17T22:07:04.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 -- start start 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2813f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d1040 0x7fda280b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda28070480 con 0x7fda28144080 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fda28144490 con 0x7fda280d1040 2024-05-17T22:07:04.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda302c1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fda28070600 con 0x7fda28073370 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53484/0 (socket says 172.21.15.83:53484) 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 -- 172.21.15.83:0/3045278526 learned_addr learned my addr 172.21.15.83:0/3045278526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2d85c640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d1040 0x7fda280b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e05d640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2813f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 -- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 msgr2=0x7fda2813f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2813f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 -- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d1040 msgr2=0x7fda280b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d1040 0x7fda280b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 -- 172.21.15.83:0/3045278526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda280b48b0 con 0x7fda28144080 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.615+0000 7fda2e85e640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fda200b0040 tx=0x7fda200196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200b2050 con 0x7fda28144080 2024-05-17T22:07:04.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda200be070 con 0x7fda28144080 2024-05-17T22:07:04.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda280b4b90 con 0x7fda28144080 2024-05-17T22:07:04.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200ada70 con 0x7fda28144080 2024-05-17T22:07:04.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda280b5150 con 0x7fda28144080 2024-05-17T22:07:04.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2806f730 con 0x7fda28144080 2024-05-17T22:07:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fda200adc10 con 0x7fda28144080 2024-05-17T22:07:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7fda2012a340 con 0x7fda28144080 2024-05-17T22:07:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda2e05d640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.619+0000 7fda2e05d640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fda24000f70 tx=0x7fda24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.623+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fda20016560 con 0x7fda28144080 2024-05-17T22:07:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.783+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda28168600 con 0x7fda28144080 2024-05-17T22:07:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda1effd640 1 -- 172.21.15.83:0/3045278526 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+13126 (secure 0 0 0) 0x7fda200f7560 con 0x7fda28144080 2024-05-17T22:07:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:04.779 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":100,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:03.409123+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6834/563832419":"2024-05-18T22:06:57.365835+0000","172.21.15.83:6836/1996242614":"2024-05-18T22:06:20.560861+0000","172.21.15.83:6834/1996242614":"2024-05-18T22:06:20.560861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:07:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 msgr2=0x7fda0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fda24000f70 tx=0x7fda24009040 comp rx=0 tx=0).stop 2024-05-17T22:07:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 msgr2=0x7fda28161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fda200b0040 tx=0x7fda200196e0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 shutdown_connections 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28144080 0x7fda28161b90 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fda0c06f400 0x7fda0c071910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda280d1040 0x7fda280b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 --2- 172.21.15.83:0/3045278526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda28073370 0x7fda2813f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 >> 172.21.15.83:0/3045278526 conn(0x7fda2806cd90 msgr2=0x7fda28146730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 shutdown_connections 2024-05-17T22:07:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.787+0000 7fda302c1640 1 -- 172.21.15.83:0/3045278526 wait complete. 2024-05-17T22:07:04.802 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-17T21:57:19.014572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-17T22:07:04.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-17T22:07:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.963+0000 7fe5e1ff1640 1 Processor -- start 2024-05-17T22:07:04.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.963+0000 7fe5e1ff1640 1 -- start start 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5e1ff1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5dc15be70 0x7fe5dc072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5e1ff1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc06f2d0 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5e1ff1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc06f5a0 con 0x7fe5dc06eec0 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5e1ff1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc06f880 con 0x7fe5dc15be70 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5dc15be70 0x7fe5dc072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5dc15be70 0x7fe5dc072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53490/0 (socket says 172.21.15.83:53490) 2024-05-17T22:07:04.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 -- 172.21.15.83:0/378417497 learned_addr learned my addr 172.21.15.83:0/378417497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:04.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 -- 172.21.15.83:0/378417497 >> v1:172.21.15.155:6789/0 conn(0x7fe5dc06eec0 legacy=0x7fe5dc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 -- 172.21.15.83:0/378417497 >> v1:172.21.15.83:6789/0 conn(0x7fe5dc0b08a0 legacy=0x7fe5dc0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.967+0000 7fe5dbfff640 1 -- 172.21.15.83:0/378417497 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc06fb60 con 0x7fe5dc15be70 2024-05-17T22:07:04.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5dbfff640 1 --2- 172.21.15.83:0/378417497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc15be70 0x7fe5dc072e80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d150 tx=0x7fe5d401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72a5e2fc074e503 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5da7fc640 1 -- 172.21.15.83:0/378417497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40ad070 con 0x7fe5dc15be70 2024-05-17T22:07:04.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5da7fc640 1 -- 172.21.15.83:0/378417497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d40a7ad0 con 0x7fe5dc15be70 2024-05-17T22:07:04.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5da7fc640 1 -- 172.21.15.83:0/378417497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40b1400 con 0x7fe5dc15be70 2024-05-17T22:07:04.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/378417497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc15be70 msgr2=0x7fe5dc072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/378417497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc15be70 0x7fe5dc072e80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d150 tx=0x7fe5d401eb70 comp rx=0 tx=0).stop 2024-05-17T22:07:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/378417497 shutdown_connections 2024-05-17T22:07:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/378417497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc15be70 0x7fe5dc072e80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/378417497 >> 172.21.15.83:0/378417497 conn(0x7fe5dc06cd90 msgr2=0x7fe5dc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/378417497 shutdown_connections 2024-05-17T22:07:04.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/378417497 wait complete. 2024-05-17T22:07:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 Processor -- start 2024-05-17T22:07:04.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.971+0000 7fe5e1ff1640 1 -- start start 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5dc06eec0 0x7fe5dc14be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5dc15be70 0x7fe5dc14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc160a40 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1608c0 con 0x7fe5dc06eec0 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc0b4bd0 con 0x7fe5dc15be70 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53498/0 (socket says 172.21.15.83:53498) 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 -- 172.21.15.83:0/2028244878 learned_addr learned my addr 172.21.15.83:0/2028244878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:04.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5db7fe640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5dc06eec0 0x7fe5dc14be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5dbfff640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5dc15be70 0x7fe5dc14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 -- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5dc15be70 msgr2=0x7fe5dc14fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5dc15be70 0x7fe5dc14fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 -- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5dc06eec0 msgr2=0x7fe5dc14be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5dc06eec0 0x7fe5dc14be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 -- 172.21.15.83:0/2028244878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc151950 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5daffd640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c3c0 tx=0x7fe5d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d00052d0 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc151c30 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00054c0 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc158070 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc072e80 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.975+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe5d001a070 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.979+0000 7fe5c3fff640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 0x7fe5c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.979+0000 7fe5db7fe640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 0x7fe5c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.979+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7fe5d008e310 con 0x7fe5dc0b08a0 2024-05-17T22:07:04.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.979+0000 7fe5db7fe640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 0x7fe5c8071940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc001650 tx=0x7fe5cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:04.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:04.983+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe5d005b4a0 con 0x7fe5dc0b08a0 2024-05-17T22:07:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.143+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5dc0b0f10 con 0x7fe5dc0b08a0 2024-05-17T22:07:05.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.143+0000 7fe5c3fff640 1 -- 172.21.15.83:0/2028244878 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fe5d005ada0 con 0x7fe5dc0b08a0 2024-05-17T22:07:05.136 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 msgr2=0x7fe5c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 0x7fe5c8071940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc001650 tx=0x7fe5cc009040 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 msgr2=0x7fe5dc158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c3c0 tx=0x7fe5d000c880 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 shutdown_connections 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5dc15be70 0x7fe5dc14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5dc0b08a0 0x7fe5dc158590 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe5c806f430 0x7fe5c8071940 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 --2- 172.21.15.83:0/2028244878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5dc06eec0 0x7fe5dc14be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 >> 172.21.15.83:0/2028244878 conn(0x7fe5dc06cd90 msgr2=0x7fe5dc0d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 shutdown_connections 2024-05-17T22:07:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.147+0000 7fe5e1ff1640 1 -- 172.21.15.83:0/2028244878 wait complete. 2024-05-17T22:07:05.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-17T22:07:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 Processor -- start 2024-05-17T22:07:05.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 -- start start 2024-05-17T22:07:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 0x7f5328159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f532815c830 con 0x7f5328072e30 2024-05-17T22:07:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f53281608a0 con 0x7f532815fe20 2024-05-17T22:07:05.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.315+0000 7f532ee3b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5328160b80 con 0x7f532815d2f0 2024-05-17T22:07:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f5327fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 0x7f5328159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532d3d8640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5328072e30 0x7f5328073240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:42126/0 (socket says 172.21.15.83:42126) 2024-05-17T22:07:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f5327fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:34368/0 (socket says 172.21.15.83:34368) 2024-05-17T22:07:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532d3d8640 1 -- 172.21.15.83:0/3232851996 learned_addr learned my addr 172.21.15.83:0/3232851996 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 -- 172.21.15.83:0/3232851996 >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 msgr2=0x7f5328160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 --2- 172.21.15.83:0/3232851996 >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 -- 172.21.15.83:0/3232851996 >> v1:172.21.15.155:6789/0 conn(0x7f5328072e30 legacy=0x7f5328073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 -- 172.21.15.83:0/3232851996 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532815caf0 con 0x7f532815d2f0 2024-05-17T22:07:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f5327fff640 1 --2- 172.21.15.83:0/3232851996 >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:07:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f532cbd7640 1 --2- 172.21.15.83:0/3232851996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 0x7f5328159c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53180097c0 tx=0x7f531801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3950504caa50620 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.319+0000 7f53277fe640 1 -- 172.21.15.83:0/3232851996 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180b0070 con 0x7f532815d2f0 2024-05-17T22:07:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f53277fe640 1 -- 172.21.15.83:0/3232851996 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53180ab600 con 0x7f532815d2f0 2024-05-17T22:07:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f53277fe640 1 -- 172.21.15.83:0/3232851996 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180ab9d0 con 0x7f532815d2f0 2024-05-17T22:07:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- 172.21.15.83:0/3232851996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 msgr2=0x7f5328159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3232851996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 0x7f5328159c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53180097c0 tx=0x7f531801c410 comp rx=0 tx=0).stop 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- 172.21.15.83:0/3232851996 shutdown_connections 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3232851996 >> v2:172.21.15.155:3300/0 conn(0x7f532815fe20 0x7f5328160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3232851996 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815d2f0 0x7f5328159c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- 172.21.15.83:0/3232851996 >> 172.21.15.83:0/3232851996 conn(0x7f532806cd90 msgr2=0x7f53280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- 172.21.15.83:0/3232851996 shutdown_connections 2024-05-17T22:07:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- 172.21.15.83:0/3232851996 wait complete. 2024-05-17T22:07:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 Processor -- start 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.323+0000 7f532ee3b640 1 -- start start 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 0x7f53281f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 0x7f5328156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53280b04f0 con 0x7f5328072e30 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53280b0370 con 0x7f532815d2f0 2024-05-17T22:07:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f53280b0670 con 0x7f532815fe20 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53504/0 (socket says 172.21.15.83:53504) 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 -- 172.21.15.83:0/3821705515 learned_addr learned my addr 172.21.15.83:0/3821705515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f5327fff640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 0x7f53281f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532d3d8640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 0x7f5328156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 -- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 msgr2=0x7f5328156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 0x7f5328156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 -- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 msgr2=0x7f53281f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 0x7f53281f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 -- 172.21.15.83:0/3821705515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5328155a60 con 0x7f5328072e30 2024-05-17T22:07:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532d3d8640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 0x7f5328156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f5327fff640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 0x7f53281f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532cbd7640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f53180ae040 tx=0x7f53180191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180ba070 con 0x7f5328072e30 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53180b43f0 con 0x7f5328072e30 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5328155d40 con 0x7f5328072e30 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180b0040 con 0x7f5328072e30 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5328156300 con 0x7f5328072e30 2024-05-17T22:07:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5328159c20 con 0x7f5328072e30 2024-05-17T22:07:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f53180b47c0 con 0x7f5328072e30 2024-05-17T22:07:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7f53181280a0 con 0x7f5328072e30 2024-05-17T22:07:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f5327fff640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.327+0000 7f5327fff640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53280b68e0 tx=0x7f531c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.331+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5318016090 con 0x7f5328072e30 2024-05-17T22:07:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:05.547+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f53280af030 con 0x7f5328072e30 2024-05-17T22:07:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.439+0000 7f53257fa640 1 -- 172.21.15.83:0/3821705515 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v430) ==== 147+0+0 (secure 0 0 0) 0x7f53180f4b60 con 0x7f5328072e30 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 msgr2=0x7f5314071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f53280b68e0 tx=0x7f531c008040 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 msgr2=0x7f53281543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f53180ae040 tx=0x7f53180191b0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 shutdown_connections 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f532815fe20 0x7f5328156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f531406f400 0x7f5314071910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f532815d2f0 0x7f53281f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 --2- 172.21.15.83:0/3821705515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5328072e30 0x7f53281543a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 >> 172.21.15.83:0/3821705515 conn(0x7f532806cd90 msgr2=0x7f53280b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 shutdown_connections 2024-05-17T22:07:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.443+0000 7f532ee3b640 1 -- 172.21.15.83:0/3821705515 wait complete. 2024-05-17T22:07:06.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-17T22:07:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 Processor -- start 2024-05-17T22:07:06.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 -- start start 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdcc15d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc0d1210 con 0x7fbdcc1600f0 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc160e50 con 0x7fbdcc0d2050 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd3ec3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc160c10 con 0x7fbdcc0d0e00 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd2460640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbdcc1600f0 0x7fbdcc160500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:49880/0 (socket says 172.21.15.83:49880) 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd2460640 1 -- 172.21.15.83:0/654799707 learned_addr learned my addr 172.21.15.83:0/654799707 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820425096 0 0) 0x7fbdcc0d1210 con 0x7fbdcc1600f0 2024-05-17T22:07:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdcc15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb8003530 con 0x7fbdcc1600f0 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3995372800 0 0) 0x7fbdb8003530 con 0x7fbdcc1600f0 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 msgr2=0x7fbdcc15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdcc15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 >> v1:172.21.15.155:6789/0 conn(0x7fbdcc0d2050 legacy=0x7fbdcc0d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc0d0320 con 0x7fbdcc1600f0 2024-05-17T22:07:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fbdc4001a80 con 0x7fbdcc1600f0 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 >> v1:172.21.15.83:6789/0 conn(0x7fbdcc1600f0 legacy=0x7fbdcc160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdb8003a40 0x7fbdb8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdb8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdb80052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc160c10 con 0x7fbdb8004810 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb800bb90 con 0x7fbdb8003a40 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb800bd90 con 0x7fbdcc0d0e00 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdb8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdb8003a40 0x7fbdb8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd2460640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdb80052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 -- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 msgr2=0x7fbdb80052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdb80052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 -- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdb8003a40 msgr2=0x7fbdb8003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdb8003a40 0x7fbdb8003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd145e640 1 -- 172.21.15.83:0/654799707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb8003530 con 0x7fbdb8004810 2024-05-17T22:07:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdb8003a40 0x7fbdb8003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.623+0000 7fbdd2460640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0d0e00 0x7fbdb80052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd145e640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdb8004c20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc001790 tx=0x7fbdbc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfe5fb46284cbe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0013e0 con 0x7fbdb8004810 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc0052d0 con 0x7fbdb8004810 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd0c5d640 1 -- 172.21.15.83:0/654799707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc003840 con 0x7fbdb8004810 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/654799707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 msgr2=0x7fbdb8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/654799707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdb8004c20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc001790 tx=0x7fbdbc01c3f0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd2460640 1 -- 172.21.15.83:0/654799707 reap_dead start 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/654799707 shutdown_connections 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/654799707 >> 172.21.15.83:0/654799707 conn(0x7fbdcc06dfd0 msgr2=0x7fbdcc06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/654799707 shutdown_connections 2024-05-17T22:07:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.627+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/654799707 wait complete. 2024-05-17T22:07:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 Processor -- start 2024-05-17T22:07:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- start start 2024-05-17T22:07:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0cadb0 0x7fbdcc0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdcc15f1f0 0x7fbdcc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc0d0970 con 0x7fbdb8004810 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1645d0 con 0x7fbdcc15f1f0 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc15d450 con 0x7fbdcc0cadb0 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53518/0 (socket says 172.21.15.83:53518) 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 -- 172.21.15.83:0/206691506 learned_addr learned my addr 172.21.15.83:0/206691506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd2460640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdcc15f1f0 0x7fbdcc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd145e640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0cadb0 0x7fbdcc0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 -- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0cadb0 msgr2=0x7fbdcc0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0cadb0 0x7fbdcc0cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 -- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdcc15f1f0 msgr2=0x7fbdcc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdcc15f1f0 0x7fbdcc15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 -- 172.21.15.83:0/206691506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc15cf10 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd1c5f640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80116a0 tx=0x7fbdc8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8004220 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc80043c0 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800f9e0 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc15fb40 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc0c97d0 con 0x7fbdb8004810 2024-05-17T22:07:06.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.631+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc15d7e0 con 0x7fbdb8004810 2024-05-17T22:07:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.635+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbdc800fb80 con 0x7fbdb8004810 2024-05-17T22:07:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.635+0000 7fbdc27fc640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 0x7fbdb0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.635+0000 7fbdd145e640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 0x7fbdb0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.635+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7fbdc808f740 con 0x7fbdb8004810 2024-05-17T22:07:06.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.635+0000 7fbdd145e640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 0x7fbdb0071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ad000 tx=0x7fbdbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:06.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.639+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbdcc15fb40 con 0x7fbdb8004810 2024-05-17T22:07:06.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.799+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbdcc162800 con 0x7fbdb8004810 2024-05-17T22:07:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.803+0000 7fbdc27fc640 1 -- 172.21.15.83:0/206691506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fbdc8019030 con 0x7fbdb8004810 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 msgr2=0x7fbdb0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 0x7fbdb0071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ad000 tx=0x7fbdbc009040 comp rx=0 tx=0).stop 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 msgr2=0x7fbdcc15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80116a0 tx=0x7fbdc8011b60 comp rx=0 tx=0).stop 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 shutdown_connections 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbdb006f400 0x7fbdb0071910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdcc15f1f0 0x7fbdcc15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbdcc0cadb0 0x7fbdcc0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 --2- 172.21.15.83:0/206691506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdb8004810 0x7fbdcc15c9d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 >> 172.21.15.83:0/206691506 conn(0x7fbdcc06dfd0 msgr2=0x7fbdcc0d5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 shutdown_connections 2024-05-17T22:07:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:06.807+0000 7fbdd3ec3640 1 -- 172.21.15.83:0/206691506 wait complete. 2024-05-17T22:07:06.819 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:07:06.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:07:06.868 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:07:06.934 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:07:06.934 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:06.934 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:06.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:07:06.943 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:07:06.943 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:06.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:07:06.948 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:06.948 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:06.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:07:06.992 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:07:06.992 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:06.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:07:06.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-17T22:07:06.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.187+0000 7f19ef5f7640 1 Processor -- start 2024-05-17T22:07:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.187+0000 7f19ef5f7640 1 -- start start 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ef5f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19e81609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ef5f7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e81465d0 con 0x7f19e81461c0 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ef5f7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e8160ef0 con 0x7f19e806ffd0 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ef5f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19e806ef30 con 0x7f19e81605a0 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f19e81461c0 0x7f19e815e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60642/0 (socket says 172.21.15.83:60642) 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 -- 172.21.15.83:0/440323648 learned_addr learned my addr 172.21.15.83:0/440323648 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214298061 0 0) 0x7f19e81465d0 con 0x7f19e81461c0 2024-05-17T22:07:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19d4003530 con 0x7f19e81461c0 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19edb94640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19e81609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3864098579 0 0) 0x7f19d4003530 con 0x7f19e81461c0 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 msgr2=0x7f19e81609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19e81609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 >> v1:172.21.15.155:6789/0 conn(0x7f19e806ffd0 legacy=0x7f19e80703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e806f210 con 0x7f19e81461c0 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f19e4001a80 con 0x7f19e81461c0 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 >> v1:172.21.15.83:6789/0 conn(0x7f19e81461c0 legacy=0x7f19e815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d4003a40 0x7f19d4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4004e70 0x7f19d40052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19e806ef30 con 0x7f19d4003a40 2024-05-17T22:07:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19d400bc80 con 0x7f19d4004e70 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19d400be80 con 0x7f19e81605a0 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d4003a40 0x7f19d4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ecb92640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19edb94640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4004e70 0x7f19d40052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 -- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 msgr2=0x7f19d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e81605a0 0x7f19d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 -- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4004e70 msgr2=0x7f19d40052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19d4004e70 0x7f19d40052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.191+0000 7f19ed393640 1 -- 172.21.15.83:0/440323648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d4003530 con 0x7f19d4003a40 2024-05-17T22:07:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ed393640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d4003a40 0x7f19d4003eb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a800 tx=0x7f19e401d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaaa5f8f24628d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e400a450 con 0x7f19d4003a40 2024-05-17T22:07:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e40072d0 con 0x7f19d4003a40 2024-05-17T22:07:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19dffff640 1 -- 172.21.15.83:0/440323648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e40076c0 con 0x7f19d4003a40 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 -- 172.21.15.83:0/440323648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d4003a40 msgr2=0x7f19d4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/440323648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d4003a40 0x7f19d4003eb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a800 tx=0x7f19e401d250 comp rx=0 tx=0).stop 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19edb94640 1 -- 172.21.15.83:0/440323648 reap_dead start 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 -- 172.21.15.83:0/440323648 shutdown_connections 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 -- 172.21.15.83:0/440323648 >> 172.21.15.83:0/440323648 conn(0x7f19e806cd90 msgr2=0x7f19e80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 -- 172.21.15.83:0/440323648 shutdown_connections 2024-05-17T22:07:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.195+0000 7f19ef5f7640 1 -- 172.21.15.83:0/440323648 wait complete. 2024-05-17T22:07:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 Processor -- start 2024-05-17T22:07:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- start start 2024-05-17T22:07:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e80b4ed0 0x7f19e8140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 0x7f19e813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19e80d10a0 con 0x7f19e815c410 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f19e815e880 con 0x7f19e813e8f0 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f19e80d1220 con 0x7f19e80b4ed0 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41328/0 (socket says 172.21.15.83:41328) 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 -- 172.21.15.83:0/1252095855 learned_addr learned my addr 172.21.15.83:0/1252095855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19edb94640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 0x7f19e813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 -- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e80b4ed0 msgr2=0x7f19e8140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ed393640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e80b4ed0 0x7f19e8140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e80b4ed0 0x7f19e8140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 -- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 msgr2=0x7f19e813ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 0x7f19e813ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 -- 172.21.15.83:0/1252095855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e813f240 con 0x7f19e815c410 2024-05-17T22:07:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19edb94640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 0x7f19e813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ecb92640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f19d800f7f0 tx=0x7f19d800fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80050d0 con 0x7f19e815c410 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d8018070 con 0x7f19e815c410 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e815c980 con 0x7f19e815c410 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d800d730 con 0x7f19e815c410 2024-05-17T22:07:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.199+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e813ab80 con 0x7f19e815c410 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e80703e0 con 0x7f19e815c410 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f19d800d8d0 con 0x7f19e815c410 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ddffb640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 0x7f19cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7f19d808eda0 con 0x7f19e815c410 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ed393640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 0x7f19cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.203+0000 7f19ed393640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 0x7f19cc071910 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f19e40ad000 tx=0x7f19e806f100 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.207+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f19e813f240 con 0x7f19e815c410 2024-05-17T22:07:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.399+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e8145a10 con 0x7f19e815c410 2024-05-17T22:07:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.399+0000 7f19ddffb640 1 -- 172.21.15.83:0/1252095855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3875 (secure 0 0 0) 0x7f19e8160ef0 con 0x7f19e815c410 2024-05-17T22:07:07.392 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:07.392 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":91,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3018492373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3018492373},{"type":"v1","addr":"172.21.15.155:6833","nonce":3018492373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1708230042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1708230042},{"type":"v1","addr":"172.21.15.155:6835","nonce":1708230042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7404,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3330984352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3330984352},{"type":"v1","addr":"172.21.15.83:6837","nonce":3330984352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2307755165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2307755165},{"type":"v1","addr":"172.21.15.83:6836","nonce":2307755165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-05-17T22:07:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 91 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 msgr2=0x7f19cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 0x7f19cc071910 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f19e40ad000 tx=0x7f19e806f100 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 msgr2=0x7f19e815b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f19d800f7f0 tx=0x7f19d800fcb0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 shutdown_connections 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f19cc06f400 0x7f19cc071910 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19e813e8f0 0x7f19e813ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e815c410 0x7f19e815b0f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 --2- 172.21.15.83:0/1252095855 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19e80b4ed0 0x7f19e8140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 >> 172.21.15.83:0/1252095855 conn(0x7f19e806cd90 msgr2=0x7f19e80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 shutdown_connections 2024-05-17T22:07:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.403+0000 7f19ef5f7640 1 -- 172.21.15.83:0/1252095855 wait complete. 2024-05-17T22:07:07.413 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-17T22:07:07.413 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-17T22:07:07.414 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T22:07:07.414 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:07.414 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-17T22:07:07.414 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-17T22:07:07.414 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T22:07:07.414 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:07.415 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-17T22:07:07.415 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-17T22:07:07.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T22:07:07.415 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:07.415 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-17T22:07:07.415 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-17T22:07:07.415 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T22:07:07.415 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:07.429 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T22:07:07.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-17T22:07:07.430 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T22:07:07.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-17T22:07:07.432 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T22:07:07.432 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-17T22:07:07.435 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T22:07:07.435 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-17T22:07:07.471 INFO:tasks.ceph.mds.c:Started 2024-05-17T22:07:07.472 DEBUG:teuthology.parallel:result is None 2024-05-17T22:07:07.472 INFO:tasks.ceph.mds.a:Started 2024-05-17T22:07:07.472 DEBUG:teuthology.parallel:result is None 2024-05-17T22:07:07.481 INFO:tasks.ceph.mds.d:Started 2024-05-17T22:07:07.481 DEBUG:teuthology.parallel:result is None 2024-05-17T22:07:07.483 INFO:tasks.ceph.mds.b:Started 2024-05-17T22:07:07.483 DEBUG:teuthology.parallel:result is None 2024-05-17T22:07:07.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-17T22:07:07.557 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2024-05-17T22:07:07.557 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:07:07.580 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T22:07:07.581 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2024-05-17T22:07:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 Processor -- start 2024-05-17T22:07:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 -- start start 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 0x7f4dec0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 0x7f4dec0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec0cdcb0 con 0x7f4dec06d1a0 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4dec0ce270 con 0x7f4dec1633b0 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df2511640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4dec0cdf90 con 0x7f4dec0d1e00 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4debfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4dec06d1a0 0x7f4dec0d16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60676/0 (socket says 172.21.15.83:60676) 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4debfff640 1 -- 172.21.15.83:0/2027125862 learned_addr learned my addr 172.21.15.83:0/2027125862 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deb7fe640 1 --2- 172.21.15.83:0/2027125862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 0x7f4dec0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df0aae640 1 --2- 172.21.15.83:0/2027125862 >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 0x7f4dec0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009037774 0 0) 0x7f4dec0cdcb0 con 0x7f4dec06d1a0 2024-05-17T22:07:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dd8003530 con 0x7f4dec06d1a0 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deb7fe640 1 -- 172.21.15.83:0/2027125862 >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 msgr2=0x7f4dec0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deb7fe640 1 --2- 172.21.15.83:0/2027125862 >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 0x7f4dec0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deb7fe640 1 -- 172.21.15.83:0/2027125862 >> v1:172.21.15.83:6789/0 conn(0x7f4dec06d1a0 legacy=0x7f4dec0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2387185185 0 0) 0x7f4dd8003530 con 0x7f4dec06d1a0 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4deb7fe640 1 -- 172.21.15.83:0/2027125862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec0b37a0 con 0x7f4dec0d1e00 2024-05-17T22:07:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.627+0000 7f4df0aae640 1 --2- 172.21.15.83:0/2027125862 >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 0x7f4dec0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:07:07.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4deb7fe640 1 --2- 172.21.15.83:0/2027125862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 0x7f4dec0b76d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0097c0 tx=0x7f4ddc01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd78832beff83e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc0b0070 con 0x7f4dec0d1e00 2024-05-17T22:07:07.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ddc0ab400 con 0x7f4dec0d1e00 2024-05-17T22:07:07.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4deaffd640 1 -- 172.21.15.83:0/2027125862 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4001700 con 0x7f4dec0d1e00 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- 172.21.15.83:0/2027125862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 msgr2=0x7f4dec0b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 --2- 172.21.15.83:0/2027125862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 0x7f4dec0b76d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0097c0 tx=0x7f4ddc01c330 comp rx=0 tx=0).stop 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- 172.21.15.83:0/2027125862 shutdown_connections 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 --2- 172.21.15.83:0/2027125862 >> v2:172.21.15.155:3300/0 conn(0x7f4dec1633b0 0x7f4dec0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 --2- 172.21.15.83:0/2027125862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec0d1e00 0x7f4dec0b76d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- 172.21.15.83:0/2027125862 >> 172.21.15.83:0/2027125862 conn(0x7f4dec0b4090 msgr2=0x7f4dec0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- 172.21.15.83:0/2027125862 shutdown_connections 2024-05-17T22:07:07.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- 172.21.15.83:0/2027125862 wait complete. 2024-05-17T22:07:07.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 Processor -- start 2024-05-17T22:07:07.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.631+0000 7f4df2511640 1 -- start start 2024-05-17T22:07:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec06d1a0 0x7f4dec159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4dec0d1e00 0x7f4dec09c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0cf700 con 0x7f4dec1633b0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0cf580 con 0x7f4dec0d1e00 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4dec0cf880 con 0x7f4dec06d1a0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41368/0 (socket says 172.21.15.83:41368) 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 -- 172.21.15.83:0/2832461586 learned_addr learned my addr 172.21.15.83:0/2832461586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4debfff640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec06d1a0 0x7f4dec159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4deb7fe640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4dec0d1e00 0x7f4dec09c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 -- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec06d1a0 msgr2=0x7f4dec159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec06d1a0 0x7f4dec159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 -- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4dec0d1e00 msgr2=0x7f4dec09c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4dec0d1e00 0x7f4dec09c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 -- 172.21.15.83:0/2832461586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec15a0f0 con 0x7f4dec1633b0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df0aae640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4de400efb0 tx=0x7f4de400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4018020 con 0x7f4dec1633b0 2024-05-17T22:07:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de4014480 con 0x7f4dec1633b0 2024-05-17T22:07:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec09ca30 con 0x7f4dec1633b0 2024-05-17T22:07:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4014850 con 0x7f4dec1633b0 2024-05-17T22:07:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec09d220 con 0x7f4dec1633b0 2024-05-17T22:07:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dec09cc90 con 0x7f4dec1633b0 2024-05-17T22:07:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4de401b020 con 0x7f4dec1633b0 2024-05-17T22:07:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 0x7f4dcc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7f4de408fbe0 con 0x7f4dec1633b0 2024-05-17T22:07:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.635+0000 7f4debfff640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 0x7f4dcc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.639+0000 7f4debfff640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 0x7f4dcc071940 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4001250 tx=0x7f4dd40011e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:07.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.643+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4de4022030 con 0x7f4dec1633b0 2024-05-17T22:07:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:07.811+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4dec0ce620 con 0x7f4dec1633b0 2024-05-17T22:07:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.839+0000 7f4de8ff9640 1 -- 172.21.15.83:0/2832461586 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) ==== 97+0+0 (secure 0 0 0) 0x7f4de4010040 con 0x7f4dec1633b0 2024-05-17T22:07:08.831 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 msgr2=0x7f4dcc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 0x7f4dcc071940 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4001250 tx=0x7f4dd40011e0 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 msgr2=0x7f4dec09dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4de400efb0 tx=0x7f4de400c490 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 shutdown_connections 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dec1633b0 0x7f4dec09dfe0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4dcc06f430 0x7f4dcc071940 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4dec0d1e00 0x7f4dec09c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 --2- 172.21.15.83:0/2832461586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4dec06d1a0 0x7f4dec159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 >> 172.21.15.83:0/2832461586 conn(0x7f4dec0b4090 msgr2=0x7f4dec15a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 shutdown_connections 2024-05-17T22:07:08.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:08.843+0000 7f4df2511640 1 -- 172.21.15.83:0/2832461586 wait complete. 2024-05-17T22:07:08.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-17T22:07:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.023+0000 7fb8298a5640 1 Processor -- start 2024-05-17T22:07:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.023+0000 7fb8298a5640 1 -- start start 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8298a5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb824073540 0x7fb824168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8298a5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb82415e790 con 0x7fb824144080 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8298a5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb82415ea40 con 0x7fb8240d1040 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8298a5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb82415ed20 con 0x7fb824073540 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8237fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb824073540 0x7fb824168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8227fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb824144080 0x7fb824072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60680/0 (socket says 172.21.15.83:60680) 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8227fc640 1 -- 172.21.15.83:0/853861373 learned_addr learned my addr 172.21.15.83:0/853861373 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8237fe640 1 --2- 172.21.15.83:0/853861373 >> v2:172.21.15.83:3300/0 conn(0x7fb824073540 0x7fb824168840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41382/0 (socket says 172.21.15.83:41382) 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8237fe640 1 -- 172.21.15.83:0/853861373 >> v1:172.21.15.155:6789/0 conn(0x7fb8240d1040 legacy=0x7fb82415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8237fe640 1 -- 172.21.15.83:0/853861373 >> v1:172.21.15.83:6789/0 conn(0x7fb824144080 legacy=0x7fb824072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb821ffb640 1 -- 172.21.15.83:0/853861373 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826529738 0 0) 0x7fb82415e790 con 0x7fb824144080 2024-05-17T22:07:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.027+0000 7fb8237fe640 1 -- 172.21.15.83:0/853861373 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82415f000 con 0x7fb824073540 2024-05-17T22:07:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8237fe640 1 --2- 172.21.15.83:0/853861373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb824073540 0x7fb824168840 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6db28fae1924be server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb821ffb640 1 -- 172.21.15.83:0/853861373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0b2070 con 0x7fb824073540 2024-05-17T22:07:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb821ffb640 1 -- 172.21.15.83:0/853861373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81c0a5a00 con 0x7fb824073540 2024-05-17T22:07:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb821ffb640 1 -- 172.21.15.83:0/853861373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0a5dd0 con 0x7fb824073540 2024-05-17T22:07:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- 172.21.15.83:0/853861373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb824073540 msgr2=0x7fb824168840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 --2- 172.21.15.83:0/853861373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb824073540 0x7fb824168840 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009d90 tx=0x7fb81c01c940 comp rx=0 tx=0).stop 2024-05-17T22:07:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- 172.21.15.83:0/853861373 shutdown_connections 2024-05-17T22:07:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 --2- 172.21.15.83:0/853861373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb824073540 0x7fb824168840 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- 172.21.15.83:0/853861373 >> 172.21.15.83:0/853861373 conn(0x7fb82406cd90 msgr2=0x7fb8240b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- 172.21.15.83:0/853861373 shutdown_connections 2024-05-17T22:07:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- 172.21.15.83:0/853861373 wait complete. 2024-05-17T22:07:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 Processor -- start 2024-05-17T22:07:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.031+0000 7fb8298a5640 1 -- start start 2024-05-17T22:07:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb824144080 0x7fb82413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb82413fa40 0x7fb82415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb82416b720 con 0x7fb8240d1040 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb82416b5a0 con 0x7fb82413fa40 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb82416b8a0 con 0x7fb824144080 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41388/0 (socket says 172.21.15.83:41388) 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 -- 172.21.15.83:0/789050892 learned_addr learned my addr 172.21.15.83:0/789050892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 -- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb824144080 msgr2=0x7fb82413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb824144080 0x7fb82413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 -- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb82413fa40 msgr2=0x7fb82415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb82413fa40 0x7fb82415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 -- 172.21.15.83:0/789050892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb824134ea0 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb822ffd640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b410 tx=0x7fb81800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818015030 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb818004020 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb824135180 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818004450 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb824132a80 con 0x7fb8240d1040 2024-05-17T22:07:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82415e080 con 0x7fb8240d1040 2024-05-17T22:07:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb818018070 con 0x7fb8240d1040 2024-05-17T22:07:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 0x7fb810071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8227fc640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 0x7fb810071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fb81808eaf0 con 0x7fb8240d1040 2024-05-17T22:07:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.035+0000 7fb8227fc640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 0x7fb810071910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c10 tx=0x7fb8140079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.039+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb81805bdc0 con 0x7fb8240d1040 2024-05-17T22:07:09.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.255+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb8240b2490 con 0x7fb8240d1040 2024-05-17T22:07:09.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.255+0000 7fb80bfff640 1 -- 172.21.15.83:0/789050892 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fb81805b6c0 con 0x7fb8240d1040 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdmiOZNLxAAb9fPC+FdmO5lfSea6+KlRg==", 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.250 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmQA2tABAA8GrrvSNJ+HoxvHSzi6zK8A==", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2024-05-17T22:07:09.251 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdme5C7NhAA07f+l4aLxsYhjX1HsY8MNw==", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmxbjyBxAAk8hx4Z00ZSD9U4f2QCf9xQ==", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:07:09.252 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCT0kdmbzYVGRAACj/EaZP4tCpVveds5Q+CXA==", 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.253 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCW0kdmfX8MIBAAFQPqiBYnXqOuUbyPBZK1dw==", 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.254 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCZ0kdmqjiVJxAADzaCeK8ok1b65H68JReDjA==", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCc0kdmsH0+LhAARyiY9TbhUNrBhtLk7C9QrA==", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.255 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCi0kdmWxPtLRAA3+l+C/eSiDE9ShinmhJOow==", 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2024-05-17T22:07:09.256 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCl0kdmbNRONRAAT7QHKFw9SW5PfwQjYEnx1w==", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCp0kdmzomPABAA0sqDpHnjOM7VQWDySfP05g==", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.257 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCs0kdmytiLCBAAwA5wGaT7Lwy3W9hwt2yZBg==", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA==", 2024-05-17T22:07:09.258 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdm5zADFhAAAcINrg0ovwZCWahseQOdfw==", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:07:09.259 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmATx/ERAAnoEtEvDn7Z+MDA/LuWIheQ==", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2024-05-17T22:07:09.260 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCQ0kdmgXk5GhAAVVZMJTilOhyGAbxjqWlXNw==", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm/5K7EhAATvAc+eDCV+vWJ72UdN4mSg==", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2024-05-17T22:07:09.261 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm19IkKhAAeB6wVDDOpCPsvm2goHLzZQ==", 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:09.262 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCP0kdm3eFnJRAA/sHP3pdekdDuK7P9Ur/nmw==", 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-05-17T22:07:09.263 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 msgr2=0x7fb810071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 0x7fb810071910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb814000c10 tx=0x7fb8140079d0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 msgr2=0x7fb824134470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b410 tx=0x7fb81800b8d0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 shutdown_connections 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb81006f400 0x7fb810071910 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb82413fa40 0x7fb82415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb824144080 0x7fb82413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 --2- 172.21.15.83:0/789050892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8240d1040 0x7fb824134470 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 >> 172.21.15.83:0/789050892 conn(0x7fb82406cd90 msgr2=0x7fb824168460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 shutdown_connections 2024-05-17T22:07:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.259+0000 7fb8298a5640 1 -- 172.21.15.83:0/789050892 wait complete. 2024-05-17T22:07:09.271 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-17T22:07:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.411+0000 7fc8d0159640 1 Processor -- start 2024-05-17T22:07:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.411+0000 7fc8d0159640 1 -- start start 2024-05-17T22:07:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8d0159640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8c80d1040 0x7fc8c8144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8d0159640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c815e080 0x7fc8c8072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8d0159640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c815ea40 con 0x7fc8c815e490 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8d0159640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c815ed20 con 0x7fc8c80d1040 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8d0159640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c815f000 con 0x7fc8c815e080 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8c80d1040 0x7fc8c8144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8c80d1040 0x7fc8c8144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41390/0 (socket says 172.21.15.83:41390) 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 -- 172.21.15.83:0/2399439975 learned_addr learned my addr 172.21.15.83:0/2399439975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cd6f4640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c815e080 0x7fc8c8072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 -- 172.21.15.83:0/2399439975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c815e080 msgr2=0x7fc8c8072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c815e080 0x7fc8c8072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 -- 172.21.15.83:0/2399439975 >> v1:172.21.15.155:6789/0 conn(0x7fc8c815e490 legacy=0x7fc8c81616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.415+0000 7fc8cdef5640 1 -- 172.21.15.83:0/2399439975 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c806ef30 con 0x7fc8c80d1040 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8cdef5640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c80d1040 0x7fc8c8144080 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009870 tx=0x7fc8c401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfa6f7791706a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8ccef3640 1 -- 172.21.15.83:0/2399439975 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40ab5f0 con 0x7fc8c80d1040 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8ccef3640 1 -- 172.21.15.83:0/2399439975 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c40b0070 con 0x7fc8c80d1040 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8ccef3640 1 -- 172.21.15.83:0/2399439975 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40ab9b0 con 0x7fc8c80d1040 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- 172.21.15.83:0/2399439975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c80d1040 msgr2=0x7fc8c8144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c80d1040 0x7fc8c8144080 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009870 tx=0x7fc8c401c560 comp rx=0 tx=0).stop 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- 172.21.15.83:0/2399439975 shutdown_connections 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c815e080 0x7fc8c8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 --2- 172.21.15.83:0/2399439975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c80d1040 0x7fc8c8144080 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- 172.21.15.83:0/2399439975 >> 172.21.15.83:0/2399439975 conn(0x7fc8c806cd90 msgr2=0x7fc8c80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- 172.21.15.83:0/2399439975 shutdown_connections 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- 172.21.15.83:0/2399439975 wait complete. 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 Processor -- start 2024-05-17T22:07:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.419+0000 7fc8d0159640 1 -- start start 2024-05-17T22:07:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1040 0x7fc8c8145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815e490 0x7fc8c813c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c816abe0 con 0x7fc8c815e080 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c816aa60 con 0x7fc8c815e490 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c816ad60 con 0x7fc8c80d1040 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41406/0 (socket says 172.21.15.83:41406) 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 -- 172.21.15.83:0/4050813080 learned_addr learned my addr 172.21.15.83:0/4050813080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cdef5640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1040 0x7fc8c8145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 -- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1040 msgr2=0x7fc8c8145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8ce6f6640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815e490 0x7fc8c813c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1040 0x7fc8c8145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 -- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815e490 msgr2=0x7fc8c813c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815e490 0x7fc8c813c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 -- 172.21.15.83:0/4050813080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c81475a0 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8cd6f4640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800d800 tx=0x7fc8b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800b6c0 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b8019070 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800ba80 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c8142980 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c8142f40 con 0x7fc8c815e080 2024-05-17T22:07:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.423+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8144080 con 0x7fc8c815e080 2024-05-17T22:07:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.427+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8b800bc20 con 0x7fc8c815e080 2024-05-17T22:07:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.427+0000 7fc8be7fc640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 0x7fc8ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.427+0000 7fc8cdef5640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 0x7fc8ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.427+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7fc8b808f270 con 0x7fc8c815e080 2024-05-17T22:07:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.427+0000 7fc8cdef5640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 0x7fc8ac071910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40af000 tx=0x7fc8c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.431+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc8b805c5d0 con 0x7fc8c815e080 2024-05-17T22:07:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.595+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc8c80b29d0 con 0x7fc8c815e080 2024-05-17T22:07:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.595+0000 7fc8be7fc640 1 -- 172.21.15.83:0/4050813080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) ==== 84+0+65 (secure 0 0 0) 0x7fc8b805bed0 con 0x7fc8c815e080 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:09.589 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-05-17T22:07:09.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 msgr2=0x7fc8ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 0x7fc8ac071910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40af000 tx=0x7fc8c4008040 comp rx=0 tx=0).stop 2024-05-17T22:07:09.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 msgr2=0x7fc8c813bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800d800 tx=0x7fc8b800dcc0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 shutdown_connections 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc8ac06f400 0x7fc8ac071910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8c815e490 0x7fc8c813c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8c815e080 0x7fc8c813bc10 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 --2- 172.21.15.83:0/4050813080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8c80d1040 0x7fc8c8145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 >> 172.21.15.83:0/4050813080 conn(0x7fc8c806cd90 msgr2=0x7fc8c815f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 shutdown_connections 2024-05-17T22:07:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.599+0000 7fc8d0159640 1 -- 172.21.15.83:0/4050813080 wait complete. 2024-05-17T22:07:09.611 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-17T22:07:09.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-17T22:07:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 Processor -- start 2024-05-17T22:07:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 -- start start 2024-05-17T22:07:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efc080af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc080b4bd0 con 0x7efc0815fa70 2024-05-17T22:07:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc080b4ea0 con 0x7efc08072e50 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc0e88d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efc080b5460 con 0x7efc080aef50 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc07fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efc0815fa70 0x7efc080b44c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60684/0 (socket says 172.21.15.83:60684) 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc07fff640 1 -- 172.21.15.83:0/3850264120 learned_addr learned my addr 172.21.15.83:0/3850264120 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1882920462 0 0) 0x7efc080b4bd0 con 0x7efc0815fa70 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbf0003530 con 0x7efc0815fa70 2024-05-17T22:07:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.767+0000 7efc077fe640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efc080af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4053022837 0 0) 0x7efbf0003530 con 0x7efc0815fa70 2024-05-17T22:07:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 msgr2=0x7efc080af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efc080af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 >> v1:172.21.15.155:6789/0 conn(0x7efc08072e50 legacy=0x7efc08073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080b5180 con 0x7efc0815fa70 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7efbf8001700 con 0x7efc0815fa70 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 >> v1:172.21.15.83:6789/0 conn(0x7efc0815fa70 legacy=0x7efc080b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efbf0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efbf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf0004e70 0x7efbf00052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc080b5460 con 0x7efbf00040e0 2024-05-17T22:07:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efbf000bd70 con 0x7efbf0004e70 2024-05-17T22:07:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efbf000bf70 con 0x7efc080aef50 2024-05-17T22:07:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efbf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc07fff640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efbf0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0ce2a640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf0004e70 0x7efbf00052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 -- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 msgr2=0x7efbf0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080aef50 0x7efbf0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 -- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf0004e70 msgr2=0x7efbf00052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf0004e70 0x7efbf00052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 -- 172.21.15.83:0/3850264120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf0003530 con 0x7efbf00040e0 2024-05-17T22:07:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0ce2a640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf0004e70 0x7efbf00052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc077fe640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efbf0004810 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00ba30 tx=0x7efbfc01c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109b8372e1382aee server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00b600 con 0x7efbf00040e0 2024-05-17T22:07:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc0b1070 con 0x7efbf00040e0 2024-05-17T22:07:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc06ffd640 1 -- 172.21.15.83:0/3850264120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0ac870 con 0x7efbf00040e0 2024-05-17T22:07:09.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 -- 172.21.15.83:0/3850264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 msgr2=0x7efbf0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3850264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efbf0004810 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00ba30 tx=0x7efbfc01c690 comp rx=0 tx=0).stop 2024-05-17T22:07:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0ce2a640 1 -- 172.21.15.83:0/3850264120 reap_dead start 2024-05-17T22:07:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 -- 172.21.15.83:0/3850264120 shutdown_connections 2024-05-17T22:07:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 -- 172.21.15.83:0/3850264120 >> 172.21.15.83:0/3850264120 conn(0x7efc0806cd90 msgr2=0x7efc080b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:09.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 -- 172.21.15.83:0/3850264120 shutdown_connections 2024-05-17T22:07:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.771+0000 7efc0e88d640 1 -- 172.21.15.83:0/3850264120 wait complete. 2024-05-17T22:07:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 Processor -- start 2024-05-17T22:07:09.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- start start 2024-05-17T22:07:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 0x7efc080b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080b18b0 0x7efc08142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc08070d00 con 0x7efbf00040e0 2024-05-17T22:07:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efc08070b80 con 0x7efc08160120 2024-05-17T22:07:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7efc0816b7e0 con 0x7efc080b18b0 2024-05-17T22:07:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41420/0 (socket says 172.21.15.83:41420) 2024-05-17T22:07:09.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 -- 172.21.15.83:0/3568602669 learned_addr learned my addr 172.21.15.83:0/3568602669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0ce2a640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080b18b0 0x7efc08142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc077fe640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 0x7efc080b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 -- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080b18b0 msgr2=0x7efc08142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080b18b0 0x7efc08142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 -- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 msgr2=0x7efc080b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 0x7efc080b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 -- 172.21.15.83:0/3568602669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080b4bd0 con 0x7efbf00040e0 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc077fe640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 0x7efc080b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc07fff640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efbf8001ec0 tx=0x7efbf800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800c870 con 0x7efbf00040e0 2024-05-17T22:07:09.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf8018070 con 0x7efbf00040e0 2024-05-17T22:07:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf800cc70 con 0x7efbf00040e0 2024-05-17T22:07:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc080b1cc0 con 0x7efbf00040e0 2024-05-17T22:07:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.775+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc081413b0 con 0x7efbf00040e0 2024-05-17T22:07:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc08073260 con 0x7efbf00040e0 2024-05-17T22:07:09.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efbf801d460 con 0x7efbf00040e0 2024-05-17T22:07:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc04ff9640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 0x7efbf4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7efbf808ed00 con 0x7efbf00040e0 2024-05-17T22:07:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc077fe640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 0x7efbf4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.779+0000 7efc077fe640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 0x7efbf4071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0b0000 tx=0x7efbfc0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:09.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.783+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7efc080b1cc0 con 0x7efbf00040e0 2024-05-17T22:07:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:09.931+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efc080d2020 con 0x7efbf00040e0 2024-05-17T22:07:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.843+0000 7efc04ff9640 1 -- 172.21.15.83:0/3568602669 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) ==== 138+0+0 (secure 0 0 0) 0x7efbf805bfd0 con 0x7efbf00040e0 2024-05-17T22:07:10.838 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 msgr2=0x7efbf4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 0x7efbf4071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0b0000 tx=0x7efbfc0046d0 comp rx=0 tx=0).stop 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 msgr2=0x7efc081408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efbf8001ec0 tx=0x7efbf800ec80 comp rx=0 tx=0).stop 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 shutdown_connections 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efc080b18b0 0x7efc08142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:10.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7efbf406f400 0x7efbf4071910 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:10.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc08160120 0x7efc080b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:10.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 --2- 172.21.15.83:0/3568602669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf00040e0 0x7efc081408f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:10.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 >> 172.21.15.83:0/3568602669 conn(0x7efc0806cd90 msgr2=0x7efc080b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:10.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 shutdown_connections 2024-05-17T22:07:10.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:10.851+0000 7efc0e88d640 1 -- 172.21.15.83:0/3568602669 wait complete. 2024-05-17T22:07:10.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-17T22:07:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 Processor -- start 2024-05-17T22:07:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 -- start start 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa08073590 0x7faa080d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa08143f20 0x7faa081447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa080d10d0 con 0x7faa080b3d00 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7faa080b4110 con 0x7faa08143f20 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa0ccad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa08070770 con 0x7faa08073590 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa08143f20 0x7faa081447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa08143f20 0x7faa081447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41428/0 (socket says 172.21.15.83:41428) 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 -- 172.21.15.83:0/2088288405 learned_addr learned my addr 172.21.15.83:0/2088288405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa05d9b640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa08073590 0x7faa080d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 -- 172.21.15.83:0/2088288405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa08073590 msgr2=0x7faa080d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa08073590 0x7faa080d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 -- 172.21.15.83:0/2088288405 >> v1:172.21.15.155:6789/0 conn(0x7faa080b3d00 legacy=0x7faa08072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.043+0000 7faa06d9d640 1 -- 172.21.15.83:0/2088288405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa08070120 con 0x7faa08143f20 2024-05-17T22:07:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa06d9d640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08143f20 0x7faa081447f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9951df3b371f435 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0559a640 1 -- 172.21.15.83:0/2088288405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000a5790 con 0x7faa08143f20 2024-05-17T22:07:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0559a640 1 -- 172.21.15.83:0/2088288405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa000a5930 con 0x7faa08143f20 2024-05-17T22:07:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0559a640 1 -- 172.21.15.83:0/2088288405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000a5d00 con 0x7faa08143f20 2024-05-17T22:07:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 -- 172.21.15.83:0/2088288405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08143f20 msgr2=0x7faa081447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08143f20 0x7faa081447f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0001ca30 comp rx=0 tx=0).stop 2024-05-17T22:07:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 -- 172.21.15.83:0/2088288405 shutdown_connections 2024-05-17T22:07:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08143f20 0x7faa081447f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 --2- 172.21.15.83:0/2088288405 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa08073590 0x7faa080d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 -- 172.21.15.83:0/2088288405 >> 172.21.15.83:0/2088288405 conn(0x7faa0806cd90 msgr2=0x7faa080b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 -- 172.21.15.83:0/2088288405 shutdown_connections 2024-05-17T22:07:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.047+0000 7faa0ccad640 1 -- 172.21.15.83:0/2088288405 wait complete. 2024-05-17T22:07:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 Processor -- start 2024-05-17T22:07:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- start start 2024-05-17T22:07:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa080b3d00 0x7faa0813f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa08143f20 0x7faa0815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa08070c00 con 0x7faa08073590 2024-05-17T22:07:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faa08145540 con 0x7faa08143f20 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7faa081f8310 con 0x7faa080b3d00 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41442/0 (socket says 172.21.15.83:41442) 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 -- 172.21.15.83:0/3259241366 learned_addr learned my addr 172.21.15.83:0/3259241366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa05d9b640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa080b3d00 0x7faa0813f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa06d9d640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa08143f20 0x7faa0815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 -- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa080b3d00 msgr2=0x7faa0813f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa080b3d00 0x7faa0813f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 -- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa08143f20 msgr2=0x7faa0815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa08143f20 0x7faa0815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 -- 172.21.15.83:0/3259241366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0813f570 con 0x7faa08073590 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0659c640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c420 tx=0x7fa9f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017070 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f80052d0 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa08161c40 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80054c0 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa08162200 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.051+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa081447f0 con 0x7faa08073590 2024-05-17T22:07:11.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.055+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa9f801a070 con 0x7faa08073590 2024-05-17T22:07:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.055+0000 7fa9f6ffd640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 0x7fa9e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.055+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5227+0+0 (secure 0 0 0) 0x7fa9f808e650 con 0x7faa08073590 2024-05-17T22:07:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.055+0000 7faa05d9b640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 0x7fa9e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.055+0000 7faa05d9b640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 0x7fa9e8071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faa081426d0 tx=0x7fa9fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.059+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa9f8021020 con 0x7faa08073590 2024-05-17T22:07:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.219+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faa08142420 con 0x7faa08073590 2024-05-17T22:07:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.858+0000 7fa9f6ffd640 1 -- 172.21.15.83:0/3259241366 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) ==== 170+0+0 (secure 0 0 0) 0x7fa9f805b2d0 con 0x7faa08073590 2024-05-17T22:07:11.854 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-05-17T22:07:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 msgr2=0x7fa9e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 0x7fa9e8071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faa081426d0 tx=0x7fa9fc008040 comp rx=0 tx=0).stop 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 msgr2=0x7faa081f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c420 tx=0x7fa9f800c8e0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 shutdown_connections 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa9e806f400 0x7fa9e8071910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa08143f20 0x7faa0815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faa080b3d00 0x7faa0813f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.862+0000 7faa0ccad640 1 --2- 172.21.15.83:0/3259241366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa08073590 0x7faa081f9a40 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:11.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.866+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 >> 172.21.15.83:0/3259241366 conn(0x7faa0806cd90 msgr2=0x7faa08141620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:11.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.866+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 shutdown_connections 2024-05-17T22:07:11.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:11.866+0000 7faa0ccad640 1 -- 172.21.15.83:0/3259241366 wait complete. 2024-05-17T22:07:11.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-17T22:07:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 Processor -- start 2024-05-17T22:07:12.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 -- start start 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1b4c06c790 0x7f1b4c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4c06cc90 con 0x7f1b4c0b5670 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4c06cf70 con 0x7f1b4c15fcc0 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b5253c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4c0711a0 con 0x7f1b4c06c790 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b50ad9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b4c0b5670 0x7f1b4c0b5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60696/0 (socket says 172.21.15.83:60696) 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b50ad9640 1 -- 172.21.15.83:0/1576691581 learned_addr learned my addr 172.21.15.83:0/1576691581 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710123603 0 0) 0x7f1b4c06cc90 con 0x7f1b4c0b5670 2024-05-17T22:07:12.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b34003530 con 0x7f1b4c0b5670 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.166+0000 7f1b4bfff640 1 --2- 172.21.15.83:0/1576691581 >> v2:172.21.15.155:3300/0 conn(0x7f1b4c06c790 0x7f1b4c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856035277 0 0) 0x7f1b34003530 con 0x7f1b4c0b5670 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 >> v2:172.21.15.155:3300/0 conn(0x7f1b4c06c790 msgr2=0x7f1b4c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 --2- 172.21.15.83:0/1576691581 >> v2:172.21.15.155:3300/0 conn(0x7f1b4c06c790 0x7f1b4c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 >> v1:172.21.15.155:6789/0 conn(0x7f1b4c15fcc0 legacy=0x7f1b4c070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c06d250 con 0x7f1b4c0b5670 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1b44001a80 con 0x7f1b4c0b5670 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 >> v1:172.21.15.83:6789/0 conn(0x7f1b4c0b5670 legacy=0x7f1b4c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b34003a60 0x7f1b34003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b34004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b34005280 0x7f1b3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0711a0 con 0x7f1b34004810 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3400c210 con 0x7f1b34003a60 2024-05-17T22:07:12.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3400c410 con 0x7f1b34005280 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b34004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4bfff640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b34003a60 0x7f1b34003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 -- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b34005280 msgr2=0x7f1b3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b34005280 0x7f1b3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 -- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b34003a60 msgr2=0x7f1b34003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b34003a60 0x7f1b34003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b50ad9640 1 -- 172.21.15.83:0/1576691581 reap_dead start 2024-05-17T22:07:12.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 -- 172.21.15.83:0/1576691581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b34003530 con 0x7f1b34004810 2024-05-17T22:07:12.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b34004c20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1b40001790 tx=0x7f1b4001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c644941267a209f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:12.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400013e0 con 0x7f1b34004810 2024-05-17T22:07:12.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b400052d0 con 0x7f1b34004810 2024-05-17T22:07:12.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b4affd640 1 -- 172.21.15.83:0/1576691581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40003840 con 0x7f1b34004810 2024-05-17T22:07:12.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 -- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 msgr2=0x7f1b34004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b34004c20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1b40001790 tx=0x7f1b4001c3f0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 -- 172.21.15.83:0/1576691581 shutdown_connections 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1576691581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b34004c20 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 -- 172.21.15.83:0/1576691581 >> 172.21.15.83:0/1576691581 conn(0x7f1b4c0d20b0 msgr2=0x7f1b4c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 -- 172.21.15.83:0/1576691581 shutdown_connections 2024-05-17T22:07:12.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.170+0000 7f1b5253c640 1 -- 172.21.15.83:0/1576691581 wait complete. 2024-05-17T22:07:12.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 Processor -- start 2024-05-17T22:07:12.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 -- start start 2024-05-17T22:07:12.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4c154ac0 0x7f1b4c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 0x7f1b4c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c06dd30 con 0x7f1b34004810 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c06d810 con 0x7f1b4c0af520 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.174+0000 7f1b5253c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4c06deb0 con 0x7f1b4c154ac0 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41446/0 (socket says 172.21.15.83:41446) 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 -- 172.21.15.83:0/1614999589 learned_addr learned my addr 172.21.15.83:0/1614999589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b50ad9640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 0x7f1b4c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4c154ac0 0x7f1b4c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 -- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4c154ac0 msgr2=0x7f1b4c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4c154ac0 0x7f1b4c1526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 -- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 msgr2=0x7f1b4c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 0x7f1b4c0af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 -- 172.21.15.83:0/1614999589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c0afe70 con 0x7f1b34004810 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b50ad9640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 0x7f1b4c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:12.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4bfff640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c011500 tx=0x7f1b3c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c015870 con 0x7f1b34004810 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c015a10 con 0x7f1b34004810 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c152e20 con 0x7f1b34004810 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c015dc0 con 0x7f1b34004810 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c153100 con 0x7f1b34004810 2024-05-17T22:07:12.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c070a90 con 0x7f1b34004810 2024-05-17T22:07:12.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1b3c013400 con 0x7f1b34004810 2024-05-17T22:07:12.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 0x7f1b38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:12.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5609+0+0 (secure 0 0 0) 0x7f1b3c08f7a0 con 0x7f1b34004810 2024-05-17T22:07:12.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 0x7f1b38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:12.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.178+0000 7f1b4b7fe640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 0x7f1b38071910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad000 tx=0x7f1b400011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:12.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.182+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1b4c0afe70 con 0x7f1b34004810 2024-05-17T22:07:12.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.382+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1b4c153340 con 0x7f1b34004810 2024-05-17T22:07:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.862+0000 7f1b48ff9640 1 -- 172.21.15.83:0/1614999589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v93) ==== 174+0+0 (secure 0 0 0) 0x7f1b3c019030 con 0x7f1b34004810 2024-05-17T22:07:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 16 and data pool 17 2024-05-17T22:07:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 msgr2=0x7f1b38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 0x7f1b38071910 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad000 tx=0x7f1b400011a0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 msgr2=0x7f1b4c147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c011500 tx=0x7f1b3c0119c0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 shutdown_connections 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f1b3806f400 0x7f1b38071910 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad000 tx=0x7f1b400011a0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b4c0af520 0x7f1b4c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b4c154ac0 0x7f1b4c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 --2- 172.21.15.83:0/1614999589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b34004810 0x7f1b4c147880 secure :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c011500 tx=0x7f1b3c0119c0 comp rx=0 tx=0).stop 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 >> 172.21.15.83:0/1614999589 conn(0x7f1b4c0d20b0 msgr2=0x7f1b4c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 shutdown_connections 2024-05-17T22:07:12.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:12.866+0000 7f1b5253c640 1 -- 172.21.15.83:0/1614999589 wait complete. 2024-05-17T22:07:12.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:07:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 Processor -- start 2024-05-17T22:07:13.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 -- start start 2024-05-17T22:07:13.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffa7c06ffd0 0x7ffa7c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c1465d0 con 0x7ffa7c1605a0 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c070920 con 0x7ffa7c1461c0 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa83e76640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c06ef30 con 0x7ffa7c06ffd0 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa82413640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffa7c1605a0 0x7ffa7c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60712/0 (socket says 172.21.15.83:60712) 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa82413640 1 -- 172.21.15.83:0/1231348585 learned_addr learned my addr 172.21.15.83:0/1231348585 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81411640 1 --2- 172.21.15.83:0/1231348585 >> v2:172.21.15.155:3300/0 conn(0x7ffa7c06ffd0 0x7ffa7c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291856651 0 0) 0x7ffa7c1465d0 con 0x7ffa7c1605a0 2024-05-17T22:07:13.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa68003530 con 0x7ffa7c1605a0 2024-05-17T22:07:13.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3578094432 0 0) 0x7ffa68003530 con 0x7ffa7c1605a0 2024-05-17T22:07:13.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 >> v2:172.21.15.155:3300/0 conn(0x7ffa7c06ffd0 msgr2=0x7ffa7c0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:13.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 --2- 172.21.15.83:0/1231348585 >> v2:172.21.15.155:3300/0 conn(0x7ffa7c06ffd0 0x7ffa7c0703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 >> v1:172.21.15.155:6789/0 conn(0x7ffa7c1461c0 legacy=0x7ffa7c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c06f210 con 0x7ffa7c1605a0 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7ffa74001a80 con 0x7ffa7c1605a0 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 >> v1:172.21.15.83:6789/0 conn(0x7ffa7c1605a0 legacy=0x7ffa7c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 0x7ffa68003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa68004810 0x7ffa68004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68005260 0x7ffa6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c06ef30 con 0x7ffa68003a60 2024-05-17T22:07:13.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6800c1f0 con 0x7ffa68004810 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6800c3f0 con 0x7ffa68005260 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 0x7ffa68003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81411640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa68004810 0x7ffa68004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa82413640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68005260 0x7ffa6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 -- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68005260 msgr2=0x7ffa6800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68005260 0x7ffa6800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 -- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa68004810 msgr2=0x7ffa68004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa68004810 0x7ffa68004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81c12640 1 -- 172.21.15.83:0/1231348585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68003530 con 0x7ffa68003a60 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa82413640 1 -- 172.21.15.83:0/1231348585 reap_dead start 2024-05-17T22:07:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.066+0000 7ffa81411640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa68004810 0x7ffa68004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa81c12640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 0x7ffa68003ed0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0b11f0 tx=0x7ffa7801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eeddf79ca1ebab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780013e0 con 0x7ffa68003a60 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa780052d0 con 0x7ffa68003a60 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa80c10640 1 -- 172.21.15.83:0/1231348585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78003840 con 0x7ffa68003a60 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 -- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 msgr2=0x7ffa68003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 0x7ffa68003ed0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0b11f0 tx=0x7ffa7801c380 comp rx=0 tx=0).stop 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 -- 172.21.15.83:0/1231348585 shutdown_connections 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 --2- 172.21.15.83:0/1231348585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa68003a60 0x7ffa68003ed0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 -- 172.21.15.83:0/1231348585 >> 172.21.15.83:0/1231348585 conn(0x7ffa7c06cd90 msgr2=0x7ffa7c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 -- 172.21.15.83:0/1231348585 shutdown_connections 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.070+0000 7ffa83e76640 1 -- 172.21.15.83:0/1231348585 wait complete. 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 Processor -- start 2024-05-17T22:07:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- start start 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68003a60 0x7ffa7c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa7c1f8a00 0x7ffa7c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c160dc0 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0d0890 con 0x7ffa7c1f8a00 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c140b00 con 0x7ffa68003a60 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41462/0 (socket says 172.21.15.83:41462) 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 -- 172.21.15.83:0/3227400271 learned_addr learned my addr 172.21.15.83:0/3227400271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa81411640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa7c1f8a00 0x7ffa7c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa81c12640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68003a60 0x7ffa7c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 -- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68003a60 msgr2=0x7ffa7c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68003a60 0x7ffa7c15bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 -- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa7c1f8a00 msgr2=0x7ffa7c0b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa7c1f8a00 0x7ffa7c0b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 -- 172.21.15.83:0/3227400271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c13ed20 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa82413640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c072dc0 tx=0x7ffa7400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74004d70 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7400bb80 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c1f94a0 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007070 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.074+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c13efc0 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c0d0c20 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffa7401b060 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa727fc640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 0x7ffa60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa81c12640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 0x7ffa60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7408f990 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.078+0000 7ffa81c12640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 0x7ffa60071940 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ac000 tx=0x7ffa780011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.082+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffa74022020 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.234+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa7c1f9230 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa727fc640 1 -- 172.21.15.83:0/3227400271 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16616 (secure 0 0 0) 0x7ffa7405c860 con 0x7ffa7c1f8e10 2024-05-17T22:07:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:13.232 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":108,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:12.387634+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:07:09.949308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-17T22:07:11.233706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:07:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 msgr2=0x7ffa60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 0x7ffa60071940 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffa780ac000 tx=0x7ffa780011a0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 msgr2=0x7ffa7c0b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c072dc0 tx=0x7ffa7400b4a0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 shutdown_connections 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa7c1f8e10 0x7ffa7c0b2330 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa6006f430 0x7ffa60071940 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa7c1f8a00 0x7ffa7c0b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 --2- 172.21.15.83:0/3227400271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa68003a60 0x7ffa7c15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 >> 172.21.15.83:0/3227400271 conn(0x7ffa7c06cd90 msgr2=0x7ffa7c143e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 shutdown_connections 2024-05-17T22:07:13.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.238+0000 7ffa83e76640 1 -- 172.21.15.83:0/3227400271 wait complete. 2024-05-17T22:07:13.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:07:13.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 Processor -- start 2024-05-17T22:07:13.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 -- start start 2024-05-17T22:07:13.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd2d815ffa0 0x7fd2d80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d80d34e0 con 0x7fd2d80d1150 2024-05-17T22:07:13.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d80d3780 con 0x7fd2d806e3d0 2024-05-17T22:07:13.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dfe10640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d80d3a60 con 0x7fd2d815ffa0 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2ddbac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd2d815ffa0 0x7fd2d80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dd3ab640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd2d80d1150 0x7fd2d806dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60720/0 (socket says 172.21.15.83:60720) 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2dd3ab640 1 -- 172.21.15.83:0/3612250828 learned_addr learned my addr 172.21.15.83:0/3612250828 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2ddbac640 1 -- 172.21.15.83:0/3612250828 >> v1:172.21.15.155:6789/0 conn(0x7fd2d806e3d0 legacy=0x7fd2d80d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2ddbac640 1 -- 172.21.15.83:0/3612250828 >> v1:172.21.15.83:6789/0 conn(0x7fd2d80d1150 legacy=0x7fd2d806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.406+0000 7fd2ddbac640 1 -- 172.21.15.83:0/3612250828 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d80d3d40 con 0x7fd2d815ffa0 2024-05-17T22:07:13.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2ddbac640 1 --2- 172.21.15.83:0/3612250828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d815ffa0 0x7fd2d80d0c10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097c0 tx=0x7fd2d401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc4417e5e416879 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dcbaa640 1 -- 172.21.15.83:0/3612250828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40b0070 con 0x7fd2d815ffa0 2024-05-17T22:07:13.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dcbaa640 1 -- 172.21.15.83:0/3612250828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d40ab600 con 0x7fd2d815ffa0 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dcbaa640 1 -- 172.21.15.83:0/3612250828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40ab9d0 con 0x7fd2d815ffa0 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 -- 172.21.15.83:0/3612250828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d815ffa0 msgr2=0x7fd2d80d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/3612250828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d815ffa0 0x7fd2d80d0c10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097c0 tx=0x7fd2d401c330 comp rx=0 tx=0).stop 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 -- 172.21.15.83:0/3612250828 shutdown_connections 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/3612250828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d815ffa0 0x7fd2d80d0c10 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 -- 172.21.15.83:0/3612250828 >> 172.21.15.83:0/3612250828 conn(0x7fd2d80d28f0 msgr2=0x7fd2d80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:13.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 -- 172.21.15.83:0/3612250828 shutdown_connections 2024-05-17T22:07:13.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.410+0000 7fd2dfe10640 1 -- 172.21.15.83:0/3612250828 wait complete. 2024-05-17T22:07:13.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 Processor -- start 2024-05-17T22:07:13.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- start start 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2d80d1150 0x7fd2d8161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2d815ffa0 0x7fd2d815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d80d17e0 con 0x7fd2d80cc920 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d80d1660 con 0x7fd2d815ffa0 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d80d4970 con 0x7fd2d80d1150 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41478/0 (socket says 172.21.15.83:41478) 2024-05-17T22:07:13.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 -- 172.21.15.83:0/968468520 learned_addr learned my addr 172.21.15.83:0/968468520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dd3ab640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2d815ffa0 0x7fd2d815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2ddbac640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2d80d1150 0x7fd2d8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 -- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2d80d1150 msgr2=0x7fd2d8161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2d80d1150 0x7fd2d8161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 -- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2d815ffa0 msgr2=0x7fd2d815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2d815ffa0 0x7fd2d815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 -- 172.21.15.83:0/968468520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d80cd650 con 0x7fd2d80cc920 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2de3ad640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b9e0 tx=0x7fd2d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0003b80 con 0x7fd2d80cc920 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d0003d20 con 0x7fd2d80cc920 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d8160550 con 0x7fd2d80cc920 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000cb60 con 0x7fd2d80cc920 2024-05-17T22:07:13.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.414+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d8160b10 con 0x7fd2d80cc920 2024-05-17T22:07:13.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d80cd3c0 con 0x7fd2d80cc920 2024-05-17T22:07:13.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2d0006d10 con 0x7fd2d80cc920 2024-05-17T22:07:13.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2ce7fc640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 0x7fd2bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7fd2d008ed70 con 0x7fd2d80cc920 2024-05-17T22:07:13.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2ddbac640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 0x7fd2bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.418+0000 7fd2ddbac640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 0x7fd2bc071910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40ad000 tx=0x7fd2d40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.422+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd2d005bcf0 con 0x7fd2d80cc920 2024-05-17T22:07:13.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.582+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2d80d0c10 con 0x7fd2d80cc920 2024-05-17T22:07:13.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2ce7fc640 1 -- 172.21.15.83:0/968468520 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16616 (secure 0 0 0) 0x7fd2d005b5f0 con 0x7fd2d80cc920 2024-05-17T22:07:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:13.579 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":108,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:12.387634+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:07:09.949308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-17T22:07:11.233706+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:07:13.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 msgr2=0x7fd2bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 0x7fd2bc071910 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40ad000 tx=0x7fd2d40a6000 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 msgr2=0x7fd2d80ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b9e0 tx=0x7fd2d000bea0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 shutdown_connections 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d80cc920 0x7fd2d80ccd30 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd2bc06f400 0x7fd2bc071910 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2d815ffa0 0x7fd2d815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 --2- 172.21.15.83:0/968468520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2d80d1150 0x7fd2d8161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 >> 172.21.15.83:0/968468520 conn(0x7fd2d80d28f0 msgr2=0x7fd2d80d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 shutdown_connections 2024-05-17T22:07:13.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.586+0000 7fd2dfe10640 1 -- 172.21.15.83:0/968468520 wait complete. 2024-05-17T22:07:13.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-17T22:07:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 Processor -- start 2024-05-17T22:07:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 -- start start 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c009ba00 0x7f69c009be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69c0071400 0x7f69c0071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c009c350 con 0x7f69c00b88b0 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f69c00d1a40 con 0x7f69c0071400 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c6348640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69c009d040 con 0x7f69c009ba00 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69c0071400 0x7f69c0071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f69c0071400 0x7f69c0071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41494/0 (socket says 172.21.15.83:41494) 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69bffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f69c00b88b0 0x7f69c00d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60726/0 (socket says 172.21.15.83:60726) 2024-05-17T22:07:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 -- 172.21.15.83:0/1954433974 learned_addr learned my addr 172.21.15.83:0/1954433974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69b7fff640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c009ba00 0x7f69c009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 -- 172.21.15.83:0/1954433974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c009ba00 msgr2=0x7f69c009be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c009ba00 0x7f69c009be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 -- 172.21.15.83:0/1954433974 >> v1:172.21.15.83:6789/0 conn(0x7f69c00b88b0 legacy=0x7f69c00d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69c48e5640 1 -- 172.21.15.83:0/1954433974 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c009c710 con 0x7f69c0071400 2024-05-17T22:07:13.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.766+0000 7f69bf7fe640 1 -- 172.21.15.83:0/1954433974 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309816220 0 0) 0x7f69b0001120 con 0x7f69c00b88b0 2024-05-17T22:07:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c48e5640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c0071400 0x7f69c0071810 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009e80 tx=0x7f69b801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67696a86d5fd50b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69bf7fe640 1 -- 172.21.15.83:0/1954433974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80a56c0 con 0x7f69c0071400 2024-05-17T22:07:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69bf7fe640 1 -- 172.21.15.83:0/1954433974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b80a5860 con 0x7f69c0071400 2024-05-17T22:07:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69bf7fe640 1 -- 172.21.15.83:0/1954433974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80a5c30 con 0x7f69c0071400 2024-05-17T22:07:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 -- 172.21.15.83:0/1954433974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c0071400 msgr2=0x7f69c0071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c0071400 0x7f69c0071810 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009e80 tx=0x7f69b801c8f0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 -- 172.21.15.83:0/1954433974 shutdown_connections 2024-05-17T22:07:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c0071400 0x7f69c0071810 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 --2- 172.21.15.83:0/1954433974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c009ba00 0x7f69c009be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 -- 172.21.15.83:0/1954433974 >> 172.21.15.83:0/1954433974 conn(0x7f69c00d3620 msgr2=0x7f69c00d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 -- 172.21.15.83:0/1954433974 shutdown_connections 2024-05-17T22:07:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.770+0000 7f69c6348640 1 -- 172.21.15.83:0/1954433974 wait complete. 2024-05-17T22:07:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 Processor -- start 2024-05-17T22:07:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- start start 2024-05-17T22:07:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 0x7f69c0155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69c009ba00 0x7f69c00c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69c016a870 con 0x7f69c00b88b0 2024-05-17T22:07:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f69c016a6f0 con 0x7f69c009ba00 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f69c016a9f0 con 0x7f69c0071400 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41506/0 (socket says 172.21.15.83:41506) 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 -- 172.21.15.83:0/2361281242 learned_addr learned my addr 172.21.15.83:0/2361281242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69bffff640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 0x7f69c0155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69b7fff640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69c009ba00 0x7f69c00c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 -- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 msgr2=0x7f69c0155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 0x7f69c0155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 -- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69c009ba00 msgr2=0x7f69c00c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69c009ba00 0x7f69c00c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 -- 172.21.15.83:0/2361281242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01563c0 con 0x7f69c00b88b0 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69bffff640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 0x7f69c0155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c48e5640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f69b801a520 tx=0x7f69b8019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80bc020 con 0x7f69c00b88b0 2024-05-17T22:07:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b80b8670 con 0x7f69c00b88b0 2024-05-17T22:07:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c00b3060 con 0x7f69c00b88b0 2024-05-17T22:07:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8009510 con 0x7f69c00b88b0 2024-05-17T22:07:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c00b3620 con 0x7f69c00b88b0 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.774+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c0156130 con 0x7f69c00b88b0 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.778+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69b80bf020 con 0x7f69c00b88b0 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.778+0000 7f69bd7fa640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 0x7f69a4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.778+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f69b812a450 con 0x7f69c00b88b0 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.778+0000 7f69bffff640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 0x7f69a4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.778+0000 7f69bffff640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 0x7f69a4071940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f69b0007fd0 tx=0x7f69b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:13.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.782+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f69b80c6030 con 0x7f69c00b88b0 2024-05-17T22:07:13.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:13.986+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f69c0161190 con 0x7f69c00b88b0 2024-05-17T22:07:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.890+0000 7f69bd7fa640 1 -- 172.21.15.83:0/2361281242 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) ==== 118+0+0 (secure 0 0 0) 0x7f69b8016600 con 0x7f69c00b88b0 2024-05-17T22:07:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 msgr2=0x7f69a4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 0x7f69a4071940 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f69b0007fd0 tx=0x7f69b0009040 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 msgr2=0x7f69c0154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f69b801a520 tx=0x7f69b8019690 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 shutdown_connections 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69c00b88b0 0x7f69c0154870 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f69a406f430 0x7f69a4071940 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69c009ba00 0x7f69c00c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 --2- 172.21.15.83:0/2361281242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69c0071400 0x7f69c0155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 >> 172.21.15.83:0/2361281242 conn(0x7f69c00d3620 msgr2=0x7f69c00d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 shutdown_connections 2024-05-17T22:07:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:14.894+0000 7f69c6348640 1 -- 172.21.15.83:0/2361281242 wait complete. 2024-05-17T22:07:14.917 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-17T22:07:14.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-17T22:07:15.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 Processor -- start 2024-05-17T22:07:15.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 -- start start 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9d640cf830 0x7f9d640b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640b5640 con 0x7f9d6415d700 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640b58e0 con 0x7f9d640cdf40 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.082+0000 7f9d6ab58640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9d64070870 con 0x7f9d640cf830 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d690f5640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9d6415d700 0x7f9d641663d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60738/0 (socket says 172.21.15.83:60738) 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d690f5640 1 -- 172.21.15.83:0/1073499914 learned_addr learned my addr 172.21.15.83:0/1073499914 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1073499914 >> v2:172.21.15.155:3300/0 conn(0x7f9d640cf830 0x7f9d640b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2378469693 0 0) 0x7f9d640b5640 con 0x7f9d6415d700 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d4c003530 con 0x7f9d6415d700 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1705914898 0 0) 0x7f9d4c003530 con 0x7f9d6415d700 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 >> v2:172.21.15.155:3300/0 conn(0x7f9d640cf830 msgr2=0x7f9d640b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 --2- 172.21.15.83:0/1073499914 >> v2:172.21.15.155:3300/0 conn(0x7f9d640cf830 0x7f9d640b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 >> v1:172.21.15.155:6789/0 conn(0x7f9d640cdf40 legacy=0x7f9d6415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6406f060 con 0x7f9d6415d700 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9d5c001aa0 con 0x7f9d6415d700 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 >> v1:172.21.15.83:6789/0 conn(0x7f9d6415d700 legacy=0x7f9d641663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4c003a60 0x7f9d4c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 0x7f9d4c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c005260 0x7f9d4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c00c1c0 con 0x7f9d4c004810 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c00c370 con 0x7f9d4c005260 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4c00c570 con 0x7f9d4c003a60 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 0x7f9d4c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d690f5640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c005260 0x7f9d4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4c003a60 0x7f9d4c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 -- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4c003a60 msgr2=0x7f9d4c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4c003a60 0x7f9d4c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 -- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c005260 msgr2=0x7f9d4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c005260 0x7f9d4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d690f5640 1 -- 172.21.15.83:0/1073499914 reap_dead start 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 -- 172.21.15.83:0/1073499914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c003530 con 0x7f9d4c004810 2024-05-17T22:07:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d4c003a60 0x7f9d4c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d63fff640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 0x7f9d4c004c20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001790 tx=0x7f9d5801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c52852569fd0c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580013e0 con 0x7f9d4c004810 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d580052d0 con 0x7f9d4c004810 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.086+0000 7f9d637fe640 1 -- 172.21.15.83:0/1073499914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58003840 con 0x7f9d4c004810 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 msgr2=0x7f9d4c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 0x7f9d4c004c20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001790 tx=0x7f9d5801c3d0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1073499914 shutdown_connections 2024-05-17T22:07:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1073499914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4c004810 0x7f9d4c004c20 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1073499914 >> 172.21.15.83:0/1073499914 conn(0x7f9d6406cd90 msgr2=0x7f9d640b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1073499914 shutdown_connections 2024-05-17T22:07:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1073499914 wait complete. 2024-05-17T22:07:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 Processor -- start 2024-05-17T22:07:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- start start 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 0x7f9d6414a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d640ac250 0x7f9d640ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6415dc90 con 0x7f9d640c6430 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6415db10 con 0x7f9d4c004810 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d6ab58640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6415cff0 con 0x7f9d640ac250 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.090+0000 7f9d690f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41526/0 (socket says 172.21.15.83:41526) 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 -- 172.21.15.83:0/1227160793 learned_addr learned my addr 172.21.15.83:0/1227160793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 0x7f9d6414a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d63fff640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d640ac250 0x7f9d640ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 -- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d640ac250 msgr2=0x7f9d640ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d640ac250 0x7f9d640ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 -- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 msgr2=0x7f9d6414a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 0x7f9d6414a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 -- 172.21.15.83:0/1227160793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640c6d80 con 0x7f9d640c6430 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 0x7f9d6414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d690f5640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f9d640d1460 tx=0x7f9d5c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c010040 con 0x7f9d640c6430 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c01a070 con 0x7f9d640c6430 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c013a00 con 0x7f9d640c6430 2024-05-17T22:07:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d640adcd0 con 0x7f9d640c6430 2024-05-17T22:07:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6414ad60 con 0x7f9d640c6430 2024-05-17T22:07:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6415d380 con 0x7f9d640c6430 2024-05-17T22:07:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d5c013ba0 con 0x7f9d640c6430 2024-05-17T22:07:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 0x7f9d50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f9d5c092b70 con 0x7f9d640c6430 2024-05-17T22:07:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 0x7f9d50071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.094+0000 7f9d688f4640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 0x7f9d50071910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d6406b180 tx=0x7f9d54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.098+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9d640c6d80 con 0x7f9d640c6430 2024-05-17T22:07:15.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.290+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9d640accd0 con 0x7f9d640c6430 2024-05-17T22:07:15.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.906+0000 7f9d617fa640 1 -- 172.21.15.83:0/1227160793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) ==== 122+0+0 (secure 0 0 0) 0x7f9d5c05fbd0 con 0x7f9d640c6430 2024-05-17T22:07:15.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 msgr2=0x7f9d50071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 0x7f9d50071910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d6406b180 tx=0x7f9d54008040 comp rx=0 tx=0).stop 2024-05-17T22:07:15.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 msgr2=0x7f9d640c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:15.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f9d640d1460 tx=0x7f9d5c0119a0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 shutdown_connections 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d640c6430 0x7f9d640c6840 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d640ac250 0x7f9d640ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9d5006f400 0x7f9d50071910 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 --2- 172.21.15.83:0/1227160793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4c004810 0x7f9d6414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 >> 172.21.15.83:0/1227160793 conn(0x7f9d6406cd90 msgr2=0x7f9d640b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 shutdown_connections 2024-05-17T22:07:15.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:15.910+0000 7f9d6ab58640 1 -- 172.21.15.83:0/1227160793 wait complete. 2024-05-17T22:07:15.930 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-17T22:07:15.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:16.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 Processor -- start 2024-05-17T22:07:16.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 -- start start 2024-05-17T22:07:16.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f202406ffd0 0x7f20240703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f20241609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20241465d0 con 0x7f20241461c0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f2024160ef0 con 0x7f202406ffd0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20292f8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f202406ef30 con 0x7f20241605a0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2022ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f20241461c0 0x7f202415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60744/0 (socket says 172.21.15.83:60744) 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2022ffd640 1 -- 172.21.15.83:0/2392272557 learned_addr learned my addr 172.21.15.83:0/2392272557 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20237fe640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f20241609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 --2- 172.21.15.83:0/2392272557 >> v2:172.21.15.155:3300/0 conn(0x7f202406ffd0 0x7f20240703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538069025 0 0) 0x7f20241465d0 con 0x7f20241461c0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f200c003530 con 0x7f20241461c0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2845212402 0 0) 0x7f200c003530 con 0x7f20241461c0 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 msgr2=0x7f20241609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f20241609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 >> v2:172.21.15.155:3300/0 conn(0x7f202406ffd0 msgr2=0x7f20240703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 --2- 172.21.15.83:0/2392272557 >> v2:172.21.15.155:3300/0 conn(0x7f202406ffd0 0x7f20240703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024070920 con 0x7f20241461c0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f2014001700 con 0x7f20241461c0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 >> v1:172.21.15.83:6789/0 conn(0x7f20241461c0 legacy=0x7f202415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f200c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003fe0 0x7f200c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f200c004d50 0x7f200c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2024160ef0 con 0x7f200c003fe0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f202406ef30 con 0x7f200c004d50 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f200c00bad0 con 0x7f20241605a0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003fe0 0x7f200c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2022ffd640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f200c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20237fe640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f200c004d50 0x7f200c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 -- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 msgr2=0x7f200c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f200c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 -- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f200c004d50 msgr2=0x7f200c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f200c004d50 0x7f200c005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f20227fc640 1 -- 172.21.15.83:0/2392272557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c003530 con 0x7f200c003fe0 2024-05-17T22:07:16.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.086+0000 7f2022ffd640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20241605a0 0x7f200c003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20227fc640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003fe0 0x7f200c004810 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f2018009ae0 tx=0x7f201801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e4a99602e17fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180b2070 con 0x7f200c003fe0 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180a5560 con 0x7f200c003fe0 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f2021ffb640 1 -- 172.21.15.83:0/2392272557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180096b0 con 0x7f200c003fe0 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 -- 172.21.15.83:0/2392272557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003fe0 msgr2=0x7f200c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 --2- 172.21.15.83:0/2392272557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c003fe0 0x7f200c004810 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f2018009ae0 tx=0x7f201801ca70 comp rx=0 tx=0).stop 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20237fe640 1 -- 172.21.15.83:0/2392272557 reap_dead start 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 -- 172.21.15.83:0/2392272557 shutdown_connections 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 -- 172.21.15.83:0/2392272557 >> 172.21.15.83:0/2392272557 conn(0x7f202406cd90 msgr2=0x7f20240b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 -- 172.21.15.83:0/2392272557 shutdown_connections 2024-05-17T22:07:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.090+0000 7f20292f8640 1 -- 172.21.15.83:0/2392272557 wait complete. 2024-05-17T22:07:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 Processor -- start 2024-05-17T22:07:16.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- start start 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f200c003fe0 0x7f20240b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20240c6540 0x7f20240b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f202406f7d0 con 0x7f20240b5e60 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f202406f4f0 con 0x7f20240c6540 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f202406fab0 con 0x7f200c003fe0 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41544/0 (socket says 172.21.15.83:41544) 2024-05-17T22:07:16.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 -- 172.21.15.83:0/4125495922 learned_addr learned my addr 172.21.15.83:0/4125495922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20227fc640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20240c6540 0x7f20240b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 -- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f200c003fe0 msgr2=0x7f20240b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f200c003fe0 0x7f20240b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 -- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20240c6540 msgr2=0x7f20240b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20240c6540 0x7f20240b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 -- 172.21.15.83:0/4125495922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20240aec90 con 0x7f20240b5e60 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20237fe640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2024160500 tx=0x7f201c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c01c070 con 0x7f20240b5e60 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201c017400 con 0x7f20240b5e60 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0177f0 con 0x7f20240b5e60 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20240aef70 con 0x7f20240b5e60 2024-05-17T22:07:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.094+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20240af530 con 0x7f20240b5e60 2024-05-17T22:07:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240aea00 con 0x7f20240b5e60 2024-05-17T22:07:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f201c01f070 con 0x7f20240b5e60 2024-05-17T22:07:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f200b7fe640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 0x7f2010071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f201c08f2f0 con 0x7f20240b5e60 2024-05-17T22:07:16.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f2022ffd640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 0x7f2010071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.098+0000 7f2022ffd640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 0x7f2010071910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2014000f70 tx=0x7f20140013c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.102+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f201c012070 con 0x7f20240b5e60 2024-05-17T22:07:16.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.294+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20240703e0 con 0x7f20240b5e60 2024-05-17T22:07:16.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.294+0000 7f200b7fe640 1 -- 172.21.15.83:0/4125495922 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5180 (secure 0 0 0) 0x7f201c05bb90 con 0x7f20240b5e60 2024-05-17T22:07:16.289 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:16.289 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7980,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:15.298736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1193425174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1193425174},{"type":"v1","addr":"172.21.15.155:6833","nonce":1193425174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7992,"qdb_cluster":[7992]},"id":8}]} 2024-05-17T22:07:16.290 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 97 2024-05-17T22:07:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 msgr2=0x7f2010071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 0x7f2010071910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2014000f70 tx=0x7f20140013c0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 msgr2=0x7f20240afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2024160500 tx=0x7f201c00f4a0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 shutdown_connections 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20240b5e60 0x7f20240afd30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f201006f400 0x7f2010071910 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20240c6540 0x7f20240b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 --2- 172.21.15.83:0/4125495922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f200c003fe0 0x7f20240b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 >> 172.21.15.83:0/4125495922 conn(0x7f202406cd90 msgr2=0x7f202415e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 shutdown_connections 2024-05-17T22:07:16.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.298+0000 7f20292f8640 1 -- 172.21.15.83:0/4125495922 wait complete. 2024-05-17T22:07:16.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:07:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 Processor -- start 2024-05-17T22:07:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 -- start start 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f3590070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35900b6310 0x7f35900b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359006f6d0 con 0x7f35901607d0 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f359006f9b0 con 0x7f35900b6310 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f35976ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f359006fc90 con 0x7f359015d1b0 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f359548a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f35901607d0 0x7f359006d970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60752/0 (socket says 172.21.15.83:60752) 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f359548a640 1 -- 172.21.15.83:0/3291466089 learned_addr learned my addr 172.21.15.83:0/3291466089 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3291466089 >> v2:172.21.15.83:3300/0 conn(0x7f35900b6310 0x7f35900b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3587fff640 1 -- 172.21.15.83:0/3291466089 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453171747 0 0) 0x7f359006f6d0 con 0x7f35901607d0 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3587fff640 1 -- 172.21.15.83:0/3291466089 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f357c003530 con 0x7f35901607d0 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3595c8b640 1 -- 172.21.15.83:0/3291466089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 msgr2=0x7f3590070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3594c89640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f3590070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f3590070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3595c8b640 1 -- 172.21.15.83:0/3291466089 >> v1:172.21.15.83:6789/0 conn(0x7f35901607d0 legacy=0x7f359006d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.454+0000 7f3595c8b640 1 -- 172.21.15.83:0/3291466089 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359006ff70 con 0x7f35900b6310 2024-05-17T22:07:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35900b6310 0x7f35900b6720 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f358800a960 tx=0x7f358801d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79457bec9c81de26 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f3587fff640 1 -- 172.21.15.83:0/3291466089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b3070 con 0x7f35900b6310 2024-05-17T22:07:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f3587fff640 1 -- 172.21.15.83:0/3291466089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880a6790 con 0x7f35900b6310 2024-05-17T22:07:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f3587fff640 1 -- 172.21.15.83:0/3291466089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358800b040 con 0x7f35900b6310 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- 172.21.15.83:0/3291466089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35900b6310 msgr2=0x7f35900b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35900b6310 0x7f35900b6720 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f358800a960 tx=0x7f358801d870 comp rx=0 tx=0).stop 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- 172.21.15.83:0/3291466089 shutdown_connections 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35900b6310 0x7f35900b6720 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 --2- 172.21.15.83:0/3291466089 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f3590070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- 172.21.15.83:0/3291466089 >> 172.21.15.83:0/3291466089 conn(0x7f3590073670 msgr2=0x7f3590073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- 172.21.15.83:0/3291466089 shutdown_connections 2024-05-17T22:07:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- 172.21.15.83:0/3291466089 wait complete. 2024-05-17T22:07:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 Processor -- start 2024-05-17T22:07:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.458+0000 7f35976ee640 1 -- start start 2024-05-17T22:07:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35900b6310 0x7f359014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f35901646e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35900b4fa0 con 0x7f35901607d0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35900b4e20 con 0x7f35900b6310 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f35900b5120 con 0x7f359015d1b0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41564/0 (socket says 172.21.15.83:41564) 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 -- 172.21.15.83:0/3019480012 learned_addr learned my addr 172.21.15.83:0/3019480012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 -- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 msgr2=0x7f35901646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3594c89640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f35901646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f359548a640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35900b6310 0x7f359014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f35901646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 -- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35900b6310 msgr2=0x7f359014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35900b6310 0x7f359014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 -- 172.21.15.83:0/3019480012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359014dca0 con 0x7f35901607d0 2024-05-17T22:07:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3595c8b640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f358800a960 tx=0x7f358801a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880bc070 con 0x7f35901607d0 2024-05-17T22:07:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880a6940 con 0x7f35901607d0 2024-05-17T22:07:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3590163640 con 0x7f35901607d0 2024-05-17T22:07:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b3040 con 0x7f35901607d0 2024-05-17T22:07:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3590163c00 con 0x7f35901607d0 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35901633b0 con 0x7f35901607d0 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35880bf070 con 0x7f35901607d0 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.462+0000 7f3585ffb640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 0x7f3574071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.466+0000 7f359548a640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 0x7f3574071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.466+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f3588128900 con 0x7f35901607d0 2024-05-17T22:07:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.466+0000 7f359548a640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 0x7f3574071910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f358c0015b0 tx=0x7f358c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.470+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3588017460 con 0x7f35901607d0 2024-05-17T22:07:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.618+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35900b6720 con 0x7f35901607d0 2024-05-17T22:07:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.618+0000 7f3585ffb640 1 -- 172.21.15.83:0/3019480012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16603 (secure 0 0 0) 0x7f35880f57f0 con 0x7f35901607d0 2024-05-17T22:07:16.613 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:16.613 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:13.868335+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:07:09.949308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-17T22:07:11.233706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 msgr2=0x7f3574071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 0x7f3574071910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f358c0015b0 tx=0x7f358c009040 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 msgr2=0x7f35901f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f358800a960 tx=0x7f358801a610 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 shutdown_connections 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35901607d0 0x7f35901f6a70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f359015d1b0 0x7f35901646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f357406f400 0x7f3574071910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 --2- 172.21.15.83:0/3019480012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35900b6310 0x7f359014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 >> 172.21.15.83:0/3019480012 conn(0x7f3590073670 msgr2=0x7f35900b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 shutdown_connections 2024-05-17T22:07:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.622+0000 7f35976ee640 1 -- 172.21.15.83:0/3019480012 wait complete. 2024-05-17T22:07:16.634 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:07:16.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:07:16.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 Processor -- start 2024-05-17T22:07:16.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 -- start start 2024-05-17T22:07:16.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60c40cf2e0 0x7f60c409b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c40d1950 con 0x7f60c4072d70 2024-05-17T22:07:16.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c40d1670 con 0x7f60c409c570 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c95c8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f60c4071280 con 0x7f60c40cf2e0 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f60c4072d70 0x7f60c40ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60756/0 (socket says 172.21.15.83:60756) 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 -- 172.21.15.83:0/2829449107 learned_addr learned my addr 172.21.15.83:0/2829449107 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/2829449107 >> v2:172.21.15.155:3300/0 conn(0x7f60c40cf2e0 0x7f60c409b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 411070866 0 0) 0x7f60c40d1950 con 0x7f60c4072d70 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60ac003530 con 0x7f60c4072d70 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629646531 0 0) 0x7f60ac003530 con 0x7f60c4072d70 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 >> v2:172.21.15.155:3300/0 conn(0x7f60c40cf2e0 msgr2=0x7f60c409b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 --2- 172.21.15.83:0/2829449107 >> v2:172.21.15.155:3300/0 conn(0x7f60c40cf2e0 0x7f60c409b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 >> v1:172.21.15.155:6789/0 conn(0x7f60c409c570 legacy=0x7f60c40ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c4071560 con 0x7f60c4072d70 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f60bc001a80 con 0x7f60c4072d70 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 >> v1:172.21.15.83:6789/0 conn(0x7f60c4072d70 legacy=0x7f60c40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac003a60 0x7f60ac003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60ac004810 0x7f60ac004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 0x7f60ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60c4071280 con 0x7f60ac005260 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac00c1e0 con 0x7f60ac003a60 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac00c3e0 con 0x7f60ac004810 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 0x7f60ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac003a60 0x7f60ac003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 -- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60ac004810 msgr2=0x7f60ac004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60ac004810 0x7f60ac004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 -- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac003a60 msgr2=0x7f60ac003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac003a60 0x7f60ac003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.782+0000 7f60c37fe640 1 -- 172.21.15.83:0/2829449107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac003530 con 0x7f60ac005260 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c37fe640 1 -- 172.21.15.83:0/2829449107 reap_dead start 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c37fe640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 0x7f60ac00b9d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f60c40d1130 tx=0x7f60bc01d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be031464816dea server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00a4e0 con 0x7f60ac005260 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc0ac620 con 0x7f60ac005260 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c1ffb640 1 -- 172.21.15.83:0/2829449107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0aca10 con 0x7f60ac005260 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 -- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 msgr2=0x7f60ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 0x7f60ac00b9d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f60c40d1130 tx=0x7f60bc01d260 comp rx=0 tx=0).stop 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 -- 172.21.15.83:0/2829449107 shutdown_connections 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 --2- 172.21.15.83:0/2829449107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60ac005260 0x7f60ac00b9d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 -- 172.21.15.83:0/2829449107 >> 172.21.15.83:0/2829449107 conn(0x7f60c40b5120 msgr2=0x7f60c40b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 -- 172.21.15.83:0/2829449107 shutdown_connections 2024-05-17T22:07:16.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.786+0000 7f60c95c8640 1 -- 172.21.15.83:0/2829449107 wait complete. 2024-05-17T22:07:16.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 Processor -- start 2024-05-17T22:07:16.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- start start 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 0x7f60c415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60c415fef0 0x7f60c40b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60c40d1380 con 0x7f60c415fae0 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60c40d1200 con 0x7f60ac005260 2024-05-17T22:07:16.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f60c40b8de0 con 0x7f60c415fef0 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41584/0 (socket says 172.21.15.83:41584) 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 -- 172.21.15.83:0/3951255649 learned_addr learned my addr 172.21.15.83:0/3951255649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 0x7f60c415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c37fe640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60c415fef0 0x7f60c40b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 -- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60c415fef0 msgr2=0x7f60c40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60c415fef0 0x7f60c40b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 -- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 msgr2=0x7f60c415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 0x7f60c415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 -- 172.21.15.83:0/3951255649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c41f75f0 con 0x7f60c415fae0 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 0x7f60c415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:16.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c27fc640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c500 tx=0x7f60b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b80052d0 con 0x7f60c415fae0 2024-05-17T22:07:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b8005470 con 0x7f60c415fae0 2024-05-17T22:07:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c40b1100 con 0x7f60c415fae0 2024-05-17T22:07:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8012830 con 0x7f60c415fae0 2024-05-17T22:07:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c406daf0 con 0x7f60c415fae0 2024-05-17T22:07:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c409ce60 con 0x7f60c415fae0 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.790+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60b80129d0 con 0x7f60c415fae0 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.794+0000 7f60ab7fe640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 0x7f60b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.794+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f60b808ef00 con 0x7f60c415fae0 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.794+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 0x7f60b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.794+0000 7f60c2ffd640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 0x7f60b0071910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60b40099a0 tx=0x7f60b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.798+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f60b805be90 con 0x7f60c415fae0 2024-05-17T22:07:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.946+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60c40b0e90 con 0x7f60b006f400 2024-05-17T22:07:16.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.954+0000 7f60ab7fe640 1 -- 172.21.15.83:0/3951255649 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252470 (secure 0 0 0) 0x7f60c40b0e90 con 0x7f60b006f400 2024-05-17T22:07:16.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:16.951 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:07:16.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 msgr2=0x7f60b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 0x7f60b0071910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f60b40099a0 tx=0x7f60b4008040 comp rx=0 tx=0).stop 2024-05-17T22:07:16.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 msgr2=0x7f60c40b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f60b800c500 tx=0x7f60b800c9c0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 shutdown_connections 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60c415fef0 0x7f60c40b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60c415fae0 0x7f60c40b1780 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f60b006f400 0x7f60b0071910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 --2- 172.21.15.83:0/3951255649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60ac005260 0x7f60c415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 >> 172.21.15.83:0/3951255649 conn(0x7f60c40b5120 msgr2=0x7f60c409b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 shutdown_connections 2024-05-17T22:07:16.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:16.958+0000 7f60c95c8640 1 -- 172.21.15.83:0/3951255649 wait complete. 2024-05-17T22:07:16.969 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-05-17T22:07:15.184606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261200,"kb_used_data":11732,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713520,"statfs":{"total":773094113280,"available":772826644480,"internally_reserved":0,"allocated":12013568,"data_stored":5837440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.332748"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907579+0000","last_change":"2024-05-17T22:07:12.898558+0000","last_active":"2024-05-17T22:07:13.907579+0000","last_peered":"2024-05-17T22:07:13.907579+0000","last_clean":"2024-05-17T22:07:13.907579+0000","last_became_active":"2024-05-17T22:07:12.898216+0000","last_became_peered":"2024-05-17T22:07:12.898216+0000","last_unstale":"2024-05-17T22:07:13.907579+0000","last_undegraded":"2024-05-17T22:07:13.907579+0000","last_fullsized":"2024-05-17T22:07:13.907579+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:23.201583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909573+0000","last_change":"2024-05-17T22:07:12.904636+0000","last_active":"2024-05-17T22:07:13.909573+0000","last_peered":"2024-05-17T22:07:13.909573+0000","last_clean":"2024-05-17T22:07:13.909573+0000","last_became_active":"2024-05-17T22:07:12.904281+0000","last_became_peered":"2024-05-17T22:07:12.904281+0000","last_unstale":"2024-05-17T22:07:13.909573+0000","last_undegraded":"2024-05-17T22:07:13.909573+0000","last_fullsized":"2024-05-17T22:07:13.909573+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:19.602360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906404+0000","last_change":"2024-05-17T22:07:12.907162+0000","last_active":"2024-05-17T22:07:13.906404+0000","last_peered":"2024-05-17T22:07:13.906404+0000","last_clean":"2024-05-17T22:07:13.906404+0000","last_became_active":"2024-05-17T22:07:12.906815+0000","last_became_peered":"2024-05-17T22:07:12.906815+0000","last_unstale":"2024-05-17T22:07:13.906404+0000","last_undegraded":"2024-05-17T22:07:13.906404+0000","last_fullsized":"2024-05-17T22:07:13.906404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:48.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907821+0000","last_change":"2024-05-17T22:07:12.907580+0000","last_active":"2024-05-17T22:07:13.907821+0000","last_peered":"2024-05-17T22:07:13.907821+0000","last_clean":"2024-05-17T22:07:13.907821+0000","last_became_active":"2024-05-17T22:07:12.907265+0000","last_became_peered":"2024-05-17T22:07:12.907265+0000","last_unstale":"2024-05-17T22:07:13.907821+0000","last_undegraded":"2024-05-17T22:07:13.907821+0000","last_fullsized":"2024-05-17T22:07:13.907821+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:52.468764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877367+0000","last_change":"2024-05-17T22:07:12.892120+0000","last_active":"2024-05-17T22:07:13.877367+0000","last_peered":"2024-05-17T22:07:13.877367+0000","last_clean":"2024-05-17T22:07:13.877367+0000","last_became_active":"2024-05-17T22:07:12.891463+0000","last_became_peered":"2024-05-17T22:07:12.891463+0000","last_unstale":"2024-05-17T22:07:13.877367+0000","last_undegraded":"2024-05-17T22:07:13.877367+0000","last_fullsized":"2024-05-17T22:07:13.877367+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:25.955094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876477+0000","last_change":"2024-05-17T22:07:12.895722+0000","last_active":"2024-05-17T22:07:13.876477+0000","last_peered":"2024-05-17T22:07:13.876477+0000","last_clean":"2024-05-17T22:07:13.876477+0000","last_became_active":"2024-05-17T22:07:12.895369+0000","last_became_peered":"2024-05-17T22:07:12.895369+0000","last_unstale":"2024-05-17T22:07:13.876477+0000","last_undegraded":"2024-05-17T22:07:13.876477+0000","last_fullsized":"2024-05-17T22:07:13.876477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:58:15.447104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875659+0000","last_change":"2024-05-17T22:07:12.885255+0000","last_active":"2024-05-17T22:07:13.875659+0000","last_peered":"2024-05-17T22:07:13.875659+0000","last_clean":"2024-05-17T22:07:13.875659+0000","last_became_active":"2024-05-17T22:07:12.884812+0000","last_became_peered":"2024-05-17T22:07:12.884812+0000","last_unstale":"2024-05-17T22:07:13.875659+0000","last_undegraded":"2024-05-17T22:07:13.875659+0000","last_fullsized":"2024-05-17T22:07:13.875659+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:14.773485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.905803+0000","last_change":"2024-05-17T22:07:12.903134+0000","last_active":"2024-05-17T22:07:13.905803+0000","last_peered":"2024-05-17T22:07:13.905803+0000","last_clean":"2024-05-17T22:07:13.905803+0000","last_became_active":"2024-05-17T22:07:12.902597+0000","last_became_peered":"2024-05-17T22:07:12.902597+0000","last_unstale":"2024-05-17T22:07:13.905803+0000","last_undegraded":"2024-05-17T22:07:13.905803+0000","last_fullsized":"2024-05-17T22:07:13.905803+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907757+0000","last_change":"2024-05-17T22:07:12.897894+0000","last_active":"2024-05-17T22:07:13.907757+0000","last_peered":"2024-05-17T22:07:13.907757+0000","last_clean":"2024-05-17T22:07:13.907757+0000","last_became_active":"2024-05-17T22:07:12.897591+0000","last_became_peered":"2024-05-17T22:07:12.897591+0000","last_unstale":"2024-05-17T22:07:13.907757+0000","last_undegraded":"2024-05-17T22:07:13.907757+0000","last_fullsized":"2024-05-17T22:07:13.907757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:08.027980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908031+0000","last_change":"2024-05-17T22:07:12.889761+0000","last_active":"2024-05-17T22:07:13.908031+0000","last_peered":"2024-05-17T22:07:13.908031+0000","last_clean":"2024-05-17T22:07:13.908031+0000","last_became_active":"2024-05-17T22:07:12.889317+0000","last_became_peered":"2024-05-17T22:07:12.889317+0000","last_unstale":"2024-05-17T22:07:13.908031+0000","last_undegraded":"2024-05-17T22:07:13.908031+0000","last_fullsized":"2024-05-17T22:07:13.908031+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:02.914711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908963+0000","last_change":"2024-05-17T22:07:12.893157+0000","last_active":"2024-05-17T22:07:13.908963+0000","last_peered":"2024-05-17T22:07:13.908963+0000","last_clean":"2024-05-17T22:07:13.908963+0000","last_became_active":"2024-05-17T22:07:12.892725+0000","last_became_peered":"2024-05-17T22:07:12.892725+0000","last_unstale":"2024-05-17T22:07:13.908963+0000","last_undegraded":"2024-05-17T22:07:13.908963+0000","last_fullsized":"2024-05-17T22:07:13.908963+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:29:16.183947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908243+0000","last_change":"2024-05-17T22:07:12.889722+0000","last_active":"2024-05-17T22:07:13.908243+0000","last_peered":"2024-05-17T22:07:13.908243+0000","last_clean":"2024-05-17T22:07:13.908243+0000","last_became_active":"2024-05-17T22:07:12.889275+0000","last_became_peered":"2024-05-17T22:07:12.889275+0000","last_unstale":"2024-05-17T22:07:13.908243+0000","last_undegraded":"2024-05-17T22:07:13.908243+0000","last_fullsized":"2024-05-17T22:07:13.908243+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:18:32.272231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907001+0000","last_change":"2024-05-17T22:07:12.911084+0000","last_active":"2024-05-17T22:07:13.907001+0000","last_peered":"2024-05-17T22:07:13.907001+0000","last_clean":"2024-05-17T22:07:13.907001+0000","last_became_active":"2024-05-17T22:07:12.910758+0000","last_became_peered":"2024-05-17T22:07:12.910758+0000","last_unstale":"2024-05-17T22:07:13.907001+0000","last_undegraded":"2024-05-17T22:07:13.907001+0000","last_fullsized":"2024-05-17T22:07:13.907001+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:21.451864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906303+0000","last_change":"2024-05-17T22:07:12.903576+0000","last_active":"2024-05-17T22:07:13.906303+0000","last_peered":"2024-05-17T22:07:13.906303+0000","last_clean":"2024-05-17T22:07:13.906303+0000","last_became_active":"2024-05-17T22:07:12.902898+0000","last_became_peered":"2024-05-17T22:07:12.902898+0000","last_unstale":"2024-05-17T22:07:13.906303+0000","last_undegraded":"2024-05-17T22:07:13.906303+0000","last_fullsized":"2024-05-17T22:07:13.906303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:52.037932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876446+0000","last_change":"2024-05-17T22:07:12.895003+0000","last_active":"2024-05-17T22:07:13.876446+0000","last_peered":"2024-05-17T22:07:13.876446+0000","last_clean":"2024-05-17T22:07:13.876446+0000","last_became_active":"2024-05-17T22:07:12.894681+0000","last_became_peered":"2024-05-17T22:07:12.894681+0000","last_unstale":"2024-05-17T22:07:13.876446+0000","last_undegraded":"2024-05-17T22:07:13.876446+0000","last_fullsized":"2024-05-17T22:07:13.876446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:11.887993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906880+0000","last_change":"2024-05-17T22:07:12.903470+0000","last_active":"2024-05-17T22:07:13.906880+0000","last_peered":"2024-05-17T22:07:13.906880+0000","last_clean":"2024-05-17T22:07:13.906880+0000","last_became_active":"2024-05-17T22:07:12.903122+0000","last_became_peered":"2024-05-17T22:07:12.903122+0000","last_unstale":"2024-05-17T22:07:13.906880+0000","last_undegraded":"2024-05-17T22:07:13.906880+0000","last_fullsized":"2024-05-17T22:07:13.906880+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:39:31.135430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909818+0000","last_change":"2024-05-17T22:07:12.893675+0000","last_active":"2024-05-17T22:07:13.909818+0000","last_peered":"2024-05-17T22:07:13.909818+0000","last_clean":"2024-05-17T22:07:13.909818+0000","last_became_active":"2024-05-17T22:07:12.893195+0000","last_became_peered":"2024-05-17T22:07:12.893195+0000","last_unstale":"2024-05-17T22:07:13.909818+0000","last_undegraded":"2024-05-17T22:07:13.909818+0000","last_fullsized":"2024-05-17T22:07:13.909818+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908145+0000","last_change":"2024-05-17T22:07:12.904079+0000","last_active":"2024-05-17T22:07:13.908145+0000","last_peered":"2024-05-17T22:07:13.908145+0000","last_clean":"2024-05-17T22:07:13.908145+0000","last_became_active":"2024-05-17T22:07:12.903788+0000","last_became_peered":"2024-05-17T22:07:12.903788+0000","last_unstale":"2024-05-17T22:07:13.908145+0000","last_undegraded":"2024-05-17T22:07:13.908145+0000","last_fullsized":"2024-05-17T22:07:13.908145+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:27.421743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906525+0000","last_change":"2024-05-17T22:07:12.911244+0000","last_active":"2024-05-17T22:07:13.906525+0000","last_peered":"2024-05-17T22:07:13.906525+0000","last_clean":"2024-05-17T22:07:13.906525+0000","last_became_active":"2024-05-17T22:07:12.910887+0000","last_became_peered":"2024-05-17T22:07:12.910887+0000","last_unstale":"2024-05-17T22:07:13.906525+0000","last_undegraded":"2024-05-17T22:07:13.906525+0000","last_fullsized":"2024-05-17T22:07:13.906525+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:27.984446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909296+0000","last_change":"2024-05-17T22:07:12.909303+0000","last_active":"2024-05-17T22:07:13.909296+0000","last_peered":"2024-05-17T22:07:13.909296+0000","last_clean":"2024-05-17T22:07:13.909296+0000","last_became_active":"2024-05-17T22:07:12.908890+0000","last_became_peered":"2024-05-17T22:07:12.908890+0000","last_unstale":"2024-05-17T22:07:13.909296+0000","last_undegraded":"2024-05-17T22:07:13.909296+0000","last_fullsized":"2024-05-17T22:07:13.909296+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:27.421743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907969+0000","last_change":"2024-05-17T22:07:12.895060+0000","last_active":"2024-05-17T22:07:13.907969+0000","last_peered":"2024-05-17T22:07:13.907969+0000","last_clean":"2024-05-17T22:07:13.907969+0000","last_became_active":"2024-05-17T22:07:12.894762+0000","last_became_peered":"2024-05-17T22:07:12.894762+0000","last_unstale":"2024-05-17T22:07:13.907969+0000","last_undegraded":"2024-05-17T22:07:13.907969+0000","last_fullsized":"2024-05-17T22:07:13.907969+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:29.625033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909313+0000","last_change":"2024-05-17T22:07:12.894526+0000","last_active":"2024-05-17T22:07:13.909313+0000","last_peered":"2024-05-17T22:07:13.909313+0000","last_clean":"2024-05-17T22:07:13.909313+0000","last_became_active":"2024-05-17T22:07:12.894012+0000","last_became_peered":"2024-05-17T22:07:12.894012+0000","last_unstale":"2024-05-17T22:07:13.909313+0000","last_undegraded":"2024-05-17T22:07:13.909313+0000","last_fullsized":"2024-05-17T22:07:13.909313+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:48.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906010+0000","last_change":"2024-05-17T22:07:12.900979+0000","last_active":"2024-05-17T22:07:13.906010+0000","last_peered":"2024-05-17T22:07:13.906010+0000","last_clean":"2024-05-17T22:07:13.906010+0000","last_became_active":"2024-05-17T22:07:12.900422+0000","last_became_peered":"2024-05-17T22:07:12.900422+0000","last_unstale":"2024-05-17T22:07:13.906010+0000","last_undegraded":"2024-05-17T22:07:13.906010+0000","last_fullsized":"2024-05-17T22:07:13.906010+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:44.415742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908987+0000","last_change":"2024-05-17T22:07:12.901065+0000","last_active":"2024-05-17T22:07:13.908987+0000","last_peered":"2024-05-17T22:07:13.908987+0000","last_clean":"2024-05-17T22:07:13.908987+0000","last_became_active":"2024-05-17T22:07:12.900665+0000","last_became_peered":"2024-05-17T22:07:12.900665+0000","last_unstale":"2024-05-17T22:07:13.908987+0000","last_undegraded":"2024-05-17T22:07:13.908987+0000","last_fullsized":"2024-05-17T22:07:13.908987+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:36.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910070+0000","last_change":"2024-05-17T22:07:12.891615+0000","last_active":"2024-05-17T22:07:13.910070+0000","last_peered":"2024-05-17T22:07:13.910070+0000","last_clean":"2024-05-17T22:07:13.910070+0000","last_became_active":"2024-05-17T22:07:12.891155+0000","last_became_peered":"2024-05-17T22:07:12.891155+0000","last_unstale":"2024-05-17T22:07:13.910070+0000","last_undegraded":"2024-05-17T22:07:13.910070+0000","last_fullsized":"2024-05-17T22:07:13.910070+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908444+0000","last_change":"2024-05-17T22:07:12.886931+0000","last_active":"2024-05-17T22:07:13.908444+0000","last_peered":"2024-05-17T22:07:13.908444+0000","last_clean":"2024-05-17T22:07:13.908444+0000","last_became_active":"2024-05-17T22:07:12.886484+0000","last_became_peered":"2024-05-17T22:07:12.886484+0000","last_unstale":"2024-05-17T22:07:13.908444+0000","last_undegraded":"2024-05-17T22:07:13.908444+0000","last_fullsized":"2024-05-17T22:07:13.908444+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:09:32.656030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878340+0000","last_change":"2024-05-17T22:07:12.908746+0000","last_active":"2024-05-17T22:07:13.878340+0000","last_peered":"2024-05-17T22:07:13.878340+0000","last_clean":"2024-05-17T22:07:13.878340+0000","last_became_active":"2024-05-17T22:07:12.907430+0000","last_became_peered":"2024-05-17T22:07:12.907430+0000","last_unstale":"2024-05-17T22:07:13.878340+0000","last_undegraded":"2024-05-17T22:07:13.878340+0000","last_fullsized":"2024-05-17T22:07:13.878340+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:17.601039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909099+0000","last_change":"2024-05-17T22:07:12.907834+0000","last_active":"2024-05-17T22:07:13.909099+0000","last_peered":"2024-05-17T22:07:13.909099+0000","last_clean":"2024-05-17T22:07:13.909099+0000","last_became_active":"2024-05-17T22:07:12.906131+0000","last_became_peered":"2024-05-17T22:07:12.906131+0000","last_unstale":"2024-05-17T22:07:13.909099+0000","last_undegraded":"2024-05-17T22:07:13.909099+0000","last_fullsized":"2024-05-17T22:07:13.909099+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:52.037932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908443+0000","last_change":"2024-05-17T22:07:12.909601+0000","last_active":"2024-05-17T22:07:13.908443+0000","last_peered":"2024-05-17T22:07:13.908443+0000","last_clean":"2024-05-17T22:07:13.908443+0000","last_became_active":"2024-05-17T22:07:12.909177+0000","last_became_peered":"2024-05-17T22:07:12.909177+0000","last_unstale":"2024-05-17T22:07:13.908443+0000","last_undegraded":"2024-05-17T22:07:13.908443+0000","last_fullsized":"2024-05-17T22:07:13.908443+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:23:27.384436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875910+0000","last_change":"2024-05-17T22:07:12.895535+0000","last_active":"2024-05-17T22:07:13.875910+0000","last_peered":"2024-05-17T22:07:13.875910+0000","last_clean":"2024-05-17T22:07:13.875910+0000","last_became_active":"2024-05-17T22:07:12.894923+0000","last_became_peered":"2024-05-17T22:07:12.894923+0000","last_unstale":"2024-05-17T22:07:13.875910+0000","last_undegraded":"2024-05-17T22:07:13.875910+0000","last_fullsized":"2024-05-17T22:07:13.875910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:08.602957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907559+0000","last_change":"2024-05-17T22:07:12.891323+0000","last_active":"2024-05-17T22:07:13.907559+0000","last_peered":"2024-05-17T22:07:13.907559+0000","last_clean":"2024-05-17T22:07:13.907559+0000","last_became_active":"2024-05-17T22:07:12.890975+0000","last_became_peered":"2024-05-17T22:07:12.890975+0000","last_unstale":"2024-05-17T22:07:13.907559+0000","last_undegraded":"2024-05-17T22:07:13.907559+0000","last_fullsized":"2024-05-17T22:07:13.907559+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:25.082875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875815+0000","last_change":"2024-05-17T22:07:12.902405+0000","last_active":"2024-05-17T22:07:13.875815+0000","last_peered":"2024-05-17T22:07:13.875815+0000","last_clean":"2024-05-17T22:07:13.875815+0000","last_became_active":"2024-05-17T22:07:12.894863+0000","last_became_peered":"2024-05-17T22:07:12.894863+0000","last_unstale":"2024-05-17T22:07:13.875815+0000","last_undegraded":"2024-05-17T22:07:13.875815+0000","last_fullsized":"2024-05-17T22:07:13.875815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:58.409599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878517+0000","last_change":"2024-05-17T22:07:12.881675+0000","last_active":"2024-05-17T22:07:13.878517+0000","last_peered":"2024-05-17T22:07:13.878517+0000","last_clean":"2024-05-17T22:07:13.878517+0000","last_became_active":"2024-05-17T22:07:12.881334+0000","last_became_peered":"2024-05-17T22:07:12.881334+0000","last_unstale":"2024-05-17T22:07:13.878517+0000","last_undegraded":"2024-05-17T22:07:13.878517+0000","last_fullsized":"2024-05-17T22:07:13.878517+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:29:43.812078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876447+0000","last_change":"2024-05-17T22:07:11.901401+0000","last_active":"2024-05-17T22:07:13.876447+0000","last_peered":"2024-05-17T22:07:13.876447+0000","last_clean":"2024-05-17T22:07:13.876447+0000","last_became_active":"2024-05-17T22:07:11.899245+0000","last_became_peered":"2024-05-17T22:07:11.899245+0000","last_unstale":"2024-05-17T22:07:13.876447+0000","last_undegraded":"2024-05-17T22:07:13.876447+0000","last_fullsized":"2024-05-17T22:07:13.876447+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:57.404764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908874+0000","last_change":"2024-05-17T22:07:12.897571+0000","last_active":"2024-05-17T22:07:13.908874+0000","last_peered":"2024-05-17T22:07:13.908874+0000","last_clean":"2024-05-17T22:07:13.908874+0000","last_became_active":"2024-05-17T22:07:12.896706+0000","last_became_peered":"2024-05-17T22:07:12.896706+0000","last_unstale":"2024-05-17T22:07:13.908874+0000","last_undegraded":"2024-05-17T22:07:13.908874+0000","last_fullsized":"2024-05-17T22:07:13.908874+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906425+0000","last_change":"2024-05-17T22:07:11.909824+0000","last_active":"2024-05-17T22:07:13.906425+0000","last_peered":"2024-05-17T22:07:13.906425+0000","last_clean":"2024-05-17T22:07:13.906425+0000","last_became_active":"2024-05-17T22:07:11.908945+0000","last_became_peered":"2024-05-17T22:07:11.908945+0000","last_unstale":"2024-05-17T22:07:13.906425+0000","last_undegraded":"2024-05-17T22:07:13.906425+0000","last_fullsized":"2024-05-17T22:07:13.906425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877859+0000","last_change":"2024-05-17T22:07:12.895440+0000","last_active":"2024-05-17T22:07:13.877859+0000","last_peered":"2024-05-17T22:07:13.877859+0000","last_clean":"2024-05-17T22:07:13.877859+0000","last_became_active":"2024-05-17T22:07:12.894968+0000","last_became_peered":"2024-05-17T22:07:12.894968+0000","last_unstale":"2024-05-17T22:07:13.877859+0000","last_undegraded":"2024-05-17T22:07:13.877859+0000","last_fullsized":"2024-05-17T22:07:13.877859+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:11:15.269098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878783+0000","last_change":"2024-05-17T22:07:11.888421+0000","last_active":"2024-05-17T22:07:13.878783+0000","last_peered":"2024-05-17T22:07:13.878783+0000","last_clean":"2024-05-17T22:07:13.878783+0000","last_became_active":"2024-05-17T22:07:11.887998+0000","last_became_peered":"2024-05-17T22:07:11.887998+0000","last_unstale":"2024-05-17T22:07:13.878783+0000","last_undegraded":"2024-05-17T22:07:13.878783+0000","last_fullsized":"2024-05-17T22:07:13.878783+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:13:35.636709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906603+0000","last_change":"2024-05-17T22:07:12.911263+0000","last_active":"2024-05-17T22:07:13.906603+0000","last_peered":"2024-05-17T22:07:13.906603+0000","last_clean":"2024-05-17T22:07:13.906603+0000","last_became_active":"2024-05-17T22:07:12.910950+0000","last_became_peered":"2024-05-17T22:07:12.910950+0000","last_unstale":"2024-05-17T22:07:13.906603+0000","last_undegraded":"2024-05-17T22:07:13.906603+0000","last_fullsized":"2024-05-17T22:07:13.906603+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:34.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876800+0000","last_change":"2024-05-17T22:07:11.884897+0000","last_active":"2024-05-17T22:07:13.876800+0000","last_peered":"2024-05-17T22:07:13.876800+0000","last_clean":"2024-05-17T22:07:13.876800+0000","last_became_active":"2024-05-17T22:07:11.884426+0000","last_became_peered":"2024-05-17T22:07:11.884426+0000","last_unstale":"2024-05-17T22:07:13.876800+0000","last_undegraded":"2024-05-17T22:07:13.876800+0000","last_fullsized":"2024-05-17T22:07:13.876800+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:13:59.812540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908895+0000","last_change":"2024-05-17T22:07:12.898533+0000","last_active":"2024-05-17T22:07:13.908895+0000","last_peered":"2024-05-17T22:07:13.908895+0000","last_clean":"2024-05-17T22:07:13.908895+0000","last_became_active":"2024-05-17T22:07:12.897257+0000","last_became_peered":"2024-05-17T22:07:12.897257+0000","last_unstale":"2024-05-17T22:07:13.908895+0000","last_undegraded":"2024-05-17T22:07:13.908895+0000","last_fullsized":"2024-05-17T22:07:13.908895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:48.406012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875766+0000","last_change":"2024-05-17T22:07:11.895917+0000","last_active":"2024-05-17T22:07:13.875766+0000","last_peered":"2024-05-17T22:07:13.875766+0000","last_clean":"2024-05-17T22:07:13.875766+0000","last_became_active":"2024-05-17T22:07:11.894741+0000","last_became_peered":"2024-05-17T22:07:11.894741+0000","last_unstale":"2024-05-17T22:07:13.875766+0000","last_undegraded":"2024-05-17T22:07:13.875766+0000","last_fullsized":"2024-05-17T22:07:13.875766+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:37:53.814524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908986+0000","last_change":"2024-05-17T22:07:12.904414+0000","last_active":"2024-05-17T22:07:13.908986+0000","last_peered":"2024-05-17T22:07:13.908986+0000","last_clean":"2024-05-17T22:07:13.908986+0000","last_became_active":"2024-05-17T22:07:12.904094+0000","last_became_peered":"2024-05-17T22:07:12.904094+0000","last_unstale":"2024-05-17T22:07:13.908986+0000","last_undegraded":"2024-05-17T22:07:13.908986+0000","last_fullsized":"2024-05-17T22:07:13.908986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:37.595590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909518+0000","last_change":"2024-05-17T22:07:11.912304+0000","last_active":"2024-05-17T22:07:13.909518+0000","last_peered":"2024-05-17T22:07:13.909518+0000","last_clean":"2024-05-17T22:07:13.909518+0000","last_became_active":"2024-05-17T22:07:11.911850+0000","last_became_peered":"2024-05-17T22:07:11.911850+0000","last_unstale":"2024-05-17T22:07:13.909518+0000","last_undegraded":"2024-05-17T22:07:13.909518+0000","last_fullsized":"2024-05-17T22:07:13.909518+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:33:31.010502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906899+0000","last_change":"2024-05-17T22:07:12.911736+0000","last_active":"2024-05-17T22:07:13.906899+0000","last_peered":"2024-05-17T22:07:13.906899+0000","last_clean":"2024-05-17T22:07:13.906899+0000","last_became_active":"2024-05-17T22:07:12.911255+0000","last_became_peered":"2024-05-17T22:07:12.911255+0000","last_unstale":"2024-05-17T22:07:13.906899+0000","last_undegraded":"2024-05-17T22:07:13.906899+0000","last_fullsized":"2024-05-17T22:07:13.906899+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:52:55.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.905811+0000","last_change":"2024-05-17T22:07:11.893568+0000","last_active":"2024-05-17T22:07:13.905811+0000","last_peered":"2024-05-17T22:07:13.905811+0000","last_clean":"2024-05-17T22:07:13.905811+0000","last_became_active":"2024-05-17T22:07:11.892608+0000","last_became_peered":"2024-05-17T22:07:11.892608+0000","last_unstale":"2024-05-17T22:07:13.905811+0000","last_undegraded":"2024-05-17T22:07:13.905811+0000","last_fullsized":"2024-05-17T22:07:13.905811+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:57:57.758811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875681+0000","last_change":"2024-05-17T22:07:12.887053+0000","last_active":"2024-05-17T22:07:13.875681+0000","last_peered":"2024-05-17T22:07:13.875681+0000","last_clean":"2024-05-17T22:07:13.875681+0000","last_became_active":"2024-05-17T22:07:12.886458+0000","last_became_peered":"2024-05-17T22:07:12.886458+0000","last_unstale":"2024-05-17T22:07:13.875681+0000","last_undegraded":"2024-05-17T22:07:13.875681+0000","last_fullsized":"2024-05-17T22:07:13.875681+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:24.652042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907430+0000","last_change":"2024-05-17T22:07:11.894673+0000","last_active":"2024-05-17T22:07:13.907430+0000","last_peered":"2024-05-17T22:07:13.907430+0000","last_clean":"2024-05-17T22:07:13.907430+0000","last_became_active":"2024-05-17T22:07:11.891142+0000","last_became_peered":"2024-05-17T22:07:11.891142+0000","last_unstale":"2024-05-17T22:07:13.907430+0000","last_undegraded":"2024-05-17T22:07:13.907430+0000","last_fullsized":"2024-05-17T22:07:13.907430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:26.979611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876819+0000","last_change":"2024-05-17T22:07:12.878188+0000","last_active":"2024-05-17T22:07:13.876819+0000","last_peered":"2024-05-17T22:07:13.876819+0000","last_clean":"2024-05-17T22:07:13.876819+0000","last_became_active":"2024-05-17T22:07:12.877735+0000","last_became_peered":"2024-05-17T22:07:12.877735+0000","last_unstale":"2024-05-17T22:07:13.876819+0000","last_undegraded":"2024-05-17T22:07:13.876819+0000","last_fullsized":"2024-05-17T22:07:13.876819+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:54.351637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878954+0000","last_change":"2024-05-17T22:07:11.895060+0000","last_active":"2024-05-17T22:07:13.878954+0000","last_peered":"2024-05-17T22:07:13.878954+0000","last_clean":"2024-05-17T22:07:13.878954+0000","last_became_active":"2024-05-17T22:07:11.894710+0000","last_became_peered":"2024-05-17T22:07:11.894710+0000","last_unstale":"2024-05-17T22:07:13.878954+0000","last_undegraded":"2024-05-17T22:07:13.878954+0000","last_fullsized":"2024-05-17T22:07:13.878954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:22:25.602641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908686+0000","last_change":"2024-05-17T22:07:12.898814+0000","last_active":"2024-05-17T22:07:13.908686+0000","last_peered":"2024-05-17T22:07:13.908686+0000","last_clean":"2024-05-17T22:07:13.908686+0000","last_became_active":"2024-05-17T22:07:12.898460+0000","last_became_peered":"2024-05-17T22:07:12.898460+0000","last_unstale":"2024-05-17T22:07:13.908686+0000","last_undegraded":"2024-05-17T22:07:13.908686+0000","last_fullsized":"2024-05-17T22:07:13.908686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909349+0000","last_change":"2024-05-17T22:07:11.912716+0000","last_active":"2024-05-17T22:07:13.909349+0000","last_peered":"2024-05-17T22:07:13.909349+0000","last_clean":"2024-05-17T22:07:13.909349+0000","last_became_active":"2024-05-17T22:07:11.912339+0000","last_became_peered":"2024-05-17T22:07:11.912339+0000","last_unstale":"2024-05-17T22:07:13.909349+0000","last_undegraded":"2024-05-17T22:07:13.909349+0000","last_fullsized":"2024-05-17T22:07:13.909349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909589+0000","last_change":"2024-05-17T22:07:12.906225+0000","last_active":"2024-05-17T22:07:13.909589+0000","last_peered":"2024-05-17T22:07:13.909589+0000","last_clean":"2024-05-17T22:07:13.909589+0000","last_became_active":"2024-05-17T22:07:12.905908+0000","last_became_peered":"2024-05-17T22:07:12.905908+0000","last_unstale":"2024-05-17T22:07:13.909589+0000","last_undegraded":"2024-05-17T22:07:13.909589+0000","last_fullsized":"2024-05-17T22:07:13.909589+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:34.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910130+0000","last_change":"2024-05-17T22:07:11.895488+0000","last_active":"2024-05-17T22:07:13.910130+0000","last_peered":"2024-05-17T22:07:13.910130+0000","last_clean":"2024-05-17T22:07:13.910130+0000","last_became_active":"2024-05-17T22:07:11.892284+0000","last_became_peered":"2024-05-17T22:07:11.892284+0000","last_unstale":"2024-05-17T22:07:13.910130+0000","last_undegraded":"2024-05-17T22:07:13.910130+0000","last_fullsized":"2024-05-17T22:07:13.910130+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876323+0000","last_change":"2024-05-17T22:07:12.905530+0000","last_active":"2024-05-17T22:07:13.876323+0000","last_peered":"2024-05-17T22:07:13.876323+0000","last_clean":"2024-05-17T22:07:13.876323+0000","last_became_active":"2024-05-17T22:07:12.905205+0000","last_became_peered":"2024-05-17T22:07:12.905205+0000","last_unstale":"2024-05-17T22:07:13.876323+0000","last_undegraded":"2024-05-17T22:07:13.876323+0000","last_fullsized":"2024-05-17T22:07:13.876323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:07:03.740195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908993+0000","last_change":"2024-05-17T22:07:11.895716+0000","last_active":"2024-05-17T22:07:13.908993+0000","last_peered":"2024-05-17T22:07:13.908993+0000","last_clean":"2024-05-17T22:07:13.908993+0000","last_became_active":"2024-05-17T22:07:11.895022+0000","last_became_peered":"2024-05-17T22:07:11.895022+0000","last_unstale":"2024-05-17T22:07:13.908993+0000","last_undegraded":"2024-05-17T22:07:13.908993+0000","last_fullsized":"2024-05-17T22:07:13.908993+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:35.090102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909635+0000","last_change":"2024-05-17T22:07:12.890441+0000","last_active":"2024-05-17T22:07:13.909635+0000","last_peered":"2024-05-17T22:07:13.909635+0000","last_clean":"2024-05-17T22:07:13.909635+0000","last_became_active":"2024-05-17T22:07:12.888492+0000","last_became_peered":"2024-05-17T22:07:12.888492+0000","last_unstale":"2024-05-17T22:07:13.909635+0000","last_undegraded":"2024-05-17T22:07:13.909635+0000","last_fullsized":"2024-05-17T22:07:13.909635+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:10:08.731202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877042+0000","last_change":"2024-05-17T22:07:11.898845+0000","last_active":"2024-05-17T22:07:13.877042+0000","last_peered":"2024-05-17T22:07:13.877042+0000","last_clean":"2024-05-17T22:07:13.877042+0000","last_became_active":"2024-05-17T22:07:11.898467+0000","last_became_peered":"2024-05-17T22:07:11.898467+0000","last_unstale":"2024-05-17T22:07:13.877042+0000","last_undegraded":"2024-05-17T22:07:13.877042+0000","last_fullsized":"2024-05-17T22:07:13.877042+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:16:03.597683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908401+0000","last_change":"2024-05-17T22:07:12.900927+0000","last_active":"2024-05-17T22:07:13.908401+0000","last_peered":"2024-05-17T22:07:13.908401+0000","last_clean":"2024-05-17T22:07:13.908401+0000","last_became_active":"2024-05-17T22:07:12.900601+0000","last_became_peered":"2024-05-17T22:07:12.900601+0000","last_unstale":"2024-05-17T22:07:13.908401+0000","last_undegraded":"2024-05-17T22:07:13.908401+0000","last_fullsized":"2024-05-17T22:07:13.908401+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908023+0000","last_change":"2024-05-17T22:07:11.906951+0000","last_active":"2024-05-17T22:07:13.908023+0000","last_peered":"2024-05-17T22:07:13.908023+0000","last_clean":"2024-05-17T22:07:13.908023+0000","last_became_active":"2024-05-17T22:07:11.906186+0000","last_became_peered":"2024-05-17T22:07:11.906186+0000","last_unstale":"2024-05-17T22:07:13.908023+0000","last_undegraded":"2024-05-17T22:07:13.908023+0000","last_fullsized":"2024-05-17T22:07:13.908023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908515+0000","last_change":"2024-05-17T22:07:12.908531+0000","last_active":"2024-05-17T22:07:13.908515+0000","last_peered":"2024-05-17T22:07:13.908515+0000","last_clean":"2024-05-17T22:07:13.908515+0000","last_became_active":"2024-05-17T22:07:12.906869+0000","last_became_peered":"2024-05-17T22:07:12.906869+0000","last_unstale":"2024-05-17T22:07:13.908515+0000","last_undegraded":"2024-05-17T22:07:13.908515+0000","last_fullsized":"2024-05-17T22:07:13.908515+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875570+0000","last_change":"2024-05-17T22:07:11.885064+0000","last_active":"2024-05-17T22:07:13.875570+0000","last_peered":"2024-05-17T22:07:13.875570+0000","last_clean":"2024-05-17T22:07:13.875570+0000","last_became_active":"2024-05-17T22:07:11.884646+0000","last_became_peered":"2024-05-17T22:07:11.884646+0000","last_unstale":"2024-05-17T22:07:13.875570+0000","last_undegraded":"2024-05-17T22:07:13.875570+0000","last_fullsized":"2024-05-17T22:07:13.875570+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:15:08.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.879327+0000","last_change":"2024-05-17T22:07:12.907579+0000","last_active":"2024-05-17T22:07:13.879327+0000","last_peered":"2024-05-17T22:07:13.879327+0000","last_clean":"2024-05-17T22:07:13.879327+0000","last_became_active":"2024-05-17T22:07:12.906764+0000","last_became_peered":"2024-05-17T22:07:12.906764+0000","last_unstale":"2024-05-17T22:07:13.879327+0000","last_undegraded":"2024-05-17T22:07:13.879327+0000","last_fullsized":"2024-05-17T22:07:13.879327+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:17.525330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908086+0000","last_change":"2024-05-17T22:07:11.890717+0000","last_active":"2024-05-17T22:07:13.908086+0000","last_peered":"2024-05-17T22:07:13.908086+0000","last_clean":"2024-05-17T22:07:13.908086+0000","last_became_active":"2024-05-17T22:07:11.888502+0000","last_became_peered":"2024-05-17T22:07:11.888502+0000","last_unstale":"2024-05-17T22:07:13.908086+0000","last_undegraded":"2024-05-17T22:07:13.908086+0000","last_fullsized":"2024-05-17T22:07:13.908086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:20.447029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878479+0000","last_change":"2024-05-17T22:07:12.907027+0000","last_active":"2024-05-17T22:07:13.878479+0000","last_peered":"2024-05-17T22:07:13.878479+0000","last_clean":"2024-05-17T22:07:13.878479+0000","last_became_active":"2024-05-17T22:07:12.906731+0000","last_became_peered":"2024-05-17T22:07:12.906731+0000","last_unstale":"2024-05-17T22:07:13.878479+0000","last_undegraded":"2024-05-17T22:07:13.878479+0000","last_fullsized":"2024-05-17T22:07:13.878479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:28:09.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.879208+0000","last_change":"2024-05-17T22:07:11.896543+0000","last_active":"2024-05-17T22:07:13.879208+0000","last_peered":"2024-05-17T22:07:13.879208+0000","last_clean":"2024-05-17T22:07:13.879208+0000","last_became_active":"2024-05-17T22:07:11.896046+0000","last_became_peered":"2024-05-17T22:07:11.896046+0000","last_unstale":"2024-05-17T22:07:13.879208+0000","last_undegraded":"2024-05-17T22:07:13.879208+0000","last_fullsized":"2024-05-17T22:07:13.879208+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:21:40.545425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906149+0000","last_change":"2024-05-17T22:07:12.912550+0000","last_active":"2024-05-17T22:07:13.906149+0000","last_peered":"2024-05-17T22:07:13.906149+0000","last_clean":"2024-05-17T22:07:13.906149+0000","last_became_active":"2024-05-17T22:07:12.912293+0000","last_became_peered":"2024-05-17T22:07:12.912293+0000","last_unstale":"2024-05-17T22:07:13.906149+0000","last_undegraded":"2024-05-17T22:07:13.906149+0000","last_fullsized":"2024-05-17T22:07:13.906149+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908366+0000","last_change":"2024-05-17T22:07:11.906024+0000","last_active":"2024-05-17T22:07:13.908366+0000","last_peered":"2024-05-17T22:07:13.908366+0000","last_clean":"2024-05-17T22:07:13.908366+0000","last_became_active":"2024-05-17T22:07:11.905294+0000","last_became_peered":"2024-05-17T22:07:11.905294+0000","last_unstale":"2024-05-17T22:07:13.908366+0000","last_undegraded":"2024-05-17T22:07:13.908366+0000","last_fullsized":"2024-05-17T22:07:13.908366+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:33:31.010502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906653+0000","last_change":"2024-05-17T22:07:12.908313+0000","last_active":"2024-05-17T22:07:13.906653+0000","last_peered":"2024-05-17T22:07:13.906653+0000","last_clean":"2024-05-17T22:07:13.906653+0000","last_became_active":"2024-05-17T22:07:12.907905+0000","last_became_peered":"2024-05-17T22:07:12.907905+0000","last_unstale":"2024-05-17T22:07:13.906653+0000","last_undegraded":"2024-05-17T22:07:13.906653+0000","last_fullsized":"2024-05-17T22:07:13.906653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:06.484085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876102+0000","last_change":"2024-05-17T22:07:11.901167+0000","last_active":"2024-05-17T22:07:13.876102+0000","last_peered":"2024-05-17T22:07:13.876102+0000","last_clean":"2024-05-17T22:07:13.876102+0000","last_became_active":"2024-05-17T22:07:11.900281+0000","last_became_peered":"2024-05-17T22:07:11.900281+0000","last_unstale":"2024-05-17T22:07:13.876102+0000","last_undegraded":"2024-05-17T22:07:13.876102+0000","last_fullsized":"2024-05-17T22:07:13.876102+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:54:08.455507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906658+0000","last_change":"2024-05-17T22:07:12.908276+0000","last_active":"2024-05-17T22:07:13.906658+0000","last_peered":"2024-05-17T22:07:13.906658+0000","last_clean":"2024-05-17T22:07:13.906658+0000","last_became_active":"2024-05-17T22:07:12.907961+0000","last_became_peered":"2024-05-17T22:07:12.907961+0000","last_unstale":"2024-05-17T22:07:13.906658+0000","last_undegraded":"2024-05-17T22:07:13.906658+0000","last_fullsized":"2024-05-17T22:07:13.906658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:10:08.731202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877137+0000","last_change":"2024-05-17T22:07:11.887035+0000","last_active":"2024-05-17T22:07:13.877137+0000","last_peered":"2024-05-17T22:07:13.877137+0000","last_clean":"2024-05-17T22:07:13.877137+0000","last_became_active":"2024-05-17T22:07:11.886710+0000","last_became_peered":"2024-05-17T22:07:11.886710+0000","last_unstale":"2024-05-17T22:07:13.877137+0000","last_undegraded":"2024-05-17T22:07:13.877137+0000","last_fullsized":"2024-05-17T22:07:13.877137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:36:28.613813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907417+0000","last_change":"2024-05-17T22:07:12.903589+0000","last_active":"2024-05-17T22:07:13.907417+0000","last_peered":"2024-05-17T22:07:13.907417+0000","last_clean":"2024-05-17T22:07:13.907417+0000","last_became_active":"2024-05-17T22:07:12.903274+0000","last_became_peered":"2024-05-17T22:07:12.903274+0000","last_unstale":"2024-05-17T22:07:13.907417+0000","last_undegraded":"2024-05-17T22:07:13.907417+0000","last_fullsized":"2024-05-17T22:07:13.907417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:24:46.706528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875712+0000","last_change":"2024-05-17T22:07:11.892483+0000","last_active":"2024-05-17T22:07:13.875712+0000","last_peered":"2024-05-17T22:07:13.875712+0000","last_clean":"2024-05-17T22:07:13.875712+0000","last_became_active":"2024-05-17T22:07:11.892041+0000","last_became_peered":"2024-05-17T22:07:11.892041+0000","last_unstale":"2024-05-17T22:07:13.875712+0000","last_undegraded":"2024-05-17T22:07:13.875712+0000","last_fullsized":"2024-05-17T22:07:13.875712+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:39.388810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906221+0000","last_change":"2024-05-17T22:07:12.894904+0000","last_active":"2024-05-17T22:07:13.906221+0000","last_peered":"2024-05-17T22:07:13.906221+0000","last_clean":"2024-05-17T22:07:13.906221+0000","last_became_active":"2024-05-17T22:07:12.894272+0000","last_became_peered":"2024-05-17T22:07:12.894272+0000","last_unstale":"2024-05-17T22:07:13.906221+0000","last_undegraded":"2024-05-17T22:07:13.906221+0000","last_fullsized":"2024-05-17T22:07:13.906221+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:51.563793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876417+0000","last_change":"2024-05-17T22:07:11.888927+0000","last_active":"2024-05-17T22:07:13.876417+0000","last_peered":"2024-05-17T22:07:13.876417+0000","last_clean":"2024-05-17T22:07:13.876417+0000","last_became_active":"2024-05-17T22:07:11.888121+0000","last_became_peered":"2024-05-17T22:07:11.888121+0000","last_unstale":"2024-05-17T22:07:13.876417+0000","last_undegraded":"2024-05-17T22:07:13.876417+0000","last_fullsized":"2024-05-17T22:07:13.876417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:25.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908790+0000","last_change":"2024-05-17T22:07:12.899405+0000","last_active":"2024-05-17T22:07:13.908790+0000","last_peered":"2024-05-17T22:07:13.908790+0000","last_clean":"2024-05-17T22:07:13.908790+0000","last_became_active":"2024-05-17T22:07:12.898646+0000","last_became_peered":"2024-05-17T22:07:12.898646+0000","last_unstale":"2024-05-17T22:07:13.908790+0000","last_undegraded":"2024-05-17T22:07:13.908790+0000","last_fullsized":"2024-05-17T22:07:13.908790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:51.563793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907719+0000","last_change":"2024-05-17T22:07:11.912218+0000","last_active":"2024-05-17T22:07:13.907719+0000","last_peered":"2024-05-17T22:07:13.907719+0000","last_clean":"2024-05-17T22:07:13.907719+0000","last_became_active":"2024-05-17T22:07:11.911723+0000","last_became_peered":"2024-05-17T22:07:11.911723+0000","last_unstale":"2024-05-17T22:07:13.907719+0000","last_undegraded":"2024-05-17T22:07:13.907719+0000","last_fullsized":"2024-05-17T22:07:13.907719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:33.534048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908293+0000","last_change":"2024-05-17T22:07:12.898139+0000","last_active":"2024-05-17T22:07:13.908293+0000","last_peered":"2024-05-17T22:07:13.908293+0000","last_clean":"2024-05-17T22:07:13.908293+0000","last_became_active":"2024-05-17T22:07:12.897778+0000","last_became_peered":"2024-05-17T22:07:12.897778+0000","last_unstale":"2024-05-17T22:07:13.908293+0000","last_undegraded":"2024-05-17T22:07:13.908293+0000","last_fullsized":"2024-05-17T22:07:13.908293+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:26:41.192008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910365+0000","last_change":"2024-05-17T22:07:11.892682+0000","last_active":"2024-05-17T22:07:13.910365+0000","last_peered":"2024-05-17T22:07:13.910365+0000","last_clean":"2024-05-17T22:07:13.910365+0000","last_became_active":"2024-05-17T22:07:11.892241+0000","last_became_peered":"2024-05-17T22:07:11.892241+0000","last_unstale":"2024-05-17T22:07:13.910365+0000","last_undegraded":"2024-05-17T22:07:13.910365+0000","last_fullsized":"2024-05-17T22:07:13.910365+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:43.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910357+0000","last_change":"2024-05-17T22:07:12.909755+0000","last_active":"2024-05-17T22:07:13.910357+0000","last_peered":"2024-05-17T22:07:13.910357+0000","last_clean":"2024-05-17T22:07:13.910357+0000","last_became_active":"2024-05-17T22:07:12.909420+0000","last_became_peered":"2024-05-17T22:07:12.909420+0000","last_unstale":"2024-05-17T22:07:13.910357+0000","last_undegraded":"2024-05-17T22:07:13.910357+0000","last_fullsized":"2024-05-17T22:07:13.910357+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:55.344248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908451+0000","last_change":"2024-05-17T22:07:11.906359+0000","last_active":"2024-05-17T22:07:13.908451+0000","last_peered":"2024-05-17T22:07:13.908451+0000","last_clean":"2024-05-17T22:07:13.908451+0000","last_became_active":"2024-05-17T22:07:11.905189+0000","last_became_peered":"2024-05-17T22:07:11.905189+0000","last_unstale":"2024-05-17T22:07:13.908451+0000","last_undegraded":"2024-05-17T22:07:13.908451+0000","last_fullsized":"2024-05-17T22:07:13.908451+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:59:42.372644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908680+0000","last_change":"2024-05-17T22:07:12.890471+0000","last_active":"2024-05-17T22:07:13.908680+0000","last_peered":"2024-05-17T22:07:13.908680+0000","last_clean":"2024-05-17T22:07:13.908680+0000","last_became_active":"2024-05-17T22:07:12.890108+0000","last_became_peered":"2024-05-17T22:07:12.890108+0000","last_unstale":"2024-05-17T22:07:13.908680+0000","last_undegraded":"2024-05-17T22:07:13.908680+0000","last_fullsized":"2024-05-17T22:07:13.908680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:39.380090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910156+0000","last_change":"2024-05-17T22:07:11.912226+0000","last_active":"2024-05-17T22:07:13.910156+0000","last_peered":"2024-05-17T22:07:13.910156+0000","last_clean":"2024-05-17T22:07:13.910156+0000","last_became_active":"2024-05-17T22:07:11.911752+0000","last_became_peered":"2024-05-17T22:07:11.911752+0000","last_unstale":"2024-05-17T22:07:13.910156+0000","last_undegraded":"2024-05-17T22:07:13.910156+0000","last_fullsized":"2024-05-17T22:07:13.910156+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:47.311230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908345+0000","last_change":"2024-05-17T22:07:12.898535+0000","last_active":"2024-05-17T22:07:13.908345+0000","last_peered":"2024-05-17T22:07:13.908345+0000","last_clean":"2024-05-17T22:07:13.908345+0000","last_became_active":"2024-05-17T22:07:12.898256+0000","last_became_peered":"2024-05-17T22:07:12.898256+0000","last_unstale":"2024-05-17T22:07:13.908345+0000","last_undegraded":"2024-05-17T22:07:13.908345+0000","last_fullsized":"2024-05-17T22:07:13.908345+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:19.602360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909867+0000","last_change":"2024-05-17T22:07:11.893788+0000","last_active":"2024-05-17T22:07:13.909867+0000","last_peered":"2024-05-17T22:07:13.909867+0000","last_clean":"2024-05-17T22:07:13.909867+0000","last_became_active":"2024-05-17T22:07:11.893319+0000","last_became_peered":"2024-05-17T22:07:11.893319+0000","last_unstale":"2024-05-17T22:07:13.909867+0000","last_undegraded":"2024-05-17T22:07:13.909867+0000","last_fullsized":"2024-05-17T22:07:13.909867+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:59:42.372644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906871+0000","last_change":"2024-05-17T22:07:12.881778+0000","last_active":"2024-05-17T22:07:13.906871+0000","last_peered":"2024-05-17T22:07:13.906871+0000","last_clean":"2024-05-17T22:07:13.906871+0000","last_became_active":"2024-05-17T22:07:12.881458+0000","last_became_peered":"2024-05-17T22:07:12.881458+0000","last_unstale":"2024-05-17T22:07:13.906871+0000","last_undegraded":"2024-05-17T22:07:13.906871+0000","last_fullsized":"2024-05-17T22:07:13.906871+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:21:06.682155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875863+0000","last_change":"2024-05-17T22:07:11.905950+0000","last_active":"2024-05-17T22:07:13.875863+0000","last_peered":"2024-05-17T22:07:13.875863+0000","last_clean":"2024-05-17T22:07:13.875863+0000","last_became_active":"2024-05-17T22:07:11.905082+0000","last_became_peered":"2024-05-17T22:07:11.905082+0000","last_unstale":"2024-05-17T22:07:13.875863+0000","last_undegraded":"2024-05-17T22:07:13.875863+0000","last_fullsized":"2024-05-17T22:07:13.875863+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:21:41.759245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909427+0000","last_change":"2024-05-17T22:07:12.892224+0000","last_active":"2024-05-17T22:07:13.909427+0000","last_peered":"2024-05-17T22:07:13.909427+0000","last_clean":"2024-05-17T22:07:13.909427+0000","last_became_active":"2024-05-17T22:07:12.891873+0000","last_became_peered":"2024-05-17T22:07:12.891873+0000","last_unstale":"2024-05-17T22:07:13.909427+0000","last_undegraded":"2024-05-17T22:07:13.909427+0000","last_fullsized":"2024-05-17T22:07:13.909427+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:26:41.192008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907225+0000","last_change":"2024-05-17T22:07:11.878579+0000","last_active":"2024-05-17T22:07:13.907225+0000","last_peered":"2024-05-17T22:07:13.907225+0000","last_clean":"2024-05-17T22:07:13.907225+0000","last_became_active":"2024-05-17T22:07:11.877736+0000","last_became_peered":"2024-05-17T22:07:11.877736+0000","last_unstale":"2024-05-17T22:07:13.907225+0000","last_undegraded":"2024-05-17T22:07:13.907225+0000","last_fullsized":"2024-05-17T22:07:13.907225+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:49.345469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906261+0000","last_change":"2024-05-17T22:07:12.894974+0000","last_active":"2024-05-17T22:07:13.906261+0000","last_peered":"2024-05-17T22:07:13.906261+0000","last_clean":"2024-05-17T22:07:13.906261+0000","last_became_active":"2024-05-17T22:07:12.894632+0000","last_became_peered":"2024-05-17T22:07:12.894632+0000","last_unstale":"2024-05-17T22:07:13.906261+0000","last_undegraded":"2024-05-17T22:07:13.906261+0000","last_fullsized":"2024-05-17T22:07:13.906261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:36.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876057+0000","last_change":"2024-05-17T22:07:11.895327+0000","last_active":"2024-05-17T22:07:13.876057+0000","last_peered":"2024-05-17T22:07:13.876057+0000","last_clean":"2024-05-17T22:07:13.876057+0000","last_became_active":"2024-05-17T22:07:11.894888+0000","last_became_peered":"2024-05-17T22:07:11.894888+0000","last_unstale":"2024-05-17T22:07:13.876057+0000","last_undegraded":"2024-05-17T22:07:13.876057+0000","last_fullsized":"2024-05-17T22:07:13.876057+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:34.508676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907247+0000","last_change":"2024-05-17T22:07:12.890579+0000","last_active":"2024-05-17T22:07:13.907247+0000","last_peered":"2024-05-17T22:07:13.907247+0000","last_clean":"2024-05-17T22:07:13.907247+0000","last_became_active":"2024-05-17T22:07:12.890190+0000","last_became_peered":"2024-05-17T22:07:12.890190+0000","last_unstale":"2024-05-17T22:07:13.907247+0000","last_undegraded":"2024-05-17T22:07:13.907247+0000","last_fullsized":"2024-05-17T22:07:13.907247+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:48:15.449888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907986+0000","last_change":"2024-05-17T22:07:11.907918+0000","last_active":"2024-05-17T22:07:13.907986+0000","last_peered":"2024-05-17T22:07:13.907986+0000","last_clean":"2024-05-17T22:07:13.907986+0000","last_became_active":"2024-05-17T22:07:11.907362+0000","last_became_peered":"2024-05-17T22:07:11.907362+0000","last_unstale":"2024-05-17T22:07:13.907986+0000","last_undegraded":"2024-05-17T22:07:13.907986+0000","last_fullsized":"2024-05-17T22:07:13.907986+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:05.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909503+0000","last_change":"2024-05-17T22:07:12.902200+0000","last_active":"2024-05-17T22:07:13.909503+0000","last_peered":"2024-05-17T22:07:13.909503+0000","last_clean":"2024-05-17T22:07:13.909503+0000","last_became_active":"2024-05-17T22:07:12.901081+0000","last_became_peered":"2024-05-17T22:07:12.901081+0000","last_unstale":"2024-05-17T22:07:13.909503+0000","last_undegraded":"2024-05-17T22:07:13.909503+0000","last_fullsized":"2024-05-17T22:07:13.909503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:27.984446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875849+0000","last_change":"2024-05-17T22:07:11.889062+0000","last_active":"2024-05-17T22:07:13.875849+0000","last_peered":"2024-05-17T22:07:13.875849+0000","last_clean":"2024-05-17T22:07:13.875849+0000","last_became_active":"2024-05-17T22:07:11.888152+0000","last_became_peered":"2024-05-17T22:07:11.888152+0000","last_unstale":"2024-05-17T22:07:13.875849+0000","last_undegraded":"2024-05-17T22:07:13.875849+0000","last_fullsized":"2024-05-17T22:07:13.875849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:10.883158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":272,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906407+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:07:13.906407+0000","last_peered":"2024-05-17T22:07:13.906407+0000","last_clean":"2024-05-17T22:07:13.906407+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:07:13.906407+0000","last_undegraded":"2024-05-17T22:07:13.906407+0000","last_fullsized":"2024-05-17T22:07:13.906407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1945600,"data_stored":1171682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":587068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":581302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":582380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":580822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1957888,"data_stored":1172790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:07:16.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:07:17.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 Processor -- start 2024-05-17T22:07:17.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 -- start start 2024-05-17T22:07:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0424144080 0x7f0424072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f042406f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0424073780 con 0x7f04240d1040 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f042415e790 con 0x7f0424144080 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.166+0000 7f042be98640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f042415ea40 con 0x7f0424073370 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04240d1040 0x7f042415e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34340/0 (socket says 172.21.15.83:34340) 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 -- 172.21.15.83:0/3791168249 learned_addr learned my addr 172.21.15.83:0/3791168249 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f042a435640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f042406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309498688 0 0) 0x7f0420001120 con 0x7f04240d1040 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429433640 1 --2- 172.21.15.83:0/3791168249 >> v2:172.21.15.155:3300/0 conn(0x7f0424144080 0x7f0424072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0410003530 con 0x7f04240d1040 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123885977 0 0) 0x7f0410003530 con 0x7f04240d1040 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 msgr2=0x7f042406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f042406f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 >> v2:172.21.15.155:3300/0 conn(0x7f0424144080 msgr2=0x7f0424072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 --2- 172.21.15.83:0/3791168249 >> v2:172.21.15.155:3300/0 conn(0x7f0424144080 0x7f0424072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042415ed20 con 0x7f04240d1040 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f04200018d0 con 0x7f04240d1040 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 >> v1:172.21.15.83:6789/0 conn(0x7f04240d1040 legacy=0x7f042415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f0410003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f0410004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0410004d50 0x7f0410005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f042415e790 con 0x7f0410003a10 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f042415ea40 con 0x7f0410004d50 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f041000ba80 con 0x7f0424073370 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f0410003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f042a435640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0410004d50 0x7f0410005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429433640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f0410004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 -- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 msgr2=0x7f0410004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0424073370 0x7f0410004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 -- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0410004d50 msgr2=0x7f0410005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0410004d50 0x7f0410005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 -- 172.21.15.83:0/3791168249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0410003530 con 0x7f0410003a10 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0429c34640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f0410003e80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f042000c490 tx=0x7f042001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e162df20bec5da38 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042000c0e0 con 0x7f0410003a10 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04200a9020 con 0x7f0410003a10 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.170+0000 7f0428c32640 1 -- 172.21.15.83:0/3791168249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200a5b90 con 0x7f0410003a10 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- 172.21.15.83:0/3791168249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 msgr2=0x7f0410003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 --2- 172.21.15.83:0/3791168249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f0410003e80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f042000c490 tx=0x7f042001cbd0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042a435640 1 -- 172.21.15.83:0/3791168249 reap_dead start 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- 172.21.15.83:0/3791168249 shutdown_connections 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- 172.21.15.83:0/3791168249 >> 172.21.15.83:0/3791168249 conn(0x7f042406cd90 msgr2=0x7f04240b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- 172.21.15.83:0/3791168249 shutdown_connections 2024-05-17T22:07:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- 172.21.15.83:0/3791168249 wait complete. 2024-05-17T22:07:17.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 Processor -- start 2024-05-17T22:07:17.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.174+0000 7f042be98640 1 -- start start 2024-05-17T22:07:17.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0424163f70 0x7f04241f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04241380b0 0x7f04240ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f042406ff40 con 0x7f0410003a10 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0424144490 con 0x7f0424163f70 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f04240700c0 con 0x7f04241380b0 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34374/0 (socket says 172.21.15.83:34374) 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 -- 172.21.15.83:0/2178426723 learned_addr learned my addr 172.21.15.83:0/2178426723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429433640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0424163f70 0x7f04241f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042a435640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04241380b0 0x7f04240ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 -- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04241380b0 msgr2=0x7f04240ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04241380b0 0x7f04240ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 -- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0424163f70 msgr2=0x7f04241f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0424163f70 0x7f04241f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 -- 172.21.15.83:0/2178426723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424165c10 con 0x7f0410003a10 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429c34640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c10 tx=0x7f0410005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420001590 con 0x7f0410003a10 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04200b3030 con 0x7f0410003a10 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04200047f0 con 0x7f0410003a10 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0424162ed0 con 0x7f0410003a10 2024-05-17T22:07:17.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0424163490 con 0x7f0410003a10 2024-05-17T22:07:17.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0424072e30 con 0x7f0410003a10 2024-05-17T22:07:17.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0420004990 con 0x7f0410003a10 2024-05-17T22:07:17.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 0x7f0408071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f0429433640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 0x7f0408071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.178+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f04201288d0 con 0x7f0410003a10 2024-05-17T22:07:17.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.182+0000 7f0429433640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 0x7f0408071910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f04140090e0 tx=0x7f0414008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.186+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0424073780 con 0x7f0410003a10 2024-05-17T22:07:17.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.342+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04240b2630 con 0x7f040806f400 2024-05-17T22:07:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.346+0000 7f041a7fc640 1 -- 172.21.15.83:0/2178426723 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252470 (secure 0 0 0) 0x7f04240b2630 con 0x7f040806f400 2024-05-17T22:07:17.342 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:17.344 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 msgr2=0x7f0408071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 0x7f0408071910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f04140090e0 tx=0x7f0414008040 comp rx=0 tx=0).stop 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 msgr2=0x7f04241f8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c10 tx=0x7f0410005840 comp rx=0 tx=0).stop 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 shutdown_connections 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04241380b0 0x7f04240ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f040806f400 0x7f0408071910 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0424163f70 0x7f04241f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 --2- 172.21.15.83:0/2178426723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0410003a10 0x7f04241f8d50 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 >> 172.21.15.83:0/2178426723 conn(0x7f042406cd90 msgr2=0x7f042415bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:17.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 shutdown_connections 2024-05-17T22:07:17.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.350+0000 7f042be98640 1 -- 172.21.15.83:0/2178426723 wait complete. 2024-05-17T22:07:17.363 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-05-17T22:07:17.185259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261200,"kb_used_data":11732,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713520,"statfs":{"total":773094113280,"available":772826644480,"internally_reserved":0,"allocated":12013568,"data_stored":5837440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001820"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907579+0000","last_change":"2024-05-17T22:07:12.898558+0000","last_active":"2024-05-17T22:07:13.907579+0000","last_peered":"2024-05-17T22:07:13.907579+0000","last_clean":"2024-05-17T22:07:13.907579+0000","last_became_active":"2024-05-17T22:07:12.898216+0000","last_became_peered":"2024-05-17T22:07:12.898216+0000","last_unstale":"2024-05-17T22:07:13.907579+0000","last_undegraded":"2024-05-17T22:07:13.907579+0000","last_fullsized":"2024-05-17T22:07:13.907579+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:14:23.201583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909573+0000","last_change":"2024-05-17T22:07:12.904636+0000","last_active":"2024-05-17T22:07:13.909573+0000","last_peered":"2024-05-17T22:07:13.909573+0000","last_clean":"2024-05-17T22:07:13.909573+0000","last_became_active":"2024-05-17T22:07:12.904281+0000","last_became_peered":"2024-05-17T22:07:12.904281+0000","last_unstale":"2024-05-17T22:07:13.909573+0000","last_undegraded":"2024-05-17T22:07:13.909573+0000","last_fullsized":"2024-05-17T22:07:13.909573+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:19.602360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906404+0000","last_change":"2024-05-17T22:07:12.907162+0000","last_active":"2024-05-17T22:07:13.906404+0000","last_peered":"2024-05-17T22:07:13.906404+0000","last_clean":"2024-05-17T22:07:13.906404+0000","last_became_active":"2024-05-17T22:07:12.906815+0000","last_became_peered":"2024-05-17T22:07:12.906815+0000","last_unstale":"2024-05-17T22:07:13.906404+0000","last_undegraded":"2024-05-17T22:07:13.906404+0000","last_fullsized":"2024-05-17T22:07:13.906404+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:48.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907821+0000","last_change":"2024-05-17T22:07:12.907580+0000","last_active":"2024-05-17T22:07:13.907821+0000","last_peered":"2024-05-17T22:07:13.907821+0000","last_clean":"2024-05-17T22:07:13.907821+0000","last_became_active":"2024-05-17T22:07:12.907265+0000","last_became_peered":"2024-05-17T22:07:12.907265+0000","last_unstale":"2024-05-17T22:07:13.907821+0000","last_undegraded":"2024-05-17T22:07:13.907821+0000","last_fullsized":"2024-05-17T22:07:13.907821+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:52.468764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877367+0000","last_change":"2024-05-17T22:07:12.892120+0000","last_active":"2024-05-17T22:07:13.877367+0000","last_peered":"2024-05-17T22:07:13.877367+0000","last_clean":"2024-05-17T22:07:13.877367+0000","last_became_active":"2024-05-17T22:07:12.891463+0000","last_became_peered":"2024-05-17T22:07:12.891463+0000","last_unstale":"2024-05-17T22:07:13.877367+0000","last_undegraded":"2024-05-17T22:07:13.877367+0000","last_fullsized":"2024-05-17T22:07:13.877367+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:12:25.955094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876477+0000","last_change":"2024-05-17T22:07:12.895722+0000","last_active":"2024-05-17T22:07:13.876477+0000","last_peered":"2024-05-17T22:07:13.876477+0000","last_clean":"2024-05-17T22:07:13.876477+0000","last_became_active":"2024-05-17T22:07:12.895369+0000","last_became_peered":"2024-05-17T22:07:12.895369+0000","last_unstale":"2024-05-17T22:07:13.876477+0000","last_undegraded":"2024-05-17T22:07:13.876477+0000","last_fullsized":"2024-05-17T22:07:13.876477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:58:15.447104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875659+0000","last_change":"2024-05-17T22:07:12.885255+0000","last_active":"2024-05-17T22:07:13.875659+0000","last_peered":"2024-05-17T22:07:13.875659+0000","last_clean":"2024-05-17T22:07:13.875659+0000","last_became_active":"2024-05-17T22:07:12.884812+0000","last_became_peered":"2024-05-17T22:07:12.884812+0000","last_unstale":"2024-05-17T22:07:13.875659+0000","last_undegraded":"2024-05-17T22:07:13.875659+0000","last_fullsized":"2024-05-17T22:07:13.875659+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:00:14.773485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.905803+0000","last_change":"2024-05-17T22:07:12.903134+0000","last_active":"2024-05-17T22:07:13.905803+0000","last_peered":"2024-05-17T22:07:13.905803+0000","last_clean":"2024-05-17T22:07:13.905803+0000","last_became_active":"2024-05-17T22:07:12.902597+0000","last_became_peered":"2024-05-17T22:07:12.902597+0000","last_unstale":"2024-05-17T22:07:13.905803+0000","last_undegraded":"2024-05-17T22:07:13.905803+0000","last_fullsized":"2024-05-17T22:07:13.905803+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907757+0000","last_change":"2024-05-17T22:07:12.897894+0000","last_active":"2024-05-17T22:07:13.907757+0000","last_peered":"2024-05-17T22:07:13.907757+0000","last_clean":"2024-05-17T22:07:13.907757+0000","last_became_active":"2024-05-17T22:07:12.897591+0000","last_became_peered":"2024-05-17T22:07:12.897591+0000","last_unstale":"2024-05-17T22:07:13.907757+0000","last_undegraded":"2024-05-17T22:07:13.907757+0000","last_fullsized":"2024-05-17T22:07:13.907757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:08.027980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908031+0000","last_change":"2024-05-17T22:07:12.889761+0000","last_active":"2024-05-17T22:07:13.908031+0000","last_peered":"2024-05-17T22:07:13.908031+0000","last_clean":"2024-05-17T22:07:13.908031+0000","last_became_active":"2024-05-17T22:07:12.889317+0000","last_became_peered":"2024-05-17T22:07:12.889317+0000","last_unstale":"2024-05-17T22:07:13.908031+0000","last_undegraded":"2024-05-17T22:07:13.908031+0000","last_fullsized":"2024-05-17T22:07:13.908031+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:03:02.914711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908963+0000","last_change":"2024-05-17T22:07:12.893157+0000","last_active":"2024-05-17T22:07:13.908963+0000","last_peered":"2024-05-17T22:07:13.908963+0000","last_clean":"2024-05-17T22:07:13.908963+0000","last_became_active":"2024-05-17T22:07:12.892725+0000","last_became_peered":"2024-05-17T22:07:12.892725+0000","last_unstale":"2024-05-17T22:07:13.908963+0000","last_undegraded":"2024-05-17T22:07:13.908963+0000","last_fullsized":"2024-05-17T22:07:13.908963+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:29:16.183947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908243+0000","last_change":"2024-05-17T22:07:12.889722+0000","last_active":"2024-05-17T22:07:13.908243+0000","last_peered":"2024-05-17T22:07:13.908243+0000","last_clean":"2024-05-17T22:07:13.908243+0000","last_became_active":"2024-05-17T22:07:12.889275+0000","last_became_peered":"2024-05-17T22:07:12.889275+0000","last_unstale":"2024-05-17T22:07:13.908243+0000","last_undegraded":"2024-05-17T22:07:13.908243+0000","last_fullsized":"2024-05-17T22:07:13.908243+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:18:32.272231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907001+0000","last_change":"2024-05-17T22:07:12.911084+0000","last_active":"2024-05-17T22:07:13.907001+0000","last_peered":"2024-05-17T22:07:13.907001+0000","last_clean":"2024-05-17T22:07:13.907001+0000","last_became_active":"2024-05-17T22:07:12.910758+0000","last_became_peered":"2024-05-17T22:07:12.910758+0000","last_unstale":"2024-05-17T22:07:13.907001+0000","last_undegraded":"2024-05-17T22:07:13.907001+0000","last_fullsized":"2024-05-17T22:07:13.907001+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:21.451864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906303+0000","last_change":"2024-05-17T22:07:12.903576+0000","last_active":"2024-05-17T22:07:13.906303+0000","last_peered":"2024-05-17T22:07:13.906303+0000","last_clean":"2024-05-17T22:07:13.906303+0000","last_became_active":"2024-05-17T22:07:12.902898+0000","last_became_peered":"2024-05-17T22:07:12.902898+0000","last_unstale":"2024-05-17T22:07:13.906303+0000","last_undegraded":"2024-05-17T22:07:13.906303+0000","last_fullsized":"2024-05-17T22:07:13.906303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:52.037932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876446+0000","last_change":"2024-05-17T22:07:12.895003+0000","last_active":"2024-05-17T22:07:13.876446+0000","last_peered":"2024-05-17T22:07:13.876446+0000","last_clean":"2024-05-17T22:07:13.876446+0000","last_became_active":"2024-05-17T22:07:12.894681+0000","last_became_peered":"2024-05-17T22:07:12.894681+0000","last_unstale":"2024-05-17T22:07:13.876446+0000","last_undegraded":"2024-05-17T22:07:13.876446+0000","last_fullsized":"2024-05-17T22:07:13.876446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:11.887993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906880+0000","last_change":"2024-05-17T22:07:12.903470+0000","last_active":"2024-05-17T22:07:13.906880+0000","last_peered":"2024-05-17T22:07:13.906880+0000","last_clean":"2024-05-17T22:07:13.906880+0000","last_became_active":"2024-05-17T22:07:12.903122+0000","last_became_peered":"2024-05-17T22:07:12.903122+0000","last_unstale":"2024-05-17T22:07:13.906880+0000","last_undegraded":"2024-05-17T22:07:13.906880+0000","last_fullsized":"2024-05-17T22:07:13.906880+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T04:39:31.135430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909818+0000","last_change":"2024-05-17T22:07:12.893675+0000","last_active":"2024-05-17T22:07:13.909818+0000","last_peered":"2024-05-17T22:07:13.909818+0000","last_clean":"2024-05-17T22:07:13.909818+0000","last_became_active":"2024-05-17T22:07:12.893195+0000","last_became_peered":"2024-05-17T22:07:12.893195+0000","last_unstale":"2024-05-17T22:07:13.909818+0000","last_undegraded":"2024-05-17T22:07:13.909818+0000","last_fullsized":"2024-05-17T22:07:13.909818+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908145+0000","last_change":"2024-05-17T22:07:12.904079+0000","last_active":"2024-05-17T22:07:13.908145+0000","last_peered":"2024-05-17T22:07:13.908145+0000","last_clean":"2024-05-17T22:07:13.908145+0000","last_became_active":"2024-05-17T22:07:12.903788+0000","last_became_peered":"2024-05-17T22:07:12.903788+0000","last_unstale":"2024-05-17T22:07:13.908145+0000","last_undegraded":"2024-05-17T22:07:13.908145+0000","last_fullsized":"2024-05-17T22:07:13.908145+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:27.421743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906525+0000","last_change":"2024-05-17T22:07:12.911244+0000","last_active":"2024-05-17T22:07:13.906525+0000","last_peered":"2024-05-17T22:07:13.906525+0000","last_clean":"2024-05-17T22:07:13.906525+0000","last_became_active":"2024-05-17T22:07:12.910887+0000","last_became_peered":"2024-05-17T22:07:12.910887+0000","last_unstale":"2024-05-17T22:07:13.906525+0000","last_undegraded":"2024-05-17T22:07:13.906525+0000","last_fullsized":"2024-05-17T22:07:13.906525+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:27.984446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909296+0000","last_change":"2024-05-17T22:07:12.909303+0000","last_active":"2024-05-17T22:07:13.909296+0000","last_peered":"2024-05-17T22:07:13.909296+0000","last_clean":"2024-05-17T22:07:13.909296+0000","last_became_active":"2024-05-17T22:07:12.908890+0000","last_became_peered":"2024-05-17T22:07:12.908890+0000","last_unstale":"2024-05-17T22:07:13.909296+0000","last_undegraded":"2024-05-17T22:07:13.909296+0000","last_fullsized":"2024-05-17T22:07:13.909296+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:52:27.421743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907969+0000","last_change":"2024-05-17T22:07:12.895060+0000","last_active":"2024-05-17T22:07:13.907969+0000","last_peered":"2024-05-17T22:07:13.907969+0000","last_clean":"2024-05-17T22:07:13.907969+0000","last_became_active":"2024-05-17T22:07:12.894762+0000","last_became_peered":"2024-05-17T22:07:12.894762+0000","last_unstale":"2024-05-17T22:07:13.907969+0000","last_undegraded":"2024-05-17T22:07:13.907969+0000","last_fullsized":"2024-05-17T22:07:13.907969+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:20:29.625033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909313+0000","last_change":"2024-05-17T22:07:12.894526+0000","last_active":"2024-05-17T22:07:13.909313+0000","last_peered":"2024-05-17T22:07:13.909313+0000","last_clean":"2024-05-17T22:07:13.909313+0000","last_became_active":"2024-05-17T22:07:12.894012+0000","last_became_peered":"2024-05-17T22:07:12.894012+0000","last_unstale":"2024-05-17T22:07:13.909313+0000","last_undegraded":"2024-05-17T22:07:13.909313+0000","last_fullsized":"2024-05-17T22:07:13.909313+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:48.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906010+0000","last_change":"2024-05-17T22:07:12.900979+0000","last_active":"2024-05-17T22:07:13.906010+0000","last_peered":"2024-05-17T22:07:13.906010+0000","last_clean":"2024-05-17T22:07:13.906010+0000","last_became_active":"2024-05-17T22:07:12.900422+0000","last_became_peered":"2024-05-17T22:07:12.900422+0000","last_unstale":"2024-05-17T22:07:13.906010+0000","last_undegraded":"2024-05-17T22:07:13.906010+0000","last_fullsized":"2024-05-17T22:07:13.906010+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:44.415742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908987+0000","last_change":"2024-05-17T22:07:12.901065+0000","last_active":"2024-05-17T22:07:13.908987+0000","last_peered":"2024-05-17T22:07:13.908987+0000","last_clean":"2024-05-17T22:07:13.908987+0000","last_became_active":"2024-05-17T22:07:12.900665+0000","last_became_peered":"2024-05-17T22:07:12.900665+0000","last_unstale":"2024-05-17T22:07:13.908987+0000","last_undegraded":"2024-05-17T22:07:13.908987+0000","last_fullsized":"2024-05-17T22:07:13.908987+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:36.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910070+0000","last_change":"2024-05-17T22:07:12.891615+0000","last_active":"2024-05-17T22:07:13.910070+0000","last_peered":"2024-05-17T22:07:13.910070+0000","last_clean":"2024-05-17T22:07:13.910070+0000","last_became_active":"2024-05-17T22:07:12.891155+0000","last_became_peered":"2024-05-17T22:07:12.891155+0000","last_unstale":"2024-05-17T22:07:13.910070+0000","last_undegraded":"2024-05-17T22:07:13.910070+0000","last_fullsized":"2024-05-17T22:07:13.910070+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908444+0000","last_change":"2024-05-17T22:07:12.886931+0000","last_active":"2024-05-17T22:07:13.908444+0000","last_peered":"2024-05-17T22:07:13.908444+0000","last_clean":"2024-05-17T22:07:13.908444+0000","last_became_active":"2024-05-17T22:07:12.886484+0000","last_became_peered":"2024-05-17T22:07:12.886484+0000","last_unstale":"2024-05-17T22:07:13.908444+0000","last_undegraded":"2024-05-17T22:07:13.908444+0000","last_fullsized":"2024-05-17T22:07:13.908444+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:09:32.656030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878340+0000","last_change":"2024-05-17T22:07:12.908746+0000","last_active":"2024-05-17T22:07:13.878340+0000","last_peered":"2024-05-17T22:07:13.878340+0000","last_clean":"2024-05-17T22:07:13.878340+0000","last_became_active":"2024-05-17T22:07:12.907430+0000","last_became_peered":"2024-05-17T22:07:12.907430+0000","last_unstale":"2024-05-17T22:07:13.878340+0000","last_undegraded":"2024-05-17T22:07:13.878340+0000","last_fullsized":"2024-05-17T22:07:13.878340+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:52:17.601039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909099+0000","last_change":"2024-05-17T22:07:12.907834+0000","last_active":"2024-05-17T22:07:13.909099+0000","last_peered":"2024-05-17T22:07:13.909099+0000","last_clean":"2024-05-17T22:07:13.909099+0000","last_became_active":"2024-05-17T22:07:12.906131+0000","last_became_peered":"2024-05-17T22:07:12.906131+0000","last_unstale":"2024-05-17T22:07:13.909099+0000","last_undegraded":"2024-05-17T22:07:13.909099+0000","last_fullsized":"2024-05-17T22:07:13.909099+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:52.037932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908443+0000","last_change":"2024-05-17T22:07:12.909601+0000","last_active":"2024-05-17T22:07:13.908443+0000","last_peered":"2024-05-17T22:07:13.908443+0000","last_clean":"2024-05-17T22:07:13.908443+0000","last_became_active":"2024-05-17T22:07:12.909177+0000","last_became_peered":"2024-05-17T22:07:12.909177+0000","last_unstale":"2024-05-17T22:07:13.908443+0000","last_undegraded":"2024-05-17T22:07:13.908443+0000","last_fullsized":"2024-05-17T22:07:13.908443+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:23:27.384436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875910+0000","last_change":"2024-05-17T22:07:12.895535+0000","last_active":"2024-05-17T22:07:13.875910+0000","last_peered":"2024-05-17T22:07:13.875910+0000","last_clean":"2024-05-17T22:07:13.875910+0000","last_became_active":"2024-05-17T22:07:12.894923+0000","last_became_peered":"2024-05-17T22:07:12.894923+0000","last_unstale":"2024-05-17T22:07:13.875910+0000","last_undegraded":"2024-05-17T22:07:13.875910+0000","last_fullsized":"2024-05-17T22:07:13.875910+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:08.602957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907559+0000","last_change":"2024-05-17T22:07:12.891323+0000","last_active":"2024-05-17T22:07:13.907559+0000","last_peered":"2024-05-17T22:07:13.907559+0000","last_clean":"2024-05-17T22:07:13.907559+0000","last_became_active":"2024-05-17T22:07:12.890975+0000","last_became_peered":"2024-05-17T22:07:12.890975+0000","last_unstale":"2024-05-17T22:07:13.907559+0000","last_undegraded":"2024-05-17T22:07:13.907559+0000","last_fullsized":"2024-05-17T22:07:13.907559+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:12:25.082875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875815+0000","last_change":"2024-05-17T22:07:12.902405+0000","last_active":"2024-05-17T22:07:13.875815+0000","last_peered":"2024-05-17T22:07:13.875815+0000","last_clean":"2024-05-17T22:07:13.875815+0000","last_became_active":"2024-05-17T22:07:12.894863+0000","last_became_peered":"2024-05-17T22:07:12.894863+0000","last_unstale":"2024-05-17T22:07:13.875815+0000","last_undegraded":"2024-05-17T22:07:13.875815+0000","last_fullsized":"2024-05-17T22:07:13.875815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:58.409599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878517+0000","last_change":"2024-05-17T22:07:12.881675+0000","last_active":"2024-05-17T22:07:13.878517+0000","last_peered":"2024-05-17T22:07:13.878517+0000","last_clean":"2024-05-17T22:07:13.878517+0000","last_became_active":"2024-05-17T22:07:12.881334+0000","last_became_peered":"2024-05-17T22:07:12.881334+0000","last_unstale":"2024-05-17T22:07:13.878517+0000","last_undegraded":"2024-05-17T22:07:13.878517+0000","last_fullsized":"2024-05-17T22:07:13.878517+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:29:43.812078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876447+0000","last_change":"2024-05-17T22:07:11.901401+0000","last_active":"2024-05-17T22:07:13.876447+0000","last_peered":"2024-05-17T22:07:13.876447+0000","last_clean":"2024-05-17T22:07:13.876447+0000","last_became_active":"2024-05-17T22:07:11.899245+0000","last_became_peered":"2024-05-17T22:07:11.899245+0000","last_unstale":"2024-05-17T22:07:13.876447+0000","last_undegraded":"2024-05-17T22:07:13.876447+0000","last_fullsized":"2024-05-17T22:07:13.876447+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:15:57.404764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908874+0000","last_change":"2024-05-17T22:07:12.897571+0000","last_active":"2024-05-17T22:07:13.908874+0000","last_peered":"2024-05-17T22:07:13.908874+0000","last_clean":"2024-05-17T22:07:13.908874+0000","last_became_active":"2024-05-17T22:07:12.896706+0000","last_became_peered":"2024-05-17T22:07:12.896706+0000","last_unstale":"2024-05-17T22:07:13.908874+0000","last_undegraded":"2024-05-17T22:07:13.908874+0000","last_fullsized":"2024-05-17T22:07:13.908874+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906425+0000","last_change":"2024-05-17T22:07:11.909824+0000","last_active":"2024-05-17T22:07:13.906425+0000","last_peered":"2024-05-17T22:07:13.906425+0000","last_clean":"2024-05-17T22:07:13.906425+0000","last_became_active":"2024-05-17T22:07:11.908945+0000","last_became_peered":"2024-05-17T22:07:11.908945+0000","last_unstale":"2024-05-17T22:07:13.906425+0000","last_undegraded":"2024-05-17T22:07:13.906425+0000","last_fullsized":"2024-05-17T22:07:13.906425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877859+0000","last_change":"2024-05-17T22:07:12.895440+0000","last_active":"2024-05-17T22:07:13.877859+0000","last_peered":"2024-05-17T22:07:13.877859+0000","last_clean":"2024-05-17T22:07:13.877859+0000","last_became_active":"2024-05-17T22:07:12.894968+0000","last_became_peered":"2024-05-17T22:07:12.894968+0000","last_unstale":"2024-05-17T22:07:13.877859+0000","last_undegraded":"2024-05-17T22:07:13.877859+0000","last_fullsized":"2024-05-17T22:07:13.877859+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:11:15.269098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878783+0000","last_change":"2024-05-17T22:07:11.888421+0000","last_active":"2024-05-17T22:07:13.878783+0000","last_peered":"2024-05-17T22:07:13.878783+0000","last_clean":"2024-05-17T22:07:13.878783+0000","last_became_active":"2024-05-17T22:07:11.887998+0000","last_became_peered":"2024-05-17T22:07:11.887998+0000","last_unstale":"2024-05-17T22:07:13.878783+0000","last_undegraded":"2024-05-17T22:07:13.878783+0000","last_fullsized":"2024-05-17T22:07:13.878783+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:13:35.636709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906603+0000","last_change":"2024-05-17T22:07:12.911263+0000","last_active":"2024-05-17T22:07:13.906603+0000","last_peered":"2024-05-17T22:07:13.906603+0000","last_clean":"2024-05-17T22:07:13.906603+0000","last_became_active":"2024-05-17T22:07:12.910950+0000","last_became_peered":"2024-05-17T22:07:12.910950+0000","last_unstale":"2024-05-17T22:07:13.906603+0000","last_undegraded":"2024-05-17T22:07:13.906603+0000","last_fullsized":"2024-05-17T22:07:13.906603+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:34.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876800+0000","last_change":"2024-05-17T22:07:11.884897+0000","last_active":"2024-05-17T22:07:13.876800+0000","last_peered":"2024-05-17T22:07:13.876800+0000","last_clean":"2024-05-17T22:07:13.876800+0000","last_became_active":"2024-05-17T22:07:11.884426+0000","last_became_peered":"2024-05-17T22:07:11.884426+0000","last_unstale":"2024-05-17T22:07:13.876800+0000","last_undegraded":"2024-05-17T22:07:13.876800+0000","last_fullsized":"2024-05-17T22:07:13.876800+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:13:59.812540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908895+0000","last_change":"2024-05-17T22:07:12.898533+0000","last_active":"2024-05-17T22:07:13.908895+0000","last_peered":"2024-05-17T22:07:13.908895+0000","last_clean":"2024-05-17T22:07:13.908895+0000","last_became_active":"2024-05-17T22:07:12.897257+0000","last_became_peered":"2024-05-17T22:07:12.897257+0000","last_unstale":"2024-05-17T22:07:13.908895+0000","last_undegraded":"2024-05-17T22:07:13.908895+0000","last_fullsized":"2024-05-17T22:07:13.908895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:41:48.406012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875766+0000","last_change":"2024-05-17T22:07:11.895917+0000","last_active":"2024-05-17T22:07:13.875766+0000","last_peered":"2024-05-17T22:07:13.875766+0000","last_clean":"2024-05-17T22:07:13.875766+0000","last_became_active":"2024-05-17T22:07:11.894741+0000","last_became_peered":"2024-05-17T22:07:11.894741+0000","last_unstale":"2024-05-17T22:07:13.875766+0000","last_undegraded":"2024-05-17T22:07:13.875766+0000","last_fullsized":"2024-05-17T22:07:13.875766+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:37:53.814524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908986+0000","last_change":"2024-05-17T22:07:12.904414+0000","last_active":"2024-05-17T22:07:13.908986+0000","last_peered":"2024-05-17T22:07:13.908986+0000","last_clean":"2024-05-17T22:07:13.908986+0000","last_became_active":"2024-05-17T22:07:12.904094+0000","last_became_peered":"2024-05-17T22:07:12.904094+0000","last_unstale":"2024-05-17T22:07:13.908986+0000","last_undegraded":"2024-05-17T22:07:13.908986+0000","last_fullsized":"2024-05-17T22:07:13.908986+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T03:35:37.595590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909518+0000","last_change":"2024-05-17T22:07:11.912304+0000","last_active":"2024-05-17T22:07:13.909518+0000","last_peered":"2024-05-17T22:07:13.909518+0000","last_clean":"2024-05-17T22:07:13.909518+0000","last_became_active":"2024-05-17T22:07:11.911850+0000","last_became_peered":"2024-05-17T22:07:11.911850+0000","last_unstale":"2024-05-17T22:07:13.909518+0000","last_undegraded":"2024-05-17T22:07:13.909518+0000","last_fullsized":"2024-05-17T22:07:13.909518+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:33:31.010502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906899+0000","last_change":"2024-05-17T22:07:12.911736+0000","last_active":"2024-05-17T22:07:13.906899+0000","last_peered":"2024-05-17T22:07:13.906899+0000","last_clean":"2024-05-17T22:07:13.906899+0000","last_became_active":"2024-05-17T22:07:12.911255+0000","last_became_peered":"2024-05-17T22:07:12.911255+0000","last_unstale":"2024-05-17T22:07:13.906899+0000","last_undegraded":"2024-05-17T22:07:13.906899+0000","last_fullsized":"2024-05-17T22:07:13.906899+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:52:55.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.905811+0000","last_change":"2024-05-17T22:07:11.893568+0000","last_active":"2024-05-17T22:07:13.905811+0000","last_peered":"2024-05-17T22:07:13.905811+0000","last_clean":"2024-05-17T22:07:13.905811+0000","last_became_active":"2024-05-17T22:07:11.892608+0000","last_became_peered":"2024-05-17T22:07:11.892608+0000","last_unstale":"2024-05-17T22:07:13.905811+0000","last_undegraded":"2024-05-17T22:07:13.905811+0000","last_fullsized":"2024-05-17T22:07:13.905811+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:57:57.758811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875681+0000","last_change":"2024-05-17T22:07:12.887053+0000","last_active":"2024-05-17T22:07:13.875681+0000","last_peered":"2024-05-17T22:07:13.875681+0000","last_clean":"2024-05-17T22:07:13.875681+0000","last_became_active":"2024-05-17T22:07:12.886458+0000","last_became_peered":"2024-05-17T22:07:12.886458+0000","last_unstale":"2024-05-17T22:07:13.875681+0000","last_undegraded":"2024-05-17T22:07:13.875681+0000","last_fullsized":"2024-05-17T22:07:13.875681+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:43:24.652042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907430+0000","last_change":"2024-05-17T22:07:11.894673+0000","last_active":"2024-05-17T22:07:13.907430+0000","last_peered":"2024-05-17T22:07:13.907430+0000","last_clean":"2024-05-17T22:07:13.907430+0000","last_became_active":"2024-05-17T22:07:11.891142+0000","last_became_peered":"2024-05-17T22:07:11.891142+0000","last_unstale":"2024-05-17T22:07:13.907430+0000","last_undegraded":"2024-05-17T22:07:13.907430+0000","last_fullsized":"2024-05-17T22:07:13.907430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:26.979611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876819+0000","last_change":"2024-05-17T22:07:12.878188+0000","last_active":"2024-05-17T22:07:13.876819+0000","last_peered":"2024-05-17T22:07:13.876819+0000","last_clean":"2024-05-17T22:07:13.876819+0000","last_became_active":"2024-05-17T22:07:12.877735+0000","last_became_peered":"2024-05-17T22:07:12.877735+0000","last_unstale":"2024-05-17T22:07:13.876819+0000","last_undegraded":"2024-05-17T22:07:13.876819+0000","last_fullsized":"2024-05-17T22:07:13.876819+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:57:54.351637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878954+0000","last_change":"2024-05-17T22:07:11.895060+0000","last_active":"2024-05-17T22:07:13.878954+0000","last_peered":"2024-05-17T22:07:13.878954+0000","last_clean":"2024-05-17T22:07:13.878954+0000","last_became_active":"2024-05-17T22:07:11.894710+0000","last_became_peered":"2024-05-17T22:07:11.894710+0000","last_unstale":"2024-05-17T22:07:13.878954+0000","last_undegraded":"2024-05-17T22:07:13.878954+0000","last_fullsized":"2024-05-17T22:07:13.878954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:22:25.602641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908686+0000","last_change":"2024-05-17T22:07:12.898814+0000","last_active":"2024-05-17T22:07:13.908686+0000","last_peered":"2024-05-17T22:07:13.908686+0000","last_clean":"2024-05-17T22:07:13.908686+0000","last_became_active":"2024-05-17T22:07:12.898460+0000","last_became_peered":"2024-05-17T22:07:12.898460+0000","last_unstale":"2024-05-17T22:07:13.908686+0000","last_undegraded":"2024-05-17T22:07:13.908686+0000","last_fullsized":"2024-05-17T22:07:13.908686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909349+0000","last_change":"2024-05-17T22:07:11.912716+0000","last_active":"2024-05-17T22:07:13.909349+0000","last_peered":"2024-05-17T22:07:13.909349+0000","last_clean":"2024-05-17T22:07:13.909349+0000","last_became_active":"2024-05-17T22:07:11.912339+0000","last_became_peered":"2024-05-17T22:07:11.912339+0000","last_unstale":"2024-05-17T22:07:13.909349+0000","last_undegraded":"2024-05-17T22:07:13.909349+0000","last_fullsized":"2024-05-17T22:07:13.909349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909589+0000","last_change":"2024-05-17T22:07:12.906225+0000","last_active":"2024-05-17T22:07:13.909589+0000","last_peered":"2024-05-17T22:07:13.909589+0000","last_clean":"2024-05-17T22:07:13.909589+0000","last_became_active":"2024-05-17T22:07:12.905908+0000","last_became_peered":"2024-05-17T22:07:12.905908+0000","last_unstale":"2024-05-17T22:07:13.909589+0000","last_undegraded":"2024-05-17T22:07:13.909589+0000","last_fullsized":"2024-05-17T22:07:13.909589+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:34.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910130+0000","last_change":"2024-05-17T22:07:11.895488+0000","last_active":"2024-05-17T22:07:13.910130+0000","last_peered":"2024-05-17T22:07:13.910130+0000","last_clean":"2024-05-17T22:07:13.910130+0000","last_became_active":"2024-05-17T22:07:11.892284+0000","last_became_peered":"2024-05-17T22:07:11.892284+0000","last_unstale":"2024-05-17T22:07:13.910130+0000","last_undegraded":"2024-05-17T22:07:13.910130+0000","last_fullsized":"2024-05-17T22:07:13.910130+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:23:51.033097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876323+0000","last_change":"2024-05-17T22:07:12.905530+0000","last_active":"2024-05-17T22:07:13.876323+0000","last_peered":"2024-05-17T22:07:13.876323+0000","last_clean":"2024-05-17T22:07:13.876323+0000","last_became_active":"2024-05-17T22:07:12.905205+0000","last_became_peered":"2024-05-17T22:07:12.905205+0000","last_unstale":"2024-05-17T22:07:13.876323+0000","last_undegraded":"2024-05-17T22:07:13.876323+0000","last_fullsized":"2024-05-17T22:07:13.876323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:07:03.740195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908993+0000","last_change":"2024-05-17T22:07:11.895716+0000","last_active":"2024-05-17T22:07:13.908993+0000","last_peered":"2024-05-17T22:07:13.908993+0000","last_clean":"2024-05-17T22:07:13.908993+0000","last_became_active":"2024-05-17T22:07:11.895022+0000","last_became_peered":"2024-05-17T22:07:11.895022+0000","last_unstale":"2024-05-17T22:07:13.908993+0000","last_undegraded":"2024-05-17T22:07:13.908993+0000","last_fullsized":"2024-05-17T22:07:13.908993+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:35.090102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909635+0000","last_change":"2024-05-17T22:07:12.890441+0000","last_active":"2024-05-17T22:07:13.909635+0000","last_peered":"2024-05-17T22:07:13.909635+0000","last_clean":"2024-05-17T22:07:13.909635+0000","last_became_active":"2024-05-17T22:07:12.888492+0000","last_became_peered":"2024-05-17T22:07:12.888492+0000","last_unstale":"2024-05-17T22:07:13.909635+0000","last_undegraded":"2024-05-17T22:07:13.909635+0000","last_fullsized":"2024-05-17T22:07:13.909635+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:10:08.731202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877042+0000","last_change":"2024-05-17T22:07:11.898845+0000","last_active":"2024-05-17T22:07:13.877042+0000","last_peered":"2024-05-17T22:07:13.877042+0000","last_clean":"2024-05-17T22:07:13.877042+0000","last_became_active":"2024-05-17T22:07:11.898467+0000","last_became_peered":"2024-05-17T22:07:11.898467+0000","last_unstale":"2024-05-17T22:07:13.877042+0000","last_undegraded":"2024-05-17T22:07:13.877042+0000","last_fullsized":"2024-05-17T22:07:13.877042+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:16:03.597683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908401+0000","last_change":"2024-05-17T22:07:12.900927+0000","last_active":"2024-05-17T22:07:13.908401+0000","last_peered":"2024-05-17T22:07:13.908401+0000","last_clean":"2024-05-17T22:07:13.908401+0000","last_became_active":"2024-05-17T22:07:12.900601+0000","last_became_peered":"2024-05-17T22:07:12.900601+0000","last_unstale":"2024-05-17T22:07:13.908401+0000","last_undegraded":"2024-05-17T22:07:13.908401+0000","last_fullsized":"2024-05-17T22:07:13.908401+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:37:33.172117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908023+0000","last_change":"2024-05-17T22:07:11.906951+0000","last_active":"2024-05-17T22:07:13.908023+0000","last_peered":"2024-05-17T22:07:13.908023+0000","last_clean":"2024-05-17T22:07:13.908023+0000","last_became_active":"2024-05-17T22:07:11.906186+0000","last_became_peered":"2024-05-17T22:07:11.906186+0000","last_unstale":"2024-05-17T22:07:13.908023+0000","last_undegraded":"2024-05-17T22:07:13.908023+0000","last_fullsized":"2024-05-17T22:07:13.908023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:04:32.566249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908515+0000","last_change":"2024-05-17T22:07:12.908531+0000","last_active":"2024-05-17T22:07:13.908515+0000","last_peered":"2024-05-17T22:07:13.908515+0000","last_clean":"2024-05-17T22:07:13.908515+0000","last_became_active":"2024-05-17T22:07:12.906869+0000","last_became_peered":"2024-05-17T22:07:12.906869+0000","last_unstale":"2024-05-17T22:07:13.908515+0000","last_undegraded":"2024-05-17T22:07:13.908515+0000","last_fullsized":"2024-05-17T22:07:13.908515+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875570+0000","last_change":"2024-05-17T22:07:11.885064+0000","last_active":"2024-05-17T22:07:13.875570+0000","last_peered":"2024-05-17T22:07:13.875570+0000","last_clean":"2024-05-17T22:07:13.875570+0000","last_became_active":"2024-05-17T22:07:11.884646+0000","last_became_peered":"2024-05-17T22:07:11.884646+0000","last_unstale":"2024-05-17T22:07:13.875570+0000","last_undegraded":"2024-05-17T22:07:13.875570+0000","last_fullsized":"2024-05-17T22:07:13.875570+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:15:08.018276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.879327+0000","last_change":"2024-05-17T22:07:12.907579+0000","last_active":"2024-05-17T22:07:13.879327+0000","last_peered":"2024-05-17T22:07:13.879327+0000","last_clean":"2024-05-17T22:07:13.879327+0000","last_became_active":"2024-05-17T22:07:12.906764+0000","last_became_peered":"2024-05-17T22:07:12.906764+0000","last_unstale":"2024-05-17T22:07:13.879327+0000","last_undegraded":"2024-05-17T22:07:13.879327+0000","last_fullsized":"2024-05-17T22:07:13.879327+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:04:17.525330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908086+0000","last_change":"2024-05-17T22:07:11.890717+0000","last_active":"2024-05-17T22:07:13.908086+0000","last_peered":"2024-05-17T22:07:13.908086+0000","last_clean":"2024-05-17T22:07:13.908086+0000","last_became_active":"2024-05-17T22:07:11.888502+0000","last_became_peered":"2024-05-17T22:07:11.888502+0000","last_unstale":"2024-05-17T22:07:13.908086+0000","last_undegraded":"2024-05-17T22:07:13.908086+0000","last_fullsized":"2024-05-17T22:07:13.908086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:05:20.447029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.878479+0000","last_change":"2024-05-17T22:07:12.907027+0000","last_active":"2024-05-17T22:07:13.878479+0000","last_peered":"2024-05-17T22:07:13.878479+0000","last_clean":"2024-05-17T22:07:13.878479+0000","last_became_active":"2024-05-17T22:07:12.906731+0000","last_became_peered":"2024-05-17T22:07:12.906731+0000","last_unstale":"2024-05-17T22:07:13.878479+0000","last_undegraded":"2024-05-17T22:07:13.878479+0000","last_fullsized":"2024-05-17T22:07:13.878479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:28:09.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.879208+0000","last_change":"2024-05-17T22:07:11.896543+0000","last_active":"2024-05-17T22:07:13.879208+0000","last_peered":"2024-05-17T22:07:13.879208+0000","last_clean":"2024-05-17T22:07:13.879208+0000","last_became_active":"2024-05-17T22:07:11.896046+0000","last_became_peered":"2024-05-17T22:07:11.896046+0000","last_unstale":"2024-05-17T22:07:13.879208+0000","last_undegraded":"2024-05-17T22:07:13.879208+0000","last_fullsized":"2024-05-17T22:07:13.879208+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:21:40.545425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906149+0000","last_change":"2024-05-17T22:07:12.912550+0000","last_active":"2024-05-17T22:07:13.906149+0000","last_peered":"2024-05-17T22:07:13.906149+0000","last_clean":"2024-05-17T22:07:13.906149+0000","last_became_active":"2024-05-17T22:07:12.912293+0000","last_became_peered":"2024-05-17T22:07:12.912293+0000","last_unstale":"2024-05-17T22:07:13.906149+0000","last_undegraded":"2024-05-17T22:07:13.906149+0000","last_fullsized":"2024-05-17T22:07:13.906149+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:47:39.236555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908366+0000","last_change":"2024-05-17T22:07:11.906024+0000","last_active":"2024-05-17T22:07:13.908366+0000","last_peered":"2024-05-17T22:07:13.908366+0000","last_clean":"2024-05-17T22:07:13.908366+0000","last_became_active":"2024-05-17T22:07:11.905294+0000","last_became_peered":"2024-05-17T22:07:11.905294+0000","last_unstale":"2024-05-17T22:07:13.908366+0000","last_undegraded":"2024-05-17T22:07:13.908366+0000","last_fullsized":"2024-05-17T22:07:13.908366+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:33:31.010502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906653+0000","last_change":"2024-05-17T22:07:12.908313+0000","last_active":"2024-05-17T22:07:13.906653+0000","last_peered":"2024-05-17T22:07:13.906653+0000","last_clean":"2024-05-17T22:07:13.906653+0000","last_became_active":"2024-05-17T22:07:12.907905+0000","last_became_peered":"2024-05-17T22:07:12.907905+0000","last_unstale":"2024-05-17T22:07:13.906653+0000","last_undegraded":"2024-05-17T22:07:13.906653+0000","last_fullsized":"2024-05-17T22:07:13.906653+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:06.484085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876102+0000","last_change":"2024-05-17T22:07:11.901167+0000","last_active":"2024-05-17T22:07:13.876102+0000","last_peered":"2024-05-17T22:07:13.876102+0000","last_clean":"2024-05-17T22:07:13.876102+0000","last_became_active":"2024-05-17T22:07:11.900281+0000","last_became_peered":"2024-05-17T22:07:11.900281+0000","last_unstale":"2024-05-17T22:07:13.876102+0000","last_undegraded":"2024-05-17T22:07:13.876102+0000","last_fullsized":"2024-05-17T22:07:13.876102+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:54:08.455507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906658+0000","last_change":"2024-05-17T22:07:12.908276+0000","last_active":"2024-05-17T22:07:13.906658+0000","last_peered":"2024-05-17T22:07:13.906658+0000","last_clean":"2024-05-17T22:07:13.906658+0000","last_became_active":"2024-05-17T22:07:12.907961+0000","last_became_peered":"2024-05-17T22:07:12.907961+0000","last_unstale":"2024-05-17T22:07:13.906658+0000","last_undegraded":"2024-05-17T22:07:13.906658+0000","last_fullsized":"2024-05-17T22:07:13.906658+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:10:08.731202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.877137+0000","last_change":"2024-05-17T22:07:11.887035+0000","last_active":"2024-05-17T22:07:13.877137+0000","last_peered":"2024-05-17T22:07:13.877137+0000","last_clean":"2024-05-17T22:07:13.877137+0000","last_became_active":"2024-05-17T22:07:11.886710+0000","last_became_peered":"2024-05-17T22:07:11.886710+0000","last_unstale":"2024-05-17T22:07:13.877137+0000","last_undegraded":"2024-05-17T22:07:13.877137+0000","last_fullsized":"2024-05-17T22:07:13.877137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:36:28.613813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907417+0000","last_change":"2024-05-17T22:07:12.903589+0000","last_active":"2024-05-17T22:07:13.907417+0000","last_peered":"2024-05-17T22:07:13.907417+0000","last_clean":"2024-05-17T22:07:13.907417+0000","last_became_active":"2024-05-17T22:07:12.903274+0000","last_became_peered":"2024-05-17T22:07:12.903274+0000","last_unstale":"2024-05-17T22:07:13.907417+0000","last_undegraded":"2024-05-17T22:07:13.907417+0000","last_fullsized":"2024-05-17T22:07:13.907417+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:24:46.706528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875712+0000","last_change":"2024-05-17T22:07:11.892483+0000","last_active":"2024-05-17T22:07:13.875712+0000","last_peered":"2024-05-17T22:07:13.875712+0000","last_clean":"2024-05-17T22:07:13.875712+0000","last_became_active":"2024-05-17T22:07:11.892041+0000","last_became_peered":"2024-05-17T22:07:11.892041+0000","last_unstale":"2024-05-17T22:07:13.875712+0000","last_undegraded":"2024-05-17T22:07:13.875712+0000","last_fullsized":"2024-05-17T22:07:13.875712+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:25:39.388810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906221+0000","last_change":"2024-05-17T22:07:12.894904+0000","last_active":"2024-05-17T22:07:13.906221+0000","last_peered":"2024-05-17T22:07:13.906221+0000","last_clean":"2024-05-17T22:07:13.906221+0000","last_became_active":"2024-05-17T22:07:12.894272+0000","last_became_peered":"2024-05-17T22:07:12.894272+0000","last_unstale":"2024-05-17T22:07:13.906221+0000","last_undegraded":"2024-05-17T22:07:13.906221+0000","last_fullsized":"2024-05-17T22:07:13.906221+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:51.563793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876417+0000","last_change":"2024-05-17T22:07:11.888927+0000","last_active":"2024-05-17T22:07:13.876417+0000","last_peered":"2024-05-17T22:07:13.876417+0000","last_clean":"2024-05-17T22:07:13.876417+0000","last_became_active":"2024-05-17T22:07:11.888121+0000","last_became_peered":"2024-05-17T22:07:11.888121+0000","last_unstale":"2024-05-17T22:07:13.876417+0000","last_undegraded":"2024-05-17T22:07:13.876417+0000","last_fullsized":"2024-05-17T22:07:13.876417+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:51:25.152042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908790+0000","last_change":"2024-05-17T22:07:12.899405+0000","last_active":"2024-05-17T22:07:13.908790+0000","last_peered":"2024-05-17T22:07:13.908790+0000","last_clean":"2024-05-17T22:07:13.908790+0000","last_became_active":"2024-05-17T22:07:12.898646+0000","last_became_peered":"2024-05-17T22:07:12.898646+0000","last_unstale":"2024-05-17T22:07:13.908790+0000","last_undegraded":"2024-05-17T22:07:13.908790+0000","last_fullsized":"2024-05-17T22:07:13.908790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T07:55:51.563793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907719+0000","last_change":"2024-05-17T22:07:11.912218+0000","last_active":"2024-05-17T22:07:13.907719+0000","last_peered":"2024-05-17T22:07:13.907719+0000","last_clean":"2024-05-17T22:07:13.907719+0000","last_became_active":"2024-05-17T22:07:11.911723+0000","last_became_peered":"2024-05-17T22:07:11.911723+0000","last_unstale":"2024-05-17T22:07:13.907719+0000","last_undegraded":"2024-05-17T22:07:13.907719+0000","last_fullsized":"2024-05-17T22:07:13.907719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:14:33.534048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908293+0000","last_change":"2024-05-17T22:07:12.898139+0000","last_active":"2024-05-17T22:07:13.908293+0000","last_peered":"2024-05-17T22:07:13.908293+0000","last_clean":"2024-05-17T22:07:13.908293+0000","last_became_active":"2024-05-17T22:07:12.897778+0000","last_became_peered":"2024-05-17T22:07:12.897778+0000","last_unstale":"2024-05-17T22:07:13.908293+0000","last_undegraded":"2024-05-17T22:07:13.908293+0000","last_fullsized":"2024-05-17T22:07:13.908293+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:26:41.192008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910365+0000","last_change":"2024-05-17T22:07:11.892682+0000","last_active":"2024-05-17T22:07:13.910365+0000","last_peered":"2024-05-17T22:07:13.910365+0000","last_clean":"2024-05-17T22:07:13.910365+0000","last_became_active":"2024-05-17T22:07:11.892241+0000","last_became_peered":"2024-05-17T22:07:11.892241+0000","last_unstale":"2024-05-17T22:07:13.910365+0000","last_undegraded":"2024-05-17T22:07:13.910365+0000","last_fullsized":"2024-05-17T22:07:13.910365+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:52:43.410907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910357+0000","last_change":"2024-05-17T22:07:12.909755+0000","last_active":"2024-05-17T22:07:13.910357+0000","last_peered":"2024-05-17T22:07:13.910357+0000","last_clean":"2024-05-17T22:07:13.910357+0000","last_became_active":"2024-05-17T22:07:12.909420+0000","last_became_peered":"2024-05-17T22:07:12.909420+0000","last_unstale":"2024-05-17T22:07:13.910357+0000","last_undegraded":"2024-05-17T22:07:13.910357+0000","last_fullsized":"2024-05-17T22:07:13.910357+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:16:55.344248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908451+0000","last_change":"2024-05-17T22:07:11.906359+0000","last_active":"2024-05-17T22:07:13.908451+0000","last_peered":"2024-05-17T22:07:13.908451+0000","last_clean":"2024-05-17T22:07:13.908451+0000","last_became_active":"2024-05-17T22:07:11.905189+0000","last_became_peered":"2024-05-17T22:07:11.905189+0000","last_unstale":"2024-05-17T22:07:13.908451+0000","last_undegraded":"2024-05-17T22:07:13.908451+0000","last_fullsized":"2024-05-17T22:07:13.908451+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:59:42.372644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908680+0000","last_change":"2024-05-17T22:07:12.890471+0000","last_active":"2024-05-17T22:07:13.908680+0000","last_peered":"2024-05-17T22:07:13.908680+0000","last_clean":"2024-05-17T22:07:13.908680+0000","last_became_active":"2024-05-17T22:07:12.890108+0000","last_became_peered":"2024-05-17T22:07:12.890108+0000","last_unstale":"2024-05-17T22:07:13.908680+0000","last_undegraded":"2024-05-17T22:07:13.908680+0000","last_fullsized":"2024-05-17T22:07:13.908680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:20:39.380090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.910156+0000","last_change":"2024-05-17T22:07:11.912226+0000","last_active":"2024-05-17T22:07:13.910156+0000","last_peered":"2024-05-17T22:07:13.910156+0000","last_clean":"2024-05-17T22:07:13.910156+0000","last_became_active":"2024-05-17T22:07:11.911752+0000","last_became_peered":"2024-05-17T22:07:11.911752+0000","last_unstale":"2024-05-17T22:07:13.910156+0000","last_undegraded":"2024-05-17T22:07:13.910156+0000","last_fullsized":"2024-05-17T22:07:13.910156+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T00:24:47.311230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.908345+0000","last_change":"2024-05-17T22:07:12.898535+0000","last_active":"2024-05-17T22:07:13.908345+0000","last_peered":"2024-05-17T22:07:13.908345+0000","last_clean":"2024-05-17T22:07:13.908345+0000","last_became_active":"2024-05-17T22:07:12.898256+0000","last_became_peered":"2024-05-17T22:07:12.898256+0000","last_unstale":"2024-05-17T22:07:13.908345+0000","last_undegraded":"2024-05-17T22:07:13.908345+0000","last_fullsized":"2024-05-17T22:07:13.908345+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:33:19.602360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909867+0000","last_change":"2024-05-17T22:07:11.893788+0000","last_active":"2024-05-17T22:07:13.909867+0000","last_peered":"2024-05-17T22:07:13.909867+0000","last_clean":"2024-05-17T22:07:13.909867+0000","last_became_active":"2024-05-17T22:07:11.893319+0000","last_became_peered":"2024-05-17T22:07:11.893319+0000","last_unstale":"2024-05-17T22:07:13.909867+0000","last_undegraded":"2024-05-17T22:07:13.909867+0000","last_fullsized":"2024-05-17T22:07:13.909867+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:59:42.372644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906871+0000","last_change":"2024-05-17T22:07:12.881778+0000","last_active":"2024-05-17T22:07:13.906871+0000","last_peered":"2024-05-17T22:07:13.906871+0000","last_clean":"2024-05-17T22:07:13.906871+0000","last_became_active":"2024-05-17T22:07:12.881458+0000","last_became_peered":"2024-05-17T22:07:12.881458+0000","last_unstale":"2024-05-17T22:07:13.906871+0000","last_undegraded":"2024-05-17T22:07:13.906871+0000","last_fullsized":"2024-05-17T22:07:13.906871+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:21:06.682155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875863+0000","last_change":"2024-05-17T22:07:11.905950+0000","last_active":"2024-05-17T22:07:13.875863+0000","last_peered":"2024-05-17T22:07:13.875863+0000","last_clean":"2024-05-17T22:07:13.875863+0000","last_became_active":"2024-05-17T22:07:11.905082+0000","last_became_peered":"2024-05-17T22:07:11.905082+0000","last_unstale":"2024-05-17T22:07:13.875863+0000","last_undegraded":"2024-05-17T22:07:13.875863+0000","last_fullsized":"2024-05-17T22:07:13.875863+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:21:41.759245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909427+0000","last_change":"2024-05-17T22:07:12.892224+0000","last_active":"2024-05-17T22:07:13.909427+0000","last_peered":"2024-05-17T22:07:13.909427+0000","last_clean":"2024-05-17T22:07:13.909427+0000","last_became_active":"2024-05-17T22:07:12.891873+0000","last_became_peered":"2024-05-17T22:07:12.891873+0000","last_unstale":"2024-05-17T22:07:13.909427+0000","last_undegraded":"2024-05-17T22:07:13.909427+0000","last_fullsized":"2024-05-17T22:07:13.909427+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T23:26:41.192008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907225+0000","last_change":"2024-05-17T22:07:11.878579+0000","last_active":"2024-05-17T22:07:13.907225+0000","last_peered":"2024-05-17T22:07:13.907225+0000","last_clean":"2024-05-17T22:07:13.907225+0000","last_became_active":"2024-05-17T22:07:11.877736+0000","last_became_peered":"2024-05-17T22:07:11.877736+0000","last_unstale":"2024-05-17T22:07:13.907225+0000","last_undegraded":"2024-05-17T22:07:13.907225+0000","last_fullsized":"2024-05-17T22:07:13.907225+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:49.345469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906261+0000","last_change":"2024-05-17T22:07:12.894974+0000","last_active":"2024-05-17T22:07:13.906261+0000","last_peered":"2024-05-17T22:07:13.906261+0000","last_clean":"2024-05-17T22:07:13.906261+0000","last_became_active":"2024-05-17T22:07:12.894632+0000","last_became_peered":"2024-05-17T22:07:12.894632+0000","last_unstale":"2024-05-17T22:07:13.906261+0000","last_undegraded":"2024-05-17T22:07:13.906261+0000","last_fullsized":"2024-05-17T22:07:13.906261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T06:30:36.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.876057+0000","last_change":"2024-05-17T22:07:11.895327+0000","last_active":"2024-05-17T22:07:13.876057+0000","last_peered":"2024-05-17T22:07:13.876057+0000","last_clean":"2024-05-17T22:07:13.876057+0000","last_became_active":"2024-05-17T22:07:11.894888+0000","last_became_peered":"2024-05-17T22:07:11.894888+0000","last_unstale":"2024-05-17T22:07:13.876057+0000","last_undegraded":"2024-05-17T22:07:13.876057+0000","last_fullsized":"2024-05-17T22:07:13.876057+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T01:23:34.508676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907247+0000","last_change":"2024-05-17T22:07:12.890579+0000","last_active":"2024-05-17T22:07:13.907247+0000","last_peered":"2024-05-17T22:07:13.907247+0000","last_clean":"2024-05-17T22:07:13.907247+0000","last_became_active":"2024-05-17T22:07:12.890190+0000","last_became_peered":"2024-05-17T22:07:12.890190+0000","last_unstale":"2024-05-17T22:07:13.907247+0000","last_undegraded":"2024-05-17T22:07:13.907247+0000","last_fullsized":"2024-05-17T22:07:13.907247+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T09:48:15.449888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.907986+0000","last_change":"2024-05-17T22:07:11.907918+0000","last_active":"2024-05-17T22:07:13.907986+0000","last_peered":"2024-05-17T22:07:13.907986+0000","last_clean":"2024-05-17T22:07:13.907986+0000","last_became_active":"2024-05-17T22:07:11.907362+0000","last_became_peered":"2024-05-17T22:07:11.907362+0000","last_unstale":"2024-05-17T22:07:13.907986+0000","last_undegraded":"2024-05-17T22:07:13.907986+0000","last_fullsized":"2024-05-17T22:07:13.907986+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T05:47:05.479250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.909503+0000","last_change":"2024-05-17T22:07:12.902200+0000","last_active":"2024-05-17T22:07:13.909503+0000","last_peered":"2024-05-17T22:07:13.909503+0000","last_clean":"2024-05-17T22:07:13.909503+0000","last_became_active":"2024-05-17T22:07:12.901081+0000","last_became_peered":"2024-05-17T22:07:12.901081+0000","last_unstale":"2024-05-17T22:07:13.909503+0000","last_undegraded":"2024-05-17T22:07:13.909503+0000","last_fullsized":"2024-05-17T22:07:13.909503+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:11.850665+0000","last_clean_scrub_stamp":"2024-05-17T22:07:11.850665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:30:27.984446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.875849+0000","last_change":"2024-05-17T22:07:11.889062+0000","last_active":"2024-05-17T22:07:13.875849+0000","last_peered":"2024-05-17T22:07:13.875849+0000","last_clean":"2024-05-17T22:07:13.875849+0000","last_became_active":"2024-05-17T22:07:11.888152+0000","last_became_peered":"2024-05-17T22:07:11.888152+0000","last_unstale":"2024-05-17T22:07:13.875849+0000","last_undegraded":"2024-05-17T22:07:13.875849+0000","last_fullsized":"2024-05-17T22:07:13.875849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T22:07:10.845830+0000","last_clean_scrub_stamp":"2024-05-17T22:07:10.845830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T02:36:10.883158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":272,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-05-17T22:07:13.906407+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:07:13.906407+0000","last_peered":"2024-05-17T22:07:13.906407+0000","last_clean":"2024-05-17T22:07:13.906407+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:07:13.906407+0000","last_undegraded":"2024-05-17T22:07:13.906407+0000","last_fullsized":"2024-05-17T22:07:13.906407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1945600,"data_stored":1171682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":587068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":581302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1359872,"data_stored":582380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1343488,"data_stored":580698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":580822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1957888,"data_stored":1172790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:07:17.367 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:07:17.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-17T22:07:17.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.562+0000 7f853a987640 1 Processor -- start 2024-05-17T22:07:17.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.562+0000 7f853a987640 1 -- start start 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f85340cf830 0x7f85340b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f853415d700 0x7f85341663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85340b5640 con 0x7f85340cdf40 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8534070870 con 0x7f853415d700 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f85340b58e0 con 0x7f85340cf830 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f853415d700 0x7f85341663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f853415d700 0x7f85341663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34386/0 (socket says 172.21.15.83:34386) 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 -- 172.21.15.83:0/4280310296 learned_addr learned my addr 172.21.15.83:0/4280310296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8533fff640 1 --2- 172.21.15.83:0/4280310296 >> v2:172.21.15.155:3300/0 conn(0x7f85340cf830 0x7f85340b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 -- 172.21.15.83:0/4280310296 >> v2:172.21.15.155:3300/0 conn(0x7f85340cf830 msgr2=0x7f85340b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 --2- 172.21.15.83:0/4280310296 >> v2:172.21.15.155:3300/0 conn(0x7f85340cf830 0x7f85340b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 -- 172.21.15.83:0/4280310296 >> v1:172.21.15.155:6789/0 conn(0x7f85340cdf40 legacy=0x7f853415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 -- 172.21.15.83:0/4280310296 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853406f060 con 0x7f853415d700 2024-05-17T22:07:17.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8538f24640 1 --2- 172.21.15.83:0/4280310296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f85341663d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f853406d6e0 tx=0x7f852c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4e91b18345a926 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8532ffd640 1 -- 172.21.15.83:0/4280310296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a5710 con 0x7f853415d700 2024-05-17T22:07:17.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8532ffd640 1 -- 172.21.15.83:0/4280310296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0a58b0 con 0x7f853415d700 2024-05-17T22:07:17.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f8532ffd640 1 -- 172.21.15.83:0/4280310296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a5c80 con 0x7f853415d700 2024-05-17T22:07:17.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 -- 172.21.15.83:0/4280310296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 msgr2=0x7f85341663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.566+0000 7f853a987640 1 --2- 172.21.15.83:0/4280310296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f85341663d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f853406d6e0 tx=0x7f852c01c9f0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- 172.21.15.83:0/4280310296 shutdown_connections 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 --2- 172.21.15.83:0/4280310296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f85341663d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 --2- 172.21.15.83:0/4280310296 >> v2:172.21.15.155:3300/0 conn(0x7f85340cf830 0x7f85340b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- 172.21.15.83:0/4280310296 >> 172.21.15.83:0/4280310296 conn(0x7f853406cd90 msgr2=0x7f85340b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- 172.21.15.83:0/4280310296 shutdown_connections 2024-05-17T22:07:17.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- 172.21.15.83:0/4280310296 wait complete. 2024-05-17T22:07:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 Processor -- start 2024-05-17T22:07:17.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- start start 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 0x7f85341f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 0x7f853414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85341692b0 con 0x7f853415d700 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8534169130 con 0x7f85340cdf40 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f853a987640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8534169430 con 0x7f85340cf830 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34390/0 (socket says 172.21.15.83:34390) 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 -- 172.21.15.83:0/4229937659 learned_addr learned my addr 172.21.15.83:0/4229937659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8533fff640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 0x7f85341f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f85337fe640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 0x7f853414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 -- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 msgr2=0x7f853414dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 0x7f853414dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 -- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 msgr2=0x7f85341f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 0x7f85341f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 -- 172.21.15.83:0/4229937659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85341545c0 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8533fff640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 0x7f85341f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f85337fe640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 0x7f853414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.570+0000 7f8538f24640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f852c0af040 tx=0x7f852c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0b8670 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c0be070 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853413c150 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0b8a50 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853413aaf0 con 0x7f853415d700 2024-05-17T22:07:17.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85340b3aa0 con 0x7f853415d700 2024-05-17T22:07:17.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f852c0b8bf0 con 0x7f853415d700 2024-05-17T22:07:17.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 0x7f8520071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f852c12a790 con 0x7f853415d700 2024-05-17T22:07:17.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8533fff640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 0x7f8520071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.574+0000 7f8533fff640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 0x7f8520071940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f85240097a0 tx=0x7f8524008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.578+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f852c0c7020 con 0x7f853415d700 2024-05-17T22:07:17.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.782+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8534154350 con 0x7f853415d700 2024-05-17T22:07:17.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.786+0000 7f8530ff9640 1 -- 172.21.15.83:0/4229937659 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f852c0165e0 con 0x7f853415d700 2024-05-17T22:07:17.780 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-05-17T22:07:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCQ0kdmh3gUDRAAqPfSs9YlwtQw9Uj7S27AlA== 2024-05-17T22:07:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2024-05-17T22:07:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2024-05-17T22:07:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-17T22:07:17.781 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 msgr2=0x7f8520071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 0x7f8520071940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f85240097a0 tx=0x7f8524008040 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 msgr2=0x7f8534072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f852c0af040 tx=0x7f852c019790 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 shutdown_connections 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853415d700 0x7f8534072e80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85340cf830 0x7f853414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f852006f430 0x7f8520071940 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 --2- 172.21.15.83:0/4229937659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85340cdf40 0x7f85341f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 >> 172.21.15.83:0/4229937659 conn(0x7f853406cd90 msgr2=0x7f853415cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 shutdown_connections 2024-05-17T22:07:17.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.790+0000 7f853a987640 1 -- 172.21.15.83:0/4229937659 wait complete. 2024-05-17T22:07:17.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 Processor -- start 2024-05-17T22:07:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 -- start start 2024-05-17T22:07:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f757c06ffd0 0x7f757c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c1465d0 con 0x7f757c1605a0 2024-05-17T22:07:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c070920 con 0x7f757c1461c0 2024-05-17T22:07:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7584782640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f757c06ef30 con 0x7f757c06ffd0 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7581d1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f757c06ffd0 0x7f757c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7582d1f640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f757c1605a0 0x7f757c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34344/0 (socket says 172.21.15.83:34344) 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7582d1f640 1 -- 172.21.15.83:0/2832056409 learned_addr learned my addr 172.21.15.83:0/2832056409 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7581d1d640 1 -- 172.21.15.83:0/2832056409 >> v1:172.21.15.155:6789/0 conn(0x7f757c1461c0 legacy=0x7f757c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7581d1d640 1 -- 172.21.15.83:0/2832056409 >> v1:172.21.15.83:6789/0 conn(0x7f757c1605a0 legacy=0x7f757c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f758151c640 1 -- 172.21.15.83:0/2832056409 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849421786 0 0) 0x7f757c1465d0 con 0x7f757c1605a0 2024-05-17T22:07:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.942+0000 7f7581d1d640 1 -- 172.21.15.83:0/2832056409 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c06f210 con 0x7f757c06ffd0 2024-05-17T22:07:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7581d1d640 1 --2- 172.21.15.83:0/2832056409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c06ffd0 0x7f757c0703e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201534e12031103f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f758151c640 1 -- 172.21.15.83:0/2832056409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0a7070 con 0x7f757c06ffd0 2024-05-17T22:07:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f758151c640 1 -- 172.21.15.83:0/2832056409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c0b0070 con 0x7f757c06ffd0 2024-05-17T22:07:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f758151c640 1 -- 172.21.15.83:0/2832056409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0ab890 con 0x7f757c06ffd0 2024-05-17T22:07:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- 172.21.15.83:0/2832056409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c06ffd0 msgr2=0x7f757c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 --2- 172.21.15.83:0/2832056409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c06ffd0 0x7f757c0703e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c470 comp rx=0 tx=0).stop 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- 172.21.15.83:0/2832056409 shutdown_connections 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 --2- 172.21.15.83:0/2832056409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c06ffd0 0x7f757c0703e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- 172.21.15.83:0/2832056409 >> 172.21.15.83:0/2832056409 conn(0x7f757c06cd90 msgr2=0x7f757c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- 172.21.15.83:0/2832056409 shutdown_connections 2024-05-17T22:07:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- 172.21.15.83:0/2832056409 wait complete. 2024-05-17T22:07:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 Processor -- start 2024-05-17T22:07:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.946+0000 7f7584782640 1 -- start start 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f757c06ffd0 0x7f757c0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f757c1605a0 0x7f757c140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f757c0d10a0 con 0x7f757c1461c0 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f757c15e880 con 0x7f757c1605a0 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f757c0d1220 con 0x7f757c06ffd0 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34418/0 (socket says 172.21.15.83:34418) 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 -- 172.21.15.83:0/3905154537 learned_addr learned my addr 172.21.15.83:0/3905154537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 -- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f757c06ffd0 msgr2=0x7f757c0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f757c06ffd0 0x7f757c0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 -- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f757c1605a0 msgr2=0x7f757c140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7582d1f640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f757c1605a0 0x7f757c140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f757c1605a0 0x7f757c140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 -- 172.21.15.83:0/3905154537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c0afd50 con 0x7f757c1461c0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7581d1d640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f756c0ad040 tx=0x7f756c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0ab940 con 0x7f757c1461c0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c0abb00 con 0x7f757c1461c0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c0ae080 con 0x7f757c1461c0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0bf3f0 con 0x7f757c1461c0 2024-05-17T22:07:17.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c0ae640 con 0x7f757c1461c0 2024-05-17T22:07:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c0703e0 con 0x7f757c1461c0 2024-05-17T22:07:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f756c0c6060 con 0x7f757c1461c0 2024-05-17T22:07:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 0x7f7560071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f756c12b850 con 0x7f757c1461c0 2024-05-17T22:07:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.950+0000 7f758251e640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 0x7f7560071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.954+0000 7f758251e640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 0x7f7560071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b11f0 tx=0x7f7578006bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:17.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:17.954+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f756c0ba070 con 0x7f757c1461c0 2024-05-17T22:07:18.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f757c0b41a0 con 0x7f757c1461c0 2024-05-17T22:07:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7572ffd640 1 -- 172.21.15.83:0/3905154537 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5180 (secure 0 0 0) 0x7f756c016090 con 0x7f757c1461c0 2024-05-17T22:07:18.132 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:18.132 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7980,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:15.298736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"d","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6833/1193425174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1193425174},{"type":"v1","addr":"172.21.15.155:6833","nonce":1193425174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7992,"qdb_cluster":[7992]},"id":8}]} 2024-05-17T22:07:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 97 2024-05-17T22:07:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 msgr2=0x7f7560071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 0x7f7560071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f757c0b11f0 tx=0x7f7578006bb0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 msgr2=0x7f757c1f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f756c0ad040 tx=0x7f756c01acb0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 shutdown_connections 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f400 0x7f7560071910 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f757c1605a0 0x7f757c140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f757c1461c0 0x7f757c1f5870 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 --2- 172.21.15.83:0/3905154537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f757c06ffd0 0x7f757c0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 >> 172.21.15.83:0/3905154537 conn(0x7f757c06cd90 msgr2=0x7f757c0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 shutdown_connections 2024-05-17T22:07:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.138+0000 7f7584782640 1 -- 172.21.15.83:0/3905154537 wait complete. 2024-05-17T22:07:18.451 INFO:tasks.ceph.mds.d.smithi155.stderr:2024-05-17T22:07:18.459+0000 7fcd5bb58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:07:18.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:07:15.298736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7992}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7992': {'gid': 7992, 'name': 'd', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6833/1193425174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1193425174}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 1193425174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7992, 'qdb_cluster': [7992]} 2024-05-17T22:07:18.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:07:18.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-17T22:07:18.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 Processor -- start 2024-05-17T22:07:18.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 -- start start 2024-05-17T22:07:18.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12540b4810 0x7f12540d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540d4170 con 0x7f12540d0290 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540d4450 con 0x7f12540d2000 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f125a3ee640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f12540d4730 con 0x7f12540b4810 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f12537fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f12540d0290 0x7f125415fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34356/0 (socket says 172.21.15.83:34356) 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f12537fe640 1 -- 172.21.15.83:0/3575782293 learned_addr learned my addr 172.21.15.83:0/3575782293 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f1253fff640 1 --2- 172.21.15.83:0/3575782293 >> v2:172.21.15.155:3300/0 conn(0x7f12540b4810 0x7f12540d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511641254 0 0) 0x7f12540d4170 con 0x7f12540d0290 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.722+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f123c003530 con 0x7f12540d0290 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3379440750 0 0) 0x7f123c003530 con 0x7f12540d0290 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 >> v2:172.21.15.155:3300/0 conn(0x7f12540b4810 msgr2=0x7f12540d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 --2- 172.21.15.83:0/3575782293 >> v2:172.21.15.155:3300/0 conn(0x7f12540b4810 0x7f12540d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 >> v1:172.21.15.155:6789/0 conn(0x7f12540d2000 legacy=0x7f12540d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540d4a10 con 0x7f12540d0290 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1248001700 con 0x7f12540d0290 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 >> v1:172.21.15.83:6789/0 conn(0x7f12540d0290 legacy=0x7f125415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 0x7f123c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c004810 0x7f123c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123c005260 0x7f123c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12540d4730 con 0x7f123c003a60 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f123c00c1f0 con 0x7f123c004810 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f123c00c3f0 con 0x7f123c005260 2024-05-17T22:07:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 0x7f123c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 -- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123c005260 msgr2=0x7f123c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f12537fe640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c004810 0x7f123c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125898b640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123c005260 0x7f123c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123c005260 0x7f123c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 -- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c004810 msgr2=0x7f123c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c004810 0x7f123c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 -- 172.21.15.83:0/3575782293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c003530 con 0x7f123c003a60 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125898b640 1 -- 172.21.15.83:0/3575782293 reap_dead start 2024-05-17T22:07:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f12537fe640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c004810 0x7f123c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1253fff640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 0x7f123c003ed0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f124c009fc0 tx=0x7f124c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2c18772146a2bb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b2070 con 0x7f123c003a60 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124c0a5d20 con 0x7f123c003a60 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f1252ffd640 1 -- 172.21.15.83:0/3575782293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c009c10 con 0x7f123c003a60 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 -- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 msgr2=0x7f123c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 0x7f123c003ed0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f124c009fc0 tx=0x7f124c01cc80 comp rx=0 tx=0).stop 2024-05-17T22:07:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 -- 172.21.15.83:0/3575782293 shutdown_connections 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 --2- 172.21.15.83:0/3575782293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123c003a60 0x7f123c003ed0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 -- 172.21.15.83:0/3575782293 >> 172.21.15.83:0/3575782293 conn(0x7f125406e010 msgr2=0x7f125406e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 -- 172.21.15.83:0/3575782293 shutdown_connections 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.726+0000 7f125a3ee640 1 -- 172.21.15.83:0/3575782293 wait complete. 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 Processor -- start 2024-05-17T22:07:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 -- start start 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 0x7f12541ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 0x7f12541fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12540d2590 con 0x7f125414d340 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12540d2410 con 0x7f123c003a60 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125a3ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f12540d2710 con 0x7f12541fb710 2024-05-17T22:07:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34428/0 (socket says 172.21.15.83:34428) 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 -- 172.21.15.83:0/49575081 learned_addr learned my addr 172.21.15.83:0/49575081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f1253fff640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 0x7f12541ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f12537fe640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 0x7f12541fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 -- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 msgr2=0x7f12541fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 0x7f12541fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 -- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 msgr2=0x7f12541ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 0x7f12541ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 -- 172.21.15.83:0/49575081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12541ec8f0 con 0x7f125414d340 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f12537fe640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 0x7f12541fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f1253fff640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 0x7f12541ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f125898b640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f12540b8830 tx=0x7f124400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f1250ff9640 1 -- 172.21.15.83:0/49575081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244018070 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f1250ff9640 1 -- 172.21.15.83:0/49575081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1244003600 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.730+0000 7f1250ff9640 1 -- 172.21.15.83:0/49575081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440039f0 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12541ecbd0 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541ed190 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1250ff9640 1 -- 172.21.15.83:0/49575081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f124401b070 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1250ff9640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 0x7f1240071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1250ff9640 1 -- 172.21.15.83:0/49575081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f124408e920 con 0x7f125414d340 2024-05-17T22:07:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1253fff640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 0x7f1240071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1253fff640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 0x7f1240071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f124c0af000 tx=0x7f124c0a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1261668000 1 Processor -- start 2024-05-17T22:07:18.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.734+0000 7f1261668000 1 -- start start 2024-05-17T22:07:18.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd72fa0 0x56271feb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271feb8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56271fd56ca0 con 0x56271fd51d30 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56271fcc9110 con 0x56271fd68600 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56271fcb8cb0 con 0x56271fd72fa0 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34440/0 (socket says 172.21.15.83:34440) 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 -- 172.21.15.83:0/1039285139 learned_addr learned my addr 172.21.15.83:0/1039285139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1253fff640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd72fa0 0x56271feb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f12537fe640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271feb8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 -- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd72fa0 msgr2=0x56271feb88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd72fa0 0x56271feb88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 -- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 msgr2=0x56271feb8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271feb8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 -- 172.21.15.83:0/1039285139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271fc7d370 con 0x56271fd51d30 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f12537fe640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271feb8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f125898b640 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f124400e010 tx=0x7f124405a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533944175c671142 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1239ffb640 1 -- 172.21.15.83:0/1039285139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124405b090 con 0x56271fd51d30 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1239ffb640 1 -- 172.21.15.83:0/1039285139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124405f440 con 0x56271fd51d30 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1239ffb640 1 -- 172.21.15.83:0/1039285139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244063ef0 con 0x56271fd51d30 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- 172.21.15.83:0/1039285139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 msgr2=0x56271feb9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f124400e010 tx=0x7f124405a800 comp rx=0 tx=0).stop 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- 172.21.15.83:0/1039285139 shutdown_connections 2024-05-17T22:07:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271feb9330 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271feb8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 --2- 172.21.15.83:0/1039285139 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd72fa0 0x56271feb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- 172.21.15.83:0/1039285139 >> 172.21.15.83:0/1039285139 conn(0x56271f97a460 msgr2=0x56271fe6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- 172.21.15.83:0/1039285139 shutdown_connections 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- 172.21.15.83:0/1039285139 wait complete. 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 -- start start 2024-05-17T22:07:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.738+0000 7f1261668000 1 Processor -- start 2024-05-17T22:07:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 0x56271fe75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 0x56271fe7a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56271fc74d40 con 0x56271fd51d30 2024-05-17T22:07:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56271fcbccf0 con 0x56271fd72fa0 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56271fc61e90 con 0x56271fd68600 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 10 client.0 _ll_register_callbacks cb 0x56271fc96fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34450/0 (socket says 172.21.15.83:34450) 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 -- 172.21.15.83:0/3844533841 learned_addr learned my addr 172.21.15.83:0/3844533841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f125898b640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 0x56271fe7a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f12537fe640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 0x56271fe75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 -- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 msgr2=0x56271fe75810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 0x56271fe75810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 -- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 msgr2=0x56271fe7a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 0x56271fe7a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 -- 172.21.15.83:0/3844533841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271fc876e0 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f125898b640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 0x56271fe7a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f12537fe640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 0x56271fe75810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1253fff640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f124c01cab0 tx=0x7f123c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca1a7e3bfa8092e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0b2040 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f124c0c5440 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124405f440 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f124c0c5640 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 10 client.8118 fetch_fsmap learned FSMap version 97 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x56271fc77690 con 0x56271fd51d30 2024-05-17T22:07:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124405f440 con 0x56271fd51d30 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 97) ==== 3494+0+0 (secure 0 0 0) 0x7f124c0c57e0 con 0x56271fd51d30 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 10 client.8118 fetch_fsmap finished waiting for FSMap version 97 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 10 client.8118 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 20 client.8118 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f95f7d0 0x56271febf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 4 client.8118 mds_command: new command op to 7992 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 --> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56271fc54a00 con 0x56271f95f7d0 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f12537fe640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f95f7d0 0x56271febf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f1232ffd640 10 client.8118 ms_handle_connect on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.742+0000 7f12537fe640 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f95f7d0 0x56271febf920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ab192aaf56a263 server_cookie=4d0639bef35ac3d1 in_seq=0 out_seq=0 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1232ffd640 1 -- 172.21.15.83:0/3844533841 <== mds.0 v2:172.21.15.155:6832/1193425174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56271fc54a00 con 0x56271f95f7d0 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1232ffd640 10 client.8118 handle_command_reply: tid=0 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 client.8118 shutdown 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 msgr2=0x56271fe752d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f124c01cab0 tx=0x7f123c005840 comp rx=0 tx=0).stop 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 shutdown_connections 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f95f7d0 0x56271febf920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd72fa0 0x56271fe7a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271fd68600 0x56271fe75810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1232ffd640 0 client.8118 ms_handle_reset on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 --2- 172.21.15.83:0/3844533841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271fe752d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 >> 172.21.15.83:0/3844533841 conn(0x56271f97a460 msgr2=0x56271fe6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 shutdown_connections 2024-05-17T22:07:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 1 -- 172.21.15.83:0/3844533841 wait complete. 2024-05-17T22:07:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.746+0000 7f1261668000 20 client.8118 trim_cache size 0 max 16384 2024-05-17T22:07:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.762+0000 7f1261668000 1 Processor -- start 2024-05-17T22:07:18.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.762+0000 7f1261668000 1 -- start start 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271ff27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1254204da0 con 0x56271fd51d30 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1254204180 con 0x56271fd68600 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1254204750 con 0x56271f95f7d0 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34456/0 (socket says 172.21.15.83:34456) 2024-05-17T22:07:18.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 -- 172.21.15.83:0/1280098590 learned_addr learned my addr 172.21.15.83:0/1280098590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 -- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 msgr2=0x56271ff26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1253fff640 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff26bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f125898b640 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271ff27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 -- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 msgr2=0x56271ff27670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271ff27670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 -- 172.21.15.83:0/1280098590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56271fd3fcb0 con 0x56271fd51d30 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12537fe640 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1248005cb0 tx=0x7f1248009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178674ac1a921780 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12397fa640 1 -- 172.21.15.83:0/1280098590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124801d070 con 0x56271fd51d30 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12397fa640 1 -- 172.21.15.83:0/1280098590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1248005da0 con 0x56271fd51d30 2024-05-17T22:07:18.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f12397fa640 1 -- 172.21.15.83:0/1280098590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248018640 con 0x56271fd51d30 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- 172.21.15.83:0/1280098590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 msgr2=0x56271ff27130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1248005cb0 tx=0x7f1248009040 comp rx=0 tx=0).stop 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- 172.21.15.83:0/1280098590 shutdown_connections 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd68600 0x56271ff27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd51d30 0x56271ff27130 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 --2- 172.21.15.83:0/1280098590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- 172.21.15.83:0/1280098590 >> 172.21.15.83:0/1280098590 conn(0x56271f97a460 msgr2=0x56271feceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- 172.21.15.83:0/1280098590 shutdown_connections 2024-05-17T22:07:18.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- 172.21.15.83:0/1280098590 wait complete. 2024-05-17T22:07:18.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 -- start start 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.766+0000 7f1261668000 1 Processor -- start 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff2ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd51d30 0x56271ff2b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56271fc63900 con 0x56271fd68600 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56271fe78c20 con 0x56271fd51d30 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x56271fc87c10 con 0x56271f95f7d0 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 10 client.0 _ll_register_callbacks cb 0x7f123c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34460/0 (socket says 172.21.15.83:34460) 2024-05-17T22:07:18.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 -- 172.21.15.83:0/1632798667 learned_addr learned my addr 172.21.15.83:0/1632798667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1253fff640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff2ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f12537fe640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd51d30 0x56271ff2b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 -- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 msgr2=0x56271ff2ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff2ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 -- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd51d30 msgr2=0x56271ff2b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd51d30 0x56271ff2b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 -- 172.21.15.83:0/1632798667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56271fc784f0 con 0x56271fd68600 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f125898b640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f124408cea0 tx=0x7f1244015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8ce6b8e4cf32b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124405f440 con 0x56271fd68600 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f1244003b90 con 0x56271fd68600 2024-05-17T22:07:18.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1248005da0 con 0x56271fd68600 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f124408a030 con 0x56271fd68600 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 10 client.8124 fetch_fsmap learned FSMap version 97 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x56271fc824f0 con 0x56271fd68600 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1248005da0 con 0x56271fd68600 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 97) ==== 3494+0+0 (secure 0 0 0) 0x7f124405a2f0 con 0x56271fd68600 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 10 client.8124 fetch_fsmap finished waiting for FSMap version 97 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 10 client.8124 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 20 client.8124 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f978e80 0x56271fe6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 4 client.8124 mds_command: new command op to 7992 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 --> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] -- command(tid 0: {"prefix": "status"}) -- 0x56271fccb060 con 0x56271f978e80 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1253fff640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f978e80 0x56271fe6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1232ffd640 10 client.8124 ms_handle_connect on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.770+0000 7f1253fff640 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f978e80 0x56271fe6ff10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a67920d01b417a server_cookie=5c3ed2d0d00b48a2 in_seq=0 out_seq=0 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1232ffd640 1 -- 172.21.15.83:0/1632798667 <== mds.0 v2:172.21.15.155:6832/1193425174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x56271fccb060 con 0x56271f978e80 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1232ffd640 10 client.8124 handle_command_reply: tid=0 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 client.8124 shutdown 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 msgr2=0x56271ff2b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f124408cea0 tx=0x7f1244015040 comp rx=0 tx=0).stop 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 shutdown_connections 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x56271f978e80 0x56271fe6ff10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56271fd68600 0x56271ff2b8c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56271fd51d30 0x56271ff2b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1232ffd640 0 client.8124 ms_handle_reset on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 --2- 172.21.15.83:0/1632798667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56271f95f7d0 0x56271ff2ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 >> 172.21.15.83:0/1632798667 conn(0x56271f97a460 msgr2=0x56271fecf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 shutdown_connections 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 1 -- 172.21.15.83:0/1632798667 wait complete. 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f1261668000 20 client.8124 trim_cache size 0 max 16384 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:18.772 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7992, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.908281831, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 97, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 109, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 109, 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.184661988 2024-05-17T22:07:18.773 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:18.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 msgr2=0x7f1240071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 0x7f1240071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f124c0af000 tx=0x7f124c0a7040 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 msgr2=0x7f125414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f12540b8830 tx=0x7f124400d890 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 shutdown_connections 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f125414d340 0x7f125414d750 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12541fb710 0x7f12541fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f124006f400 0x7f1240071910 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 --2- 172.21.15.83:0/49575081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f123c003a60 0x7f12541ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 >> 172.21.15.83:0/49575081 conn(0x7f125406e010 msgr2=0x7f12540d5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 shutdown_connections 2024-05-17T22:07:18.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.774+0000 7f125a3ee640 1 -- 172.21.15.83:0/49575081 wait complete. 2024-05-17T22:07:18.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:07:18.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 Processor -- start 2024-05-17T22:07:18.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 -- start start 2024-05-17T22:07:18.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e415ffa0 0x7fc3e40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e406e3d0 0x7fc3e40d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e40d34e0 con 0x7fc3e40d1150 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e40d3780 con 0x7fc3e406e3d0 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ec5a9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e40d3a60 con 0x7fc3e415ffa0 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e406e3d0 0x7fc3e40d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3e406e3d0 0x7fc3e40d4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34476/0 (socket says 172.21.15.83:34476) 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 -- 172.21.15.83:0/844051402 learned_addr learned my addr 172.21.15.83:0/844051402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3ea345640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e415ffa0 0x7fc3e40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 -- 172.21.15.83:0/844051402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e415ffa0 msgr2=0x7fc3e40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e415ffa0 0x7fc3e40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 -- 172.21.15.83:0/844051402 >> v1:172.21.15.155:6789/0 conn(0x7fc3e40d1150 legacy=0x7fc3e406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.934+0000 7fc3eab46640 1 -- 172.21.15.83:0/844051402 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e40d3d40 con 0x7fc3e406e3d0 2024-05-17T22:07:18.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3eab46640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e406e3d0 0x7fc3e40d4260 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009e50 tx=0x7fc3dc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c2e72b5ca4899f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3e9343640 1 -- 172.21.15.83:0/844051402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0a5700 con 0x7fc3e406e3d0 2024-05-17T22:07:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3e9343640 1 -- 172.21.15.83:0/844051402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0a58a0 con 0x7fc3e406e3d0 2024-05-17T22:07:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3e9343640 1 -- 172.21.15.83:0/844051402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0a5c70 con 0x7fc3e406e3d0 2024-05-17T22:07:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/844051402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e406e3d0 msgr2=0x7fc3e40d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e406e3d0 0x7fc3e40d4260 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009e50 tx=0x7fc3dc01ca30 comp rx=0 tx=0).stop 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/844051402 shutdown_connections 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e406e3d0 0x7fc3e40d4260 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/844051402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e415ffa0 0x7fc3e40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/844051402 >> 172.21.15.83:0/844051402 conn(0x7fc3e40d28f0 msgr2=0x7fc3e40d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/844051402 shutdown_connections 2024-05-17T22:07:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.938+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/844051402 wait complete. 2024-05-17T22:07:18.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 Processor -- start 2024-05-17T22:07:18.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- start start 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40d1150 0x7fc3e4155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e415ffa0 0x7fc3e4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40d5340 con 0x7fc3e4161950 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40d51c0 con 0x7fc3e415ffa0 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e40d54c0 con 0x7fc3e40d1150 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34486/0 (socket says 172.21.15.83:34486) 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 -- 172.21.15.83:0/3869280613 learned_addr learned my addr 172.21.15.83:0/3869280613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ea345640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40d1150 0x7fc3e4155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3e9b44640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e415ffa0 0x7fc3e4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 -- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40d1150 msgr2=0x7fc3e4155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40d1150 0x7fc3e4155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 -- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e415ffa0 msgr2=0x7fc3e4155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e415ffa0 0x7fc3e4155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 -- 172.21.15.83:0/3869280613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e41620c0 con 0x7fc3e4161950 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3eab46640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc01a660 tx=0x7fc3dc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0b8670 con 0x7fc3e4161950 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0be070 con 0x7fc3e4161950 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e41623a0 con 0x7fc3e4161950 2024-05-17T22:07:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0b8a30 con 0x7fc3e4161950 2024-05-17T22:07:18.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e4162960 con 0x7fc3e4161950 2024-05-17T22:07:18.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.942+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e40d4260 con 0x7fc3e4161950 2024-05-17T22:07:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.946+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc3dc0b8bd0 con 0x7fc3e4161950 2024-05-17T22:07:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.946+0000 7fc3e2ffd640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 0x7fc3d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.946+0000 7fc3ea345640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 0x7fc3d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.946+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fc3dc12a280 con 0x7fc3e4161950 2024-05-17T22:07:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.946+0000 7fc3ea345640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 0x7fc3d0071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400b930 tx=0x7fc3d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:18.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:18.950+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc3dc016620 con 0x7fc3e4161950 2024-05-17T22:07:19.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.134+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc3e40cc6c0 con 0x7fc3e4161950 2024-05-17T22:07:19.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.134+0000 7fc3e2ffd640 1 -- 172.21.15.83:0/3869280613 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7fc3dc0ff070 con 0x7fc3e4161950 2024-05-17T22:07:19.129 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:07:19.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 msgr2=0x7fc3d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 0x7fc3d0071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400b930 tx=0x7fc3d4008040 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 msgr2=0x7fc3e415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc01a660 tx=0x7fc3dc0197d0 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 shutdown_connections 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e4161950 0x7fc3e415a260 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3d006f400 0x7fc3d0071910 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e415ffa0 0x7fc3e4155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 --2- 172.21.15.83:0/3869280613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40d1150 0x7fc3e4155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 >> 172.21.15.83:0/3869280613 conn(0x7fc3e40d28f0 msgr2=0x7fc3e406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 shutdown_connections 2024-05-17T22:07:19.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19.138+0000 7fc3ec5a9640 1 -- 172.21.15.83:0/3869280613 wait complete. 2024-05-17T22:07:19.150 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:07:19.151 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:19.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.151 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:07:19.157 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:07:19.157 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:07:19.157 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:07:19.157 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:07:19.158 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41905sec preferred_lft 41905sec 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.159 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:19.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.160 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:07:19.206 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:07:19.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.208 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:07:19.258 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:19.259 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:07:19.260 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:07:19.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:07:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:19.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.380 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:07:19.427 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:07:19.427 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:07:19.427 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:19.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:07:19.486 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:07:19.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:19.486 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:19.535 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:19.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:19.535 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:19.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:19.640 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:19.687 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:19.688 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:19.689 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:19.690 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:19.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.691 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:19.739 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:07:19.739 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:07:19.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:19.799 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:19.847 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:19.847 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:19.847 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:19.847 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:19.847 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:19.848 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:19.849 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:19.850 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:19.850 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:19.850 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:19.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:19.852 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:19.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:07:19.898+0000 7fea99ff5580 -1 init, newargv = 0x7fea88000ee0 newargc=15 2024-05-17T22:07:19.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48519]: starting ceph client 2024-05-17T22:07:19.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48519]: starting fuse 2024-05-17T22:07:20.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:21.013 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:21.061 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:21.061 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:21.061 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:21.061 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:21.061 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:21.062 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:21.063 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:21.064 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:21.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:21.064 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:21.111 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:07:21.111 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:07:21.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:07:21.111 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:07:21.111 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:07:21.112 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:07:21.113 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:21.237 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.48519.asok 2024-05-17T22:07:21.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:21.248 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48519.asok 2024-05-17T22:07:21.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:21.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48519.asok status 2024-05-17T22:07:21.426 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:07:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:07:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:07:21.426 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "48519", 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8136, 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8136 2024-05-17T22:07:21.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1638665173 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1638665173 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8136 192.168.0.1:0/1638665173", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1638665173", 2024-05-17T22:07:21.428 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 97, 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 109, 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:07:21.429 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:21.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:21.442 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:07:21.442 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:21.449 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:07:21.449 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:21.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:21.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:21.501 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> parent = os.path.join(mount_point, "parent") 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> os.mkdir(parent) 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> flushed = os.path.join(parent, "flushed") 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> os.mkdir(flushed) 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> for f in ["alpha", "bravo", "charlie"]: 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-17T22:07:21.502 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:21.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:21.643 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./parent 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/charlie 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/alpha 2024-05-17T22:07:21.718 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/bravo 2024-05-17T22:07:21.719 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:21.719 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:21.719 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:21.719 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.719 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-17T22:07:21.720 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:21.829 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:07:21.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-17T22:07:21.839 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:21.945 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:07:21.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:21.955 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:21.956 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:21.956 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:21.956 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:21.956 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-17T22:07:21.956 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:22.062 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T22:07:22.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:22.071 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:22.071 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:22.071 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:22.071 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:22.071 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-17T22:07:22.072 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:22.176 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628280 2024-05-17T22:07:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-17T22:07:22.186 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:22.294 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2024-05-17T22:07:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-17T22:07:22.306 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:22.410 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628279 2024-05-17T22:07:22.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:22.419 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:22.419 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:22.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:22.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 Processor -- start 2024-05-17T22:07:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 -- start start 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff0200d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff02009ba00 0x7ff02009be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02009c350 con 0x7ff0200ce130 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff0200d1a40 con 0x7ff02009ba00 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff026699640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff02009d040 con 0x7ff0200b88b0 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff02009ba00 0x7ff02009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff02009ba00 0x7ff02009be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34490/0 (socket says 172.21.15.83:34490) 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 -- 172.21.15.83:0/3861184579 learned_addr learned my addr 172.21.15.83:0/3861184579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01ffff640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff0200d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 -- 172.21.15.83:0/3861184579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 msgr2=0x7ff0200d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff0200d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 -- 172.21.15.83:0/3861184579 >> v1:172.21.15.155:6789/0 conn(0x7ff0200ce130 legacy=0x7ff0200ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01f7fe640 1 -- 172.21.15.83:0/3861184579 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02009c710 con 0x7ff02009ba00 2024-05-17T22:07:22.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.674+0000 7ff01ffff640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff0200d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff01f7fe640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff02009ba00 0x7ff02009be10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff0140098a0 tx=0x7ff01401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e19a79cd463330 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff01effd640 1 -- 172.21.15.83:0/3861184579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0140b0070 con 0x7ff02009ba00 2024-05-17T22:07:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff01effd640 1 -- 172.21.15.83:0/3861184579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0140ab600 con 0x7ff02009ba00 2024-05-17T22:07:22.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff01effd640 1 -- 172.21.15.83:0/3861184579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0140ab9d0 con 0x7ff02009ba00 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- 172.21.15.83:0/3861184579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff02009ba00 msgr2=0x7ff02009be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff02009ba00 0x7ff02009be10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff0140098a0 tx=0x7ff01401c240 comp rx=0 tx=0).stop 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- 172.21.15.83:0/3861184579 shutdown_connections 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff02009ba00 0x7ff02009be10 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 --2- 172.21.15.83:0/3861184579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff0200d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- 172.21.15.83:0/3861184579 >> 172.21.15.83:0/3861184579 conn(0x7ff0200d3620 msgr2=0x7ff0200d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:22.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- 172.21.15.83:0/3861184579 shutdown_connections 2024-05-17T22:07:22.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- 172.21.15.83:0/3861184579 wait complete. 2024-05-17T22:07:22.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 Processor -- start 2024-05-17T22:07:22.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.678+0000 7ff026699640 1 -- start start 2024-05-17T22:07:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff02009ba00 0x7ff0200cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff020154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0200b2dc0 con 0x7ff0200ce130 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff0200cf060 con 0x7ff02009ba00 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff0200b2f40 con 0x7ff0200b88b0 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34506/0 (socket says 172.21.15.83:34506) 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 -- 172.21.15.83:0/1198575447 learned_addr learned my addr 172.21.15.83:0/1198575447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff01f7fe640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff020154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff01ffff640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff02009ba00 0x7ff0200cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 -- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 msgr2=0x7ff020154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff020154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 -- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff02009ba00 msgr2=0x7ff0200cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff02009ba00 0x7ff0200cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 -- 172.21.15.83:0/1198575447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0200cb9c0 con 0x7ff0200ce130 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff024c36640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff01800b9e0 tx=0x7ff01800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018003b80 con 0x7ff0200ce130 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff018003d20 con 0x7ff0200ce130 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0200cbca0 con 0x7ff0200ce130 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800cb60 con 0x7ff0200ce130 2024-05-17T22:07:22.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0200cc260 con 0x7ff0200ce130 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.682+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0200b6fb0 con 0x7ff0200ce130 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.686+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff018006d10 con 0x7ff0200ce130 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.686+0000 7ff01cff9640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 0x7ff00c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.686+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7ff01808f5b0 con 0x7ff0200ce130 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.686+0000 7ff01ffff640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 0x7ff00c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:22.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.686+0000 7ff01ffff640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 0x7ff00c071910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff010006fd0 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.690+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff01805c580 con 0x7ff0200ce130 2024-05-17T22:07:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.858+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0200d21f0 con 0x7ff0200ce130 2024-05-17T22:07:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff01cff9640 1 -- 172.21.15.83:0/1198575447 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7ff01805be80 con 0x7ff0200ce130 2024-05-17T22:07:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:07:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 msgr2=0x7ff00c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 0x7ff00c071910 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff010006fd0 tx=0x7ff010008040 comp rx=0 tx=0).stop 2024-05-17T22:07:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 msgr2=0x7ff0200b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff01800b9e0 tx=0x7ff01800bea0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 shutdown_connections 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0200ce130 0x7ff0200b14a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0200b88b0 0x7ff020154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff00c06f400 0x7ff00c071910 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 --2- 172.21.15.83:0/1198575447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff02009ba00 0x7ff0200cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 >> 172.21.15.83:0/1198575447 conn(0x7ff0200d3620 msgr2=0x7ff0200ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 shutdown_connections 2024-05-17T22:07:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:22.862+0000 7ff026699640 1 -- 172.21.15.83:0/1198575447 wait complete. 2024-05-17T22:07:22.876 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:22.876 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:22.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 Processor -- start 2024-05-17T22:07:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 -- start start 2024-05-17T22:07:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 0x7f8f500d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f500d1210 con 0x7f8f501600f0 2024-05-17T22:07:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8f50160c10 con 0x7f8f500d0e00 2024-05-17T22:07:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.082+0000 7f8f556da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f501645d0 con 0x7f8f500d2050 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 0x7f8f500d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4effd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4effd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.83:58254/0 (socket says 172.21.15.83:58254) 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4effd640 1 -- 172.21.15.83:0/1387668367 learned_addr learned my addr 172.21.15.83:0/1387668367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/1387668367 >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 msgr2=0x7f8f5015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 --2- 172.21.15.83:0/1387668367 >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/1387668367 >> v1:172.21.15.155:6789/0 conn(0x7f8f501600f0 legacy=0x7f8f50160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/1387668367 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f50160e50 con 0x7f8f500d2050 2024-05-17T22:07:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4effd640 1 --2- 172.21.15.83:0/1387668367 >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:07:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.086+0000 7f8f4e7fc640 1 --2- 172.21.15.83:0/1387668367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 0x7f8f500d2460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009d90 tx=0x7f8f4401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb8187ebfc3359f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f4dffb640 1 -- 172.21.15.83:0/1387668367 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440b2070 con 0x7f8f500d2050 2024-05-17T22:07:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f4dffb640 1 -- 172.21.15.83:0/1387668367 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f440a5830 con 0x7f8f500d2050 2024-05-17T22:07:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f4dffb640 1 -- 172.21.15.83:0/1387668367 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440a5c00 con 0x7f8f500d2050 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- 172.21.15.83:0/1387668367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 msgr2=0x7f8f500d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 --2- 172.21.15.83:0/1387668367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 0x7f8f500d2460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009d90 tx=0x7f8f4401c970 comp rx=0 tx=0).stop 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- 172.21.15.83:0/1387668367 shutdown_connections 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 --2- 172.21.15.83:0/1387668367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f500d2050 0x7f8f500d2460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 --2- 172.21.15.83:0/1387668367 >> v2:172.21.15.155:3300/0 conn(0x7f8f500d0e00 0x7f8f5015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- 172.21.15.83:0/1387668367 >> 172.21.15.83:0/1387668367 conn(0x7f8f5006dfd0 msgr2=0x7f8f5006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- 172.21.15.83:0/1387668367 shutdown_connections 2024-05-17T22:07:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- 172.21.15.83:0/1387668367 wait complete. 2024-05-17T22:07:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 Processor -- start 2024-05-17T22:07:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.090+0000 7f8f556da640 1 -- start start 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500d0e00 0x7f8f5014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f501600f0 0x7f8f50152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500b4af0 con 0x7f8f500d2050 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5015d990 con 0x7f8f500d0e00 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8f500b4c70 con 0x7f8f501600f0 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34522/0 (socket says 172.21.15.83:34522) 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/457007685 learned_addr learned my addr 172.21.15.83:0/457007685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4effd640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500d0e00 0x7f8f5014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4f7fe640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f501600f0 0x7f8f50152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f501600f0 msgr2=0x7f8f50152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f501600f0 0x7f8f50152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500d0e00 msgr2=0x7f8f5014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500d0e00 0x7f8f5014a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 -- 172.21.15.83:0/457007685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500b6310 con 0x7f8f500d2050 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4e7fc640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b0040 tx=0x7f8f44019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440b2050 con 0x7f8f500d2050 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f440be070 con 0x7f8f500d2050 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f440ad850 con 0x7f8f500d2050 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f500b65f0 con 0x7f8f500d2050 2024-05-17T22:07:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f500b6bb0 con 0x7f8f500d2050 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5015d450 con 0x7f8f500d2050 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8f440ad9f0 con 0x7f8f500d2050 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f377fe640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 0x7f8f3c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.094+0000 7f8f4effd640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 0x7f8f3c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.098+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f8f4412a3d0 con 0x7f8f500d2050 2024-05-17T22:07:23.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.098+0000 7f8f4effd640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 0x7f8f3c071940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009a40 tx=0x7f8f48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.102+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8f44016560 con 0x7f8f500d2050 2024-05-17T22:07:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.274+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f500d64a0 con 0x7f8f500d2050 2024-05-17T22:07:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.274+0000 7f8f377fe640 1 -- 172.21.15.83:0/457007685 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f8f440f72c0 con 0x7f8f500d2050 2024-05-17T22:07:23.269 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 msgr2=0x7f8f3c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 0x7f8f3c071940 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009a40 tx=0x7f8f48008040 comp rx=0 tx=0).stop 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 msgr2=0x7f8f500b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b0040 tx=0x7f8f44019710 comp rx=0 tx=0).stop 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 shutdown_connections 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f501600f0 0x7f8f50152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f500d2050 0x7f8f500b7810 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8f3c06f430 0x7f8f3c071940 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 --2- 172.21.15.83:0/457007685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f500d0e00 0x7f8f5014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 >> 172.21.15.83:0/457007685 conn(0x7f8f5006dfd0 msgr2=0x7f8f5015ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 shutdown_connections 2024-05-17T22:07:23.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:23.278+0000 7f8f556da640 1 -- 172.21.15.83:0/457007685 wait complete. 2024-05-17T22:07:23.290 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:07:23.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:23.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:23.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48519]: fuse finished with error 0 and tester_r 0 2024-05-17T22:07:23.301 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:29.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:07:29.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:29.403 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:07:29.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-17T22:07:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 Processor -- start 2024-05-17T22:07:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 -- start start 2024-05-17T22:07:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 0x7f8cd815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd80b5640 con 0x7f8cd80cf830 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd8070870 con 0x7f8cd80cdf40 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdebaf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd80b58e0 con 0x7f8cd815d530 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.83:36492/0 (socket says 172.21.15.83:36492) 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2489549014 learned_addr learned my addr 172.21.15.83:0/2489549014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.622+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2489549014 >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 0x7f8cd815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2489549014 >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 msgr2=0x7f8cd815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2489549014 >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 0x7f8cd815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2489549014 >> v1:172.21.15.155:6789/0 conn(0x7f8cd80cf830 legacy=0x7f8cd80b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2489549014 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd806f060 con 0x7f8cd815d530 2024-05-17T22:07:29.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2489549014 >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 0x7f8cd815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2489549014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009e50 tx=0x7f8cd001ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c75d26784bfcb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cd77fe640 1 -- 172.21.15.83:0/2489549014 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00a7920 con 0x7f8cd815d530 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cd77fe640 1 -- 172.21.15.83:0/2489549014 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd00a7ac0 con 0x7f8cd815d530 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cd77fe640 1 -- 172.21.15.83:0/2489549014 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00af3f0 con 0x7f8cd815d530 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 -- 172.21.15.83:0/2489549014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 msgr2=0x7f8cd81663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/2489549014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009e50 tx=0x7f8cd001ec40 comp rx=0 tx=0).stop 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 -- 172.21.15.83:0/2489549014 shutdown_connections 2024-05-17T22:07:29.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/2489549014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd81663d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/2489549014 >> v2:172.21.15.155:3300/0 conn(0x7f8cd80cdf40 0x7f8cd815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 -- 172.21.15.83:0/2489549014 >> 172.21.15.83:0/2489549014 conn(0x7f8cd806cd90 msgr2=0x7f8cd80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 -- 172.21.15.83:0/2489549014 shutdown_connections 2024-05-17T22:07:29.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.626+0000 7f8cdebaf640 1 -- 172.21.15.83:0/2489549014 wait complete. 2024-05-17T22:07:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 Processor -- start 2024-05-17T22:07:29.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 -- start start 2024-05-17T22:07:29.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 0x7f8cd8141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd80b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8073a20 con 0x7f8cd80cf830 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80738a0 con 0x7f8cd80cdf40 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd8073ba0 con 0x7f8cd815d530 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45426/0 (socket says 172.21.15.83:45426) 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 -- 172.21.15.83:0/1094609902 learned_addr learned my addr 172.21.15.83:0/1094609902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 0x7f8cd8141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd80b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 -- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 msgr2=0x7f8cd80b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd80b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 -- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 msgr2=0x7f8cd8141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 0x7f8cd8141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 -- 172.21.15.83:0/1094609902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd81fa060 con 0x7f8cd80cf830 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 0x7f8cd8141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e980 tx=0x7f8ccc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd57fa640 1 -- 172.21.15.83:0/1094609902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc017070 con 0x7f8cd80cf830 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd57fa640 1 -- 172.21.15.83:0/1094609902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00c9a0 con 0x7f8cd80cf830 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd8159230 con 0x7f8cd80cf830 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.630+0000 7f8cd57fa640 1 -- 172.21.15.83:0/1094609902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cd70 con 0x7f8cd80cf830 2024-05-17T22:07:29.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd8164a00 con 0x7f8cd80cf830 2024-05-17T22:07:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cd57fa640 1 -- 172.21.15.83:0/1094609902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ccc01a070 con 0x7f8cd80cf830 2024-05-17T22:07:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cd57fa640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 0x7f8cc4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cd57fa640 1 -- 172.21.15.83:0/1094609902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc08f590 con 0x7f8cd80cf830 2024-05-17T22:07:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 0x7f8cc4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 0x7f8cc4071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8cd815a9a0 tx=0x7f8cc8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8ce5e29000 1 Processor -- start 2024-05-17T22:07:29.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.634+0000 7f8ce5e29000 1 -- start start 2024-05-17T22:07:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b1196bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b1196bdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b119553a90 con 0x55b11953c090 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1194c7120 con 0x55b11953b880 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b1194b6cc0 con 0x55b11954ef80 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45434/0 (socket says 172.21.15.83:45434) 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 -- 172.21.15.83:0/55745724 learned_addr learned my addr 172.21.15.83:0/55745724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b1196bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b1196bdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 -- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 msgr2=0x55b1196bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b1196bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 -- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 msgr2=0x55b1196bdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b1196bdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 -- 172.21.15.83:0/55745724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b119473110 con 0x55b11953c090 2024-05-17T22:07:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001c870 tx=0x7f8cd00a9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a2fa3d9f2b7aac server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cbe7fc640 1 -- 172.21.15.83:0/55745724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0009590 con 0x55b11953c090 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cbe7fc640 1 -- 172.21.15.83:0/55745724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd00090d0 con 0x55b11953c090 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8cbe7fc640 1 -- 172.21.15.83:0/55745724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00ba470 con 0x55b11953c090 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- 172.21.15.83:0/55745724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 msgr2=0x55b1196be020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001c870 tx=0x7f8cd00a9070 comp rx=0 tx=0).stop 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- 172.21.15.83:0/55745724 shutdown_connections 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b1196be020 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b1196bdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/55745724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b1196bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- 172.21.15.83:0/55745724 >> 172.21.15.83:0/55745724 conn(0x55b11958a010 msgr2=0x55b119670dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- 172.21.15.83:0/55745724 shutdown_connections 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- 172.21.15.83:0/55745724 wait complete. 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 -- start start 2024-05-17T22:07:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.638+0000 7f8ce5e29000 1 Processor -- start 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b11967f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b11967fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b119472ec0 con 0x55b11953c090 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b1194bad00 con 0x55b11953b880 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b119180680 con 0x55b11954ef80 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 10 client.0 _ll_register_callbacks cb 0x55b11956e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45446/0 (socket says 172.21.15.83:45446) 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 -- 172.21.15.83:0/4173779419 learned_addr learned my addr 172.21.15.83:0/4173779419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b11967fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b11967f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 -- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 msgr2=0x55b11967fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b11967fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 -- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 msgr2=0x55b11967f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b11967f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 -- 172.21.15.83:0/4173779419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b119475890 con 0x55b11953c090 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc08dae0 tx=0x7f8ccc05b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a7c189388a18d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc021070 con 0x55b11953c090 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc08f0c0 con 0x55b11953c090 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd00090d0 con 0x55b11953c090 2024-05-17T22:07:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc060130 con 0x55b11953c090 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 10 client.8154 fetch_fsmap learned FSMap version 97 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x55b119477650 con 0x55b11953c090 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd00090d0 con 0x55b11953c090 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 97) ==== 3494+0+0 (secure 0 0 0) 0x7f8ccc05afe0 con 0x55b11953c090 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 10 client.8154 fetch_fsmap finished waiting for FSMap version 97 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 10 client.8154 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 20 client.8154 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11915d930 0x55b1196ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 4 client.8154 mds_command: new command op to 7992 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 --> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b119452c40 con 0x55b11915d930 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11915d930 0x55b1196ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11915d930 0x55b1196ceb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d884728a6036e6 server_cookie=69c1118fddcd7859 in_seq=0 out_seq=0 2024-05-17T22:07:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.642+0000 7f8cb77fe640 10 client.8154 ms_handle_connect on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8cb77fe640 1 -- 172.21.15.83:0/4173779419 <== mds.0 v2:172.21.15.155:6832/1193425174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b119452c40 con 0x55b11915d930 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8cb77fe640 10 client.8154 handle_command_reply: tid=0 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 client.8154 shutdown 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 msgr2=0x55b11967f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc08dae0 tx=0x7f8ccc05b5b0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 shutdown_connections 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11915d930 0x55b1196ceb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11954ef80 0x55b11967fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8cb77fe640 0 client.8154 ms_handle_reset on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953c090 0x55b11967f880 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/4173779419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953b880 0x55b11967f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 >> 172.21.15.83:0/4173779419 conn(0x55b11958a010 msgr2=0x55b119671780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 shutdown_connections 2024-05-17T22:07:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 1 -- 172.21.15.83:0/4173779419 wait complete. 2024-05-17T22:07:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.646+0000 7f8ce5e29000 20 client.8154 trim_cache size 0 max 16384 2024-05-17T22:07:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 Processor -- start 2024-05-17T22:07:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- start start 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 0x55b1196714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 0x55b119671a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8162370 con 0x55b11954ef80 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8166080 con 0x55b11953c090 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd81653c0 con 0x55b11953b880 2024-05-17T22:07:29.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45452/0 (socket says 172.21.15.83:45452) 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2241187293 learned_addr learned my addr 172.21.15.83:0/2241187293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 0x55b1196714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 0x55b119671a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 msgr2=0x55b1196714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 0x55b1196714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 msgr2=0x55b119671a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 0x55b119671a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 -- 172.21.15.83:0/2241187293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1194568c0 con 0x55b11954ef80 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 0x55b1196714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 0x55b119671a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:29.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001f4e0 tx=0x7f8cd0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34ee33dd319811a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cbdffb640 1 -- 172.21.15.83:0/2241187293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00ca070 con 0x55b11954ef80 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cbdffb640 1 -- 172.21.15.83:0/2241187293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd00090d0 con 0x55b11954ef80 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8cbdffb640 1 -- 172.21.15.83:0/2241187293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00039a0 con 0x55b11954ef80 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2241187293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 msgr2=0x55b1196cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001f4e0 tx=0x7f8cd0004ac0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2241187293 shutdown_connections 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11954ef80 0x55b1196cfb70 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11953c090 0x55b119671a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2241187293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953b880 0x55b1196714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2241187293 >> 172.21.15.83:0/2241187293 conn(0x55b11958a010 msgr2=0x55b119670dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2241187293 shutdown_connections 2024-05-17T22:07:29.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.662+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2241187293 wait complete. 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- start start 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 Processor -- start 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953c090 0x55b119730540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11954ef80 0x55b119730a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8165a10 con 0x55b11953b880 2024-05-17T22:07:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b11947df10 con 0x55b11954ef80 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55b1194bf330 con 0x55b11953c090 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 10 client.0 _ll_register_callbacks cb 0x7f8ccc00cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45462/0 (socket says 172.21.15.83:45462) 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 -- 172.21.15.83:0/2764231268 learned_addr learned my addr 172.21.15.83:0/2764231268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdd14c640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11954ef80 0x55b119730a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953c090 0x55b119730540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 -- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953c090 msgr2=0x55b119730540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953c090 0x55b119730540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 -- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11954ef80 msgr2=0x55b119730a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11954ef80 0x55b119730a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 -- 172.21.15.83:0/2764231268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b119471fa0 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cdc94b640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80052c0 tx=0x7f8cc8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80801cfa33de9d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800ec90 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f8cc8010040 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd00090d0 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7f8cc8036050 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 10 client.8160 fetch_fsmap learned FSMap version 97 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x55b119171220 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd00090d0 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 97) ==== 3494+0+0 (secure 0 0 0) 0x7f8cc8039020 con 0x55b11953b880 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 10 client.8160 fetch_fsmap finished waiting for FSMap version 97 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 10 client.8160 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 20 client.8160 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:29.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11914b990 0x55b1196ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 4 client.8160 mds_command: new command op to 7992 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T22:07:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 --> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1194c9070 con 0x55b11914b990 2024-05-17T22:07:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.666+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11914b990 0x55b1196ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.670+0000 7f8cb77fe640 10 client.8160 ms_handle_connect on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.670+0000 7f8cd7fff640 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11914b990 0x55b1196ec1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73eb6d4a7afc762f server_cookie=ec18612bb8c91d62 in_seq=0 out_seq=0 2024-05-17T22:07:29.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cb77fe640 1 -- 172.21.15.83:0/2764231268 <== mds.0 v2:172.21.15.155:6832/1193425174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b1194c9070 con 0x55b11914b990 2024-05-17T22:07:29.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cb77fe640 10 client.8160 handle_command_reply: tid=0 2024-05-17T22:07:29.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 client.8160 shutdown 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 msgr2=0x55b119730000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80052c0 tx=0x7f8cc8000bd0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 shutdown_connections 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:6832/1193425174,v1:172.21.15.155:6833/1193425174] conn(0x55b11914b990 0x55b1196ec1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b11954ef80 0x55b119730a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55b11953c090 0x55b119730540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 --2- 172.21.15.83:0/2764231268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b11953b880 0x55b119730000 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cb77fe640 0 client.8160 ms_handle_reset on v2:172.21.15.155:6832/1193425174 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 >> 172.21.15.83:0/2764231268 conn(0x55b11958a010 msgr2=0x55b119671780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 shutdown_connections 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 1 -- 172.21.15.83:0/2764231268 wait complete. 2024-05-17T22:07:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8ce5e29000 20 client.8160 trim_cache size 0 max 16384 2024-05-17T22:07:29.681 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:29.681 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-05-17T22:07:29.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 msgr2=0x7f8cc4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 0x7f8cc4071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8cd815a9a0 tx=0x7f8cc8009040 comp rx=0 tx=0).stop 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 msgr2=0x7f8cd81f9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00e980 tx=0x7f8ccc00ee40 comp rx=0 tx=0).stop 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 shutdown_connections 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8cd815d530 0x7f8cd80b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd80cf830 0x7f8cd81f9b20 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8cc406f400 0x7f8cc4071910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 --2- 172.21.15.83:0/1094609902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd80cdf40 0x7f8cd8141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 >> 172.21.15.83:0/1094609902 conn(0x7f8cd806cd90 msgr2=0x7f8cd80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 shutdown_connections 2024-05-17T22:07:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.686+0000 7f8cdebaf640 1 -- 172.21.15.83:0/1094609902 wait complete. 2024-05-17T22:07:29.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:07:29.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 Processor -- start 2024-05-17T22:07:29.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 -- start start 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc82c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc82c0b0fa0 0x7fc82c161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82c0af9c0 con 0x7fc82c06c790 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc82c0afca0 con 0x7fc82c0b0fa0 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc8324dc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c0aff80 con 0x7fc82c0af3e0 2024-05-17T22:07:29.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc82c06c790 0x7fc82c156c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45576/0 (socket says 172.21.15.83:45576) 2024-05-17T22:07:29.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82bfff640 1 -- 172.21.15.83:0/4255853314 learned_addr learned my addr 172.21.15.83:0/4255853314 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:29.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc82c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc830a79640 1 --2- 172.21.15.83:0/4255853314 >> v2:172.21.15.155:3300/0 conn(0x7fc82c0b0fa0 0x7fc82c161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309002198 0 0) 0x7fc82c0af9c0 con 0x7fc82c06c790 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc814003530 con 0x7fc82c06c790 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2349304332 0 0) 0x7fc814003530 con 0x7fc82c06c790 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 msgr2=0x7fc82c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc82c0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 >> v2:172.21.15.155:3300/0 conn(0x7fc82c0b0fa0 msgr2=0x7fc82c161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 --2- 172.21.15.83:0/4255853314 >> v2:172.21.15.155:3300/0 conn(0x7fc82c0b0fa0 0x7fc82c161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c0b0260 con 0x7fc82c06c790 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fc824001a80 con 0x7fc82c06c790 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 >> v1:172.21.15.83:6789/0 conn(0x7fc82c06c790 legacy=0x7fc82c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814003a10 0x7fc814003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc814004810 0x7fc814004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc814005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0afca0 con 0x7fc814004810 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0aff80 con 0x7fc814003a10 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc81400bab0 con 0x7fc82c0af3e0 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc814004810 0x7fc814004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82bfff640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814003a10 0x7fc814003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc830a79640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc814005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 -- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 msgr2=0x7fc814005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c0af3e0 0x7fc814005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 -- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814003a10 msgr2=0x7fc814003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814003a10 0x7fc814003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.850+0000 7fc82b7fe640 1 -- 172.21.15.83:0/4255853314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc814003530 con 0x7fc814004810 2024-05-17T22:07:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc814004810 0x7fc814004c20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc82c073a40 tx=0x7fc82001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63b1e45ed975a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b2070 con 0x7fc814004810 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8200a5510 con 0x7fc814004810 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc82affd640 1 -- 172.21.15.83:0/4255853314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200096b0 con 0x7fc814004810 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 -- 172.21.15.83:0/4255853314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc814004810 msgr2=0x7fc814004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 --2- 172.21.15.83:0/4255853314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc814004810 0x7fc814004c20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc82c073a40 tx=0x7fc82001ca30 comp rx=0 tx=0).stop 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc830a79640 1 -- 172.21.15.83:0/4255853314 reap_dead start 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 -- 172.21.15.83:0/4255853314 shutdown_connections 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 -- 172.21.15.83:0/4255853314 >> 172.21.15.83:0/4255853314 conn(0x7fc82c09b8b0 msgr2=0x7fc82c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 -- 172.21.15.83:0/4255853314 shutdown_connections 2024-05-17T22:07:29.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.854+0000 7fc8324dc640 1 -- 172.21.15.83:0/4255853314 wait complete. 2024-05-17T22:07:29.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 Processor -- start 2024-05-17T22:07:29.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- start start 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814004810 0x7fc82c160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c15aeb0 0x7fc82c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c0b0a60 con 0x7fc82c155ec0 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c09be90 con 0x7fc814004810 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c0b0be0 con 0x7fc82c15aeb0 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45484/0 (socket says 172.21.15.83:45484) 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 -- 172.21.15.83:0/489865451 learned_addr learned my addr 172.21.15.83:0/489865451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82bfff640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814004810 0x7fc82c160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc830a79640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c15aeb0 0x7fc82c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 -- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c15aeb0 msgr2=0x7fc82c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c15aeb0 0x7fc82c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 -- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814004810 msgr2=0x7fc82c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814004810 0x7fc82c160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 -- 172.21.15.83:0/489865451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c15c1f0 con 0x7fc82c155ec0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc82b7fe640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc8200b0040 tx=0x7fc814005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b2040 con 0x7fc82c155ec0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82000f000 con 0x7fc82c155ec0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c15f0b0 con 0x7fc82c155ec0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200096b0 con 0x7fc82c155ec0 2024-05-17T22:07:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c15bf10 con 0x7fc82c155ec0 2024-05-17T22:07:29.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.858+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c156c00 con 0x7fc82c155ec0 2024-05-17T22:07:29.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.862+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8200c0030 con 0x7fc82c155ec0 2024-05-17T22:07:29.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.862+0000 7fc828ff9640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 0x7fc818071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:29.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.862+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fc8201290f0 con 0x7fc82c155ec0 2024-05-17T22:07:29.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.862+0000 7fc82bfff640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 0x7fc818071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:29.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.862+0000 7fc82bfff640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 0x7fc818071910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc824004c00 tx=0x7fc824009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:29.866+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc8200c7060 con 0x7fc82c155ec0 2024-05-17T22:07:30.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.062+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc82c15ac90 con 0x7fc82c155ec0 2024-05-17T22:07:30.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.062+0000 7fc828ff9640 1 -- 172.21.15.83:0/489865451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7fc820016740 con 0x7fc82c155ec0 2024-05-17T22:07:30.057 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:07:30.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 msgr2=0x7fc818071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 0x7fc818071910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc824004c00 tx=0x7fc824009040 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 msgr2=0x7fc82c149070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc8200b0040 tx=0x7fc814005840 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 shutdown_connections 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc82c15aeb0 0x7fc82c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82c155ec0 0x7fc82c149070 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc81806f400 0x7fc818071910 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 --2- 172.21.15.83:0/489865451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc814004810 0x7fc82c160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 >> 172.21.15.83:0/489865451 conn(0x7fc82c09b8b0 msgr2=0x7fc82c1625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 shutdown_connections 2024-05-17T22:07:30.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30.066+0000 7fc8324dc640 1 -- 172.21.15.83:0/489865451 wait complete. 2024-05-17T22:07:30.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:07:30.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:07:30.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:07:30.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:07:30.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:30.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:07:30.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:07:30.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:07:30.079 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:30.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.079 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:07:30.086 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:07:30.086 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:07:30.086 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.087 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41894sec preferred_lft 41894sec 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.088 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:30.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.089 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:07:30.139 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:07:30.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.141 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:30.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:30.192 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:30.192 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:07:30.192 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:07:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:07:30.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:30.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.311 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:07:30.363 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:07:30.363 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:07:30.363 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:30.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:07:30.419 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:07:30.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:30.419 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:30.467 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:30.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:30.467 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:30.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:30.572 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:30.620 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:30.620 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:30.620 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:30.621 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:30.622 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:30.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.623 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:30.671 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:07:30.671 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:07:30.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:30.731 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:30.779 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:30.780 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:30.781 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:30.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:30.782 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:30.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:07:30.830+0000 7fb8dde3b580 -1 init, newargv = 0x55b201e29cc0 newargc=15 2024-05-17T22:07:30.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48756]: starting ceph client 2024-05-17T22:07:30.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48756]: starting fuse 2024-05-17T22:07:31.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:07:31.897 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:07:31.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:31.996 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:32.043 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:32.044 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:32.045 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:32.046 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:32.046 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:32.046 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:32.046 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:32.046 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:32.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.047 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:32.095 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:07:32.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:07:32.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:07:32.095 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:07:32.095 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:07:32.095 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:32.095 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:07:32.096 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:07:32.097 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:32.220 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.48756.asok 2024-05-17T22:07:32.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:32.232 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48756.asok 2024-05-17T22:07:32.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48756.asok status 2024-05-17T22:07:32.408 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:07:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:07:32.408 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "48756", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8172, 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:07:32.409 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8172 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2009547967 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2009547967 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8172 192.168.0.1:0/2009547967", 2024-05-17T22:07:32.410 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/2009547967", 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 97, 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 109, 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:07:32.411 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:32.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.424 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:07:32.425 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:32.430 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:07:32.431 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:32.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:32.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.487 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-17T22:07:32.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:32.547 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-17T22:07:32.666 INFO:teuthology.orchestra.run.smithi083.stderr:8+0 records in 2024-05-17T22:07:32.666 INFO:teuthology.orchestra.run.smithi083.stderr:8+0 records out 2024-05-17T22:07:32.667 INFO:teuthology.orchestra.run.smithi083.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0657669 s, 128 MB/s 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-17T22:07:32.668 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:32.737 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627779 2024-05-17T22:07:32.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-17T22:07:32.747 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:32.850 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627780 2024-05-17T22:07:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:32.860 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:32.860 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:32.908 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:33.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 Processor -- start 2024-05-17T22:07:33.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 -- start start 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e80d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4e8143f20 0x7fb4e8072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 0x7fb4e806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e80b4110 con 0x7fb4e8143f20 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e8070770 con 0x7fb4e80733c0 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.114+0000 7fb4ede62640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e80737d0 con 0x7fb4e80b3d00 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4e8143f20 0x7fb4e8072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4e8143f20 0x7fb4e8072e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45488/0 (socket says 172.21.15.83:45488) 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 -- 172.21.15.83:0/3275934359 learned_addr learned my addr 172.21.15.83:0/3275934359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e7fff640 1 --2- 172.21.15.83:0/3275934359 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 0x7fb4e806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e77fe640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e80d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 -- 172.21.15.83:0/3275934359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 msgr2=0x7fb4e80d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e80d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 -- 172.21.15.83:0/3275934359 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 msgr2=0x7fb4e806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/3275934359 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 0x7fb4e806eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 -- 172.21.15.83:0/3275934359 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e80d10d0 con 0x7fb4e8143f20 2024-05-17T22:07:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e7fff640 1 --2- 172.21.15.83:0/3275934359 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 0x7fb4e806eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8143f20 0x7fb4e8072e80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8072e10 tx=0x7fb4dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ab9771923e624 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e67fc640 1 -- 172.21.15.83:0/3275934359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0aa610 con 0x7fb4e8143f20 2024-05-17T22:07:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e67fc640 1 -- 172.21.15.83:0/3275934359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4dc0af070 con 0x7fb4e8143f20 2024-05-17T22:07:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.118+0000 7fb4e67fc640 1 -- 172.21.15.83:0/3275934359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0aa9d0 con 0x7fb4e8143f20 2024-05-17T22:07:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- 172.21.15.83:0/3275934359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8143f20 msgr2=0x7fb4e8072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8143f20 0x7fb4e8072e80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8072e10 tx=0x7fb4dc01c4f0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- 172.21.15.83:0/3275934359 shutdown_connections 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 --2- 172.21.15.83:0/3275934359 >> v2:172.21.15.155:3300/0 conn(0x7fb4e80733c0 0x7fb4e806eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8143f20 0x7fb4e8072e80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 --2- 172.21.15.83:0/3275934359 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e80d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- 172.21.15.83:0/3275934359 >> 172.21.15.83:0/3275934359 conn(0x7fb4e806cd90 msgr2=0x7fb4e80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- 172.21.15.83:0/3275934359 shutdown_connections 2024-05-17T22:07:33.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- 172.21.15.83:0/3275934359 wait complete. 2024-05-17T22:07:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 Processor -- start 2024-05-17T22:07:33.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.122+0000 7fb4ede62640 1 -- start start 2024-05-17T22:07:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e8142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 0x7fb4e813b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8070c00 con 0x7fb4e80733c0 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8144330 con 0x7fb4e8143f20 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e80af040 con 0x7fb4e80b3d00 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45502/0 (socket says 172.21.15.83:45502) 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 -- 172.21.15.83:0/2670396224 learned_addr learned my addr 172.21.15.83:0/2670396224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e8142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e7fff640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 0x7fb4e813b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 -- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 msgr2=0x7fb4e8142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e8142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 -- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 msgr2=0x7fb4e813b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 0x7fb4e813b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 -- 172.21.15.83:0/2670396224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e813a6e0 con 0x7fb4e80733c0 2024-05-17T22:07:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e8142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e7fff640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 0x7fb4e813b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e77fe640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7f0 tx=0x7fb4d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8015070 con 0x7fb4e80733c0 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d8007820 con 0x7fb4e80733c0 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e80aa4b0 con 0x7fb4e80733c0 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8007c10 con 0x7fb4e80733c0 2024-05-17T22:07:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e80aaa70 con 0x7fb4e80733c0 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e806fc10 con 0x7fb4e80733c0 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb4d8018070 con 0x7fb4e80733c0 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 0x7fb4d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d808f0f0 con 0x7fb4e80733c0 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.126+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 0x7fb4d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.130+0000 7fb4e6ffd640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 0x7fb4d4071940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ae000 tx=0x7fb4dc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.130+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fb4d805bfe0 con 0x7fb4e80733c0 2024-05-17T22:07:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.306+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb4e813db00 con 0x7fb4e80733c0 2024-05-17T22:07:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.306+0000 7fb4cffff640 1 -- 172.21.15.83:0/2670396224 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7fb4d805b8e0 con 0x7fb4e80733c0 2024-05-17T22:07:33.300 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 msgr2=0x7fb4d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 0x7fb4d4071940 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ae000 tx=0x7fb4dc0a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 msgr2=0x7fb4e813a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7f0 tx=0x7fb4d800ccb0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 shutdown_connections 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb4d406f430 0x7fb4d4071940 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4e8143f20 0x7fb4e813b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4e80b3d00 0x7fb4e8142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 --2- 172.21.15.83:0/2670396224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e80733c0 0x7fb4e813a1a0 secure :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c7f0 tx=0x7fb4d800ccb0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 >> 172.21.15.83:0/2670396224 conn(0x7fb4e806cd90 msgr2=0x7fb4e8075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:33.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 shutdown_connections 2024-05-17T22:07:33.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.310+0000 7fb4ede62640 1 -- 172.21.15.83:0/2670396224 wait complete. 2024-05-17T22:07:33.321 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:33.321 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:33.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:33.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 Processor -- start 2024-05-17T22:07:33.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 -- start start 2024-05-17T22:07:33.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71a80d0a10 0x7f71a8143f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a80d0e20 con 0x7f71a8073590 2024-05-17T22:07:33.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a80d10d0 con 0x7f71a80b3d00 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71ad9bd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f71a8144460 con 0x7f71a80d0a10 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a67fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f71a8073590 0x7f71a81447f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:45588/0 (socket says 172.21.15.83:45588) 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a67fc640 1 -- 172.21.15.83:0/1299232285 learned_addr learned my addr 172.21.15.83:0/1299232285 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1299232285 >> v2:172.21.15.155:3300/0 conn(0x7f71a80d0a10 0x7f71a8143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936782042 0 0) 0x7f71a80d0e20 con 0x7f71a8073590 2024-05-17T22:07:33.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7190003530 con 0x7f71a8073590 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4280852261 0 0) 0x7f7190003530 con 0x7f71a8073590 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 >> v2:172.21.15.155:3300/0 conn(0x7f71a80d0a10 msgr2=0x7f71a8143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 --2- 172.21.15.83:0/1299232285 >> v2:172.21.15.155:3300/0 conn(0x7f71a80d0a10 0x7f71a8143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 >> v1:172.21.15.155:6789/0 conn(0x7f71a80b3d00 legacy=0x7f71a8072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a8070120 con 0x7f71a8073590 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f719c001700 con 0x7f71a8073590 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.530+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 >> v1:172.21.15.83:6789/0 conn(0x7f71a8073590 legacy=0x7f71a81447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190003a60 0x7f7190003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7190004810 0x7f7190004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 0x7f719000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8144460 con 0x7f7190005260 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f719000c1f0 con 0x7f7190003a60 2024-05-17T22:07:33.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f719000c3f0 con 0x7f7190004810 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 0x7f719000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a6ffd640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190003a60 0x7f7190003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 -- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7190004810 msgr2=0x7f7190004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7190004810 0x7f7190004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 -- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190003a60 msgr2=0x7f7190003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190003a60 0x7f7190003ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 -- 172.21.15.83:0/1299232285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7190003530 con 0x7f7190005260 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 -- 172.21.15.83:0/1299232285 reap_dead start 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 0x7f719000b9d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009ae0 tx=0x7f71a001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee8f4f1a4b29c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a00096b0 con 0x7f7190005260 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0004020 con 0x7f7190005260 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71a5ffb640 1 -- 172.21.15.83:0/1299232285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004410 con 0x7f7190005260 2024-05-17T22:07:33.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 msgr2=0x7f719000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 0x7f719000b9d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009ae0 tx=0x7f71a001cab0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1299232285 shutdown_connections 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1299232285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7190005260 0x7f719000b9d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1299232285 >> 172.21.15.83:0/1299232285 conn(0x7f71a806cd90 msgr2=0x7f71a80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1299232285 shutdown_connections 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.534+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1299232285 wait complete. 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 Processor -- start 2024-05-17T22:07:33.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 -- start start 2024-05-17T22:07:33.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190005260 0x7f71a81f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a8155320 0x7f71a813e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8070770 con 0x7f71a80ae720 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71a80b4110 con 0x7f7190005260 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71ad9bd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f71a8147600 con 0x7f71a8155320 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45506/0 (socket says 172.21.15.83:45506) 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 -- 172.21.15.83:0/1950941069 learned_addr learned my addr 172.21.15.83:0/1950941069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a77fe640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a8155320 0x7f71a813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a6ffd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190005260 0x7f71a81f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 -- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a8155320 msgr2=0x7f71a813e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a8155320 0x7f71a813e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 -- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190005260 msgr2=0x7f71a81f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190005260 0x7f71a81f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 -- 172.21.15.83:0/1950941069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a81f56b0 con 0x7f71a80ae720 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.538+0000 7f71a67fc640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f70 tx=0x7f719c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0072d0 con 0x7f71a80ae720 2024-05-17T22:07:33.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719c007470 con 0x7f71a80ae720 2024-05-17T22:07:33.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a80aef10 con 0x7f71a80ae720 2024-05-17T22:07:33.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0047f0 con 0x7f71a80ae720 2024-05-17T22:07:33.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a80af4d0 con 0x7f71a80ae720 2024-05-17T22:07:33.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a8070c00 con 0x7f71a80ae720 2024-05-17T22:07:33.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f719c011070 con 0x7f71a80ae720 2024-05-17T22:07:33.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 0x7f7194071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:33.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f719c08f0c0 con 0x7f71a80ae720 2024-05-17T22:07:33.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f71a6ffd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 0x7f7194071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:33.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.542+0000 7f71a6ffd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 0x7f7194071910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7198001c30 tx=0x7f7198009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:33.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.550+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f719c05bf30 con 0x7f71a80ae720 2024-05-17T22:07:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.706+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f71a80b0b90 con 0x7f71a80ae720 2024-05-17T22:07:33.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.706+0000 7f718ffff640 1 -- 172.21.15.83:0/1950941069 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f719c05b830 con 0x7f71a80ae720 2024-05-17T22:07:33.701 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2024-05-17T22:07:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 msgr2=0x7f7194071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 0x7f7194071910 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7198001c30 tx=0x7f7198009040 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 msgr2=0x7f71a8154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f70 tx=0x7f719c00cb90 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 shutdown_connections 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71a8155320 0x7f71a813e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71a80ae720 0x7f71a8154de0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f719406f400 0x7f7194071910 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 --2- 172.21.15.83:0/1950941069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7190005260 0x7f71a81f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 >> 172.21.15.83:0/1950941069 conn(0x7f71a806cd90 msgr2=0x7f71a8144830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 shutdown_connections 2024-05-17T22:07:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:33.710+0000 7f71ad9bd640 1 -- 172.21.15.83:0/1950941069 wait complete. 2024-05-17T22:07:33.720 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:07:33.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:33.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:33.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[48756]: fuse finished with error 0 and tester_r 0 2024-05-17T22:07:33.733 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:39.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:07:39.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:39.836 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:07:39.843 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:39.843 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:39.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:07:39.891 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi083.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:39.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:07:39.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:39.892 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:07:39.938 INFO:teuthology.orchestra.run.smithi083.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-17T22:07:39.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:07:39.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:07:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 Processor -- start 2024-05-17T22:07:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 -- start start 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f501c15cb90 0x7f501c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f501c15d4e0 0x7f501c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f501c0b65a0 con 0x7f501c15dcb0 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f501c0b6110 con 0x7f501c15d4e0 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5021486640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f501c0b5500 con 0x7f501c15cb90 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501affd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f501c15dcb0 0x7f501c0b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46860/0 (socket says 172.21.15.83:46860) 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501affd640 1 -- 172.21.15.83:0/2821525832 learned_addr learned my addr 172.21.15.83:0/2821525832 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501b7fe640 1 --2- 172.21.15.83:0/2821525832 >> v2:172.21.15.83:3300/0 conn(0x7f501c15d4e0 0x7f501c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501a7fc640 1 --2- 172.21.15.83:0/2821525832 >> v2:172.21.15.155:3300/0 conn(0x7f501c15cb90 0x7f501c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5019ffb640 1 -- 172.21.15.83:0/2821525832 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614755087 0 0) 0x7f501c0b65a0 con 0x7f501c15dcb0 2024-05-17T22:07:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f5019ffb640 1 -- 172.21.15.83:0/2821525832 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5004003530 con 0x7f501c15dcb0 2024-05-17T22:07:40.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501b7fe640 1 -- 172.21.15.83:0/2821525832 >> v2:172.21.15.155:3300/0 conn(0x7f501c15cb90 msgr2=0x7f501c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:40.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501b7fe640 1 --2- 172.21.15.83:0/2821525832 >> v2:172.21.15.155:3300/0 conn(0x7f501c15cb90 0x7f501c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501b7fe640 1 -- 172.21.15.83:0/2821525832 >> v1:172.21.15.83:6789/0 conn(0x7f501c15dcb0 legacy=0x7f501c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.150+0000 7f501b7fe640 1 -- 172.21.15.83:0/2821525832 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c0b57e0 con 0x7f501c15d4e0 2024-05-17T22:07:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f501b7fe640 1 --2- 172.21.15.83:0/2821525832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15d4e0 0x7f501c0aff80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b6290 tx=0x7f500c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8eea58073b059b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5019ffb640 1 -- 172.21.15.83:0/2821525832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c0b0070 con 0x7f501c15d4e0 2024-05-17T22:07:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5019ffb640 1 -- 172.21.15.83:0/2821525832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c004230 con 0x7f501c15d4e0 2024-05-17T22:07:40.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5019ffb640 1 -- 172.21.15.83:0/2821525832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c003620 con 0x7f501c15d4e0 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- 172.21.15.83:0/2821525832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15d4e0 msgr2=0x7f501c0aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 --2- 172.21.15.83:0/2821525832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15d4e0 0x7f501c0aff80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b6290 tx=0x7f500c01c460 comp rx=0 tx=0).stop 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- 172.21.15.83:0/2821525832 shutdown_connections 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 --2- 172.21.15.83:0/2821525832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15d4e0 0x7f501c0aff80 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 --2- 172.21.15.83:0/2821525832 >> v2:172.21.15.155:3300/0 conn(0x7f501c15cb90 0x7f501c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- 172.21.15.83:0/2821525832 >> 172.21.15.83:0/2821525832 conn(0x7f501c09b8b0 msgr2=0x7f501c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- 172.21.15.83:0/2821525832 shutdown_connections 2024-05-17T22:07:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- 172.21.15.83:0/2821525832 wait complete. 2024-05-17T22:07:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 Processor -- start 2024-05-17T22:07:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.154+0000 7f5021486640 1 -- start start 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c15cb90 0x7f501c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f501c15d4e0 0x7f501c145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0b0e50 con 0x7f501c15dcb0 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f501c0b0cd0 con 0x7f501c15cb90 2024-05-17T22:07:40.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f501c0b0fd0 con 0x7f501c15d4e0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51790/0 (socket says 172.21.15.83:51790) 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 -- 172.21.15.83:0/3991653593 learned_addr learned my addr 172.21.15.83:0/3991653593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501a7fc640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f501c15d4e0 0x7f501c145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 -- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f501c15d4e0 msgr2=0x7f501c145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f501c15d4e0 0x7f501c145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 -- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c15cb90 msgr2=0x7f501c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501affd640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c15cb90 0x7f501c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c15cb90 0x7f501c1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 -- 172.21.15.83:0/3991653593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c09fcc0 con 0x7f501c15dcb0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f501b7fe640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f500c01a090 tx=0x7f500c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c0b0040 con 0x7f501c15dcb0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c0bb070 con 0x7f501c15dcb0 2024-05-17T22:07:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c09ffa0 con 0x7f501c15dcb0 2024-05-17T22:07:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c003d50 con 0x7f501c15dcb0 2024-05-17T22:07:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c155ec0 con 0x7f501c15dcb0 2024-05-17T22:07:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c0a04c0 con 0x7f501c15dcb0 2024-05-17T22:07:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.158+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f500c0b3070 con 0x7f501c15dcb0 2024-05-17T22:07:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.162+0000 7f50037fe640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 0x7f5008071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.162+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f500c128f20 con 0x7f501c15dcb0 2024-05-17T22:07:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.162+0000 7f501affd640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 0x7f5008071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.162+0000 7f501affd640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 0x7f5008071910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f50140015b0 tx=0x7f5014009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.166+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f501c0a04c0 con 0x7f501c15dcb0 2024-05-17T22:07:40.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.354+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f501c09fa50 con 0x7f501c15dcb0 2024-05-17T22:07:40.351 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:07:40.358+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:07:40.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f50037fe640 1 -- 172.21.15.83:0/3991653593 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f500c016050 con 0x7f501c15dcb0 2024-05-17T22:07:40.354 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 msgr2=0x7f5008071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 0x7f5008071910 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f50140015b0 tx=0x7f5014009040 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 msgr2=0x7f501c154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f500c01a090 tx=0x7f500c019200 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 shutdown_connections 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f501c15dcb0 0x7f501c154070 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f501c15d4e0 0x7f501c145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f500806f400 0x7f5008071910 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 --2- 172.21.15.83:0/3991653593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f501c15cb90 0x7f501c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 >> 172.21.15.83:0/3991653593 conn(0x7f501c09b8b0 msgr2=0x7f501c15c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:40.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 shutdown_connections 2024-05-17T22:07:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.362+0000 7f5021486640 1 -- 172.21.15.83:0/3991653593 wait complete. 2024-05-17T22:07:40.373 INFO:tasks.ceph.mds.d.smithi155.stderr: -7771> 2024-05-17T22:07:18.459+0000 7fcd5bb58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:07:40.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:40.380 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-05-17T22:07:40.399 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:40.400 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:40.401 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.402 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.403 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-05-17T22:07:40.404 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-05-17T22:07:40.405 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:40.406 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:40.407 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-05-17T22:07:40.408 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-05-17T22:07:40.409 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = False 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.410 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.413 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-05-17T22:07:40.413 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.414 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:40.415 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-05-17T22:07:40.416 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.417 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.418 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-05-17T22:07:40.419 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.420 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.421 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.422 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.423 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = False 2024-05-17T22:07:40.424 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:40.425 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.425 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:40.455 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:40.457 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2024-05-17T22:07:40.477 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-05-17T22:07:40.477 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-05-17T22:07:40.477 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:40.477 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-05-17T22:07:40.477 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.478 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-05-17T22:07:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-05-17T22:07:40.481 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-05-17T22:07:40.482 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-05-17T22:07:40.483 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.484 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:40.485 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-05-17T22:07:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T22:07:40.487 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = False 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = False 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.488 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-05-17T22:07:40.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-05-17T22:07:40.491 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.492 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-05-17T22:07:40.493 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.494 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.495 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:40.496 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:40.497 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.498 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-05-17T22:07:40.499 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:40.500 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = False 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = False 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:40.501 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:40.501 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-17T22:07:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 Processor -- start 2024-05-17T22:07:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 -- start start 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab814b0 0x55a5dab7d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5daa881b0 con 0x55a5dab81b20 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5daae3ef0 con 0x55a5dac0d550 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46d188e080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a5dab677b0 con 0x55a5dab814b0 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51814/0 (socket says 172.21.15.83:51814) 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 -- 172.21.15.83:0/2697554170 learned_addr learned my addr 172.21.15.83:0/2697554170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cab6f640 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 -- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab814b0 msgr2=0x55a5dab7d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab814b0 0x55a5dab7d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 -- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 msgr2=0x55a5dab80f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab80f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 -- 172.21.15.83:0/2697554170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5daa62c10 con 0x55a5dab81b20 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cab6f640 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46cb370640 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a3f0 tx=0x7f46c40033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6ac31d58b5e46c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46c9b6d640 1 -- 172.21.15.83:0/2697554170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b7070 con 0x55a5dab81b20 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46c9b6d640 1 -- 172.21.15.83:0/2697554170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c40aab70 con 0x55a5dab81b20 2024-05-17T22:07:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.606+0000 7f46c9b6d640 1 -- 172.21.15.83:0/2697554170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b2400 con 0x55a5dab81b20 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/2697554170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 msgr2=0x55a5dac0ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a3f0 tx=0x7f46c40033b0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/2697554170 shutdown_connections 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab814b0 0x55a5dab7d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- 172.21.15.83:0/2697554170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab81b20 0x55a5dac0ba70 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/2697554170 >> 172.21.15.83:0/2697554170 conn(0x55a5dab24e40 msgr2=0x55a5dab3cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/2697554170 shutdown_connections 2024-05-17T22:07:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/2697554170 wait complete. 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 Processor -- start 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- start start 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 0x55a5dab3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab81b20 0x55a5dab7ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab7bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5dab69ba0 con 0x55a5dab814b0 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a5dab63b10 con 0x55a5dac0d550 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55a5dab3a350 con 0x55a5dab81b20 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 0x55a5dab3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 0x55a5dab3bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51830/0 (socket says 172.21.15.83:51830) 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 -- 172.21.15.83:0/3139268408 learned_addr learned my addr 172.21.15.83:0/3139268408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cbb71640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab7bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cab6f640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab81b20 0x55a5dab7ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 -- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab81b20 msgr2=0x55a5dab7ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55a5dab81b20 0x55a5dab7ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 -- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 msgr2=0x55a5dab7bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5dac0d550 0x55a5dab7bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 -- 172.21.15.83:0/3139268408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dab3a5f0 con 0x55a5dab814b0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46cb370640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 0x55a5dab3bb30 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a3f0 tx=0x7f46c401a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46c8b6b640 1 -- 172.21.15.83:0/3139268408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40aa970 con 0x55a5dab814b0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46c8b6b640 1 -- 172.21.15.83:0/3139268408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c40bbc10 con 0x55a5dab814b0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5dab3a890 con 0x55a5dab814b0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46c8b6b640 1 -- 172.21.15.83:0/3139268408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40c5470 con 0x55a5dab814b0 2024-05-17T22:07:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.610+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dab60840 con 0x55a5dab814b0 2024-05-17T22:07:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46c8b6b640 1 -- 172.21.15.83:0/3139268408 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f46c40c5610 con 0x55a5dab814b0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46c8b6b640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f46b806de10 0x7f46b8070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46cab6f640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f46b806de10 0x7f46b8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46c8b6b640 1 -- 172.21.15.83:0/3139268408 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f46c412d080 con 0x55a5dab814b0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46d188e080 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55a5dab395b0 0x55a5dab399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55a5dab39f00 con 0x55a5dab395b0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46cbb71640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55a5dab395b0 0x55a5dab399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46cab6f640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f46b806de10 0x7f46b8070320 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f46c0001230 tx=0x7f46c00013b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.614+0000 7f46cbb71640 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55a5dab395b0 0x55a5dab399c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46cbb71640 1 -- 172.21.15.83:0/3139268408 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v110'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46bc007600 con 0x55a5dab395b0 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55a5dab395b0 msgr2=0x55a5dab399c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55a5dab395b0 0x55a5dab399c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f46b806de10 msgr2=0x7f46b8070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f46b806de10 0x7f46b8070320 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f46c0001230 tx=0x7f46c00013b0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 msgr2=0x55a5dab3bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 --2- 172.21.15.83:0/3139268408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5dab814b0 0x55a5dab3bb30 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f46c400a3f0 tx=0x7f46c401a6b0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46cbb71640 1 -- 172.21.15.83:0/3139268408 reap_dead start 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 shutdown_connections 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 >> 172.21.15.83:0/3139268408 conn(0x55a5dab24e40 msgr2=0x55a5dac0ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 shutdown_connections 2024-05-17T22:07:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.618+0000 7f46d188e080 1 -- 172.21.15.83:0/3139268408 wait complete. 2024-05-17T22:07:40.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:07:40.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 Processor -- start 2024-05-17T22:07:40.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 -- start start 2024-05-17T22:07:40.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0d00b88b0 0x7fe0d00d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d009be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d009c350 con 0x7fe0d00ce130 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d00d1a40 con 0x7fe0d00b88b0 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0d5f8b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d009d040 con 0x7fe0d009ba00 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0d00b88b0 0x7fe0d00d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0d00b88b0 0x7fe0d00d11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51836/0 (socket says 172.21.15.83:51836) 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 -- 172.21.15.83:0/2861165959 learned_addr learned my addr 172.21.15.83:0/2861165959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0ceffd640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 -- 172.21.15.83:0/2861165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 msgr2=0x7fe0d009be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d009be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 -- 172.21.15.83:0/2861165959 >> v1:172.21.15.155:6789/0 conn(0x7fe0d00ce130 legacy=0x7fe0d00ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:40.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.770+0000 7fe0cf7fe640 1 -- 172.21.15.83:0/2861165959 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d009c710 con 0x7fe0d00b88b0 2024-05-17T22:07:40.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00d11f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00098a0 tx=0x7fe0c001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f9e7d1a7e0a0fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0ce7fc640 1 -- 172.21.15.83:0/2861165959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00b0070 con 0x7fe0d00b88b0 2024-05-17T22:07:40.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0ce7fc640 1 -- 172.21.15.83:0/2861165959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c00ab600 con 0x7fe0d00b88b0 2024-05-17T22:07:40.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0ce7fc640 1 -- 172.21.15.83:0/2861165959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00ab9d0 con 0x7fe0d00b88b0 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/2861165959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00b88b0 msgr2=0x7fe0d00d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00d11f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00098a0 tx=0x7fe0c001c240 comp rx=0 tx=0).stop 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/2861165959 shutdown_connections 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d009be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/2861165959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00d11f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/2861165959 >> 172.21.15.83:0/2861165959 conn(0x7fe0d00d3620 msgr2=0x7fe0d00d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/2861165959 shutdown_connections 2024-05-17T22:07:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.774+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/2861165959 wait complete. 2024-05-17T22:07:40.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 Processor -- start 2024-05-17T22:07:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- start start 2024-05-17T22:07:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d01f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00b2dc0 con 0x7fe0d00ce130 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00cf060 con 0x7fe0d00b88b0 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d00b2f40 con 0x7fe0d009ba00 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51846/0 (socket says 172.21.15.83:51846) 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 -- 172.21.15.83:0/53255810 learned_addr learned my addr 172.21.15.83:0/53255810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d01f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0ceffd640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 -- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 msgr2=0x7fe0d01f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d01f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 -- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 msgr2=0x7fe0d00b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 -- 172.21.15.83:0/53255810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d015bd00 con 0x7fe0d00ce130 2024-05-17T22:07:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0ceffd640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d01f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0cffff640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800ba40 tx=0x7fe0c800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8003b80 con 0x7fe0d00ce130 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8003d20 con 0x7fe0d00ce130 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d015bfe0 con 0x7fe0d00ce130 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800cb60 con 0x7fe0d00ce130 2024-05-17T22:07:40.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d00b7970 con 0x7fe0d00ce130 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.778+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d009be10 con 0x7fe0d00ce130 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.782+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe0c8006d10 con 0x7fe0d00ce130 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.782+0000 7fe0b7fff640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 0x7fe0bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.782+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fe0c808ed00 con 0x7fe0d00ce130 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.782+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 0x7fe0bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.782+0000 7fe0cf7fe640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 0x7fe0bc071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00ad000 tx=0x7fe0c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.786+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe0c805bcb0 con 0x7fe0d00ce130 2024-05-17T22:07:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:40.974+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe0d00a17c0 con 0x7fe0d00ce130 2024-05-17T22:07:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.146+0000 7fe0b7fff640 1 -- 172.21.15.83:0/53255810 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v99) ==== 162+0+0 (secure 0 0 0) 0x7fe0c805b5b0 con 0x7fe0d00ce130 2024-05-17T22:07:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:07:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 msgr2=0x7fe0bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 0x7fe0bc071940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00ad000 tx=0x7fe0c0008040 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 msgr2=0x7fe0d00b69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800ba40 tx=0x7fe0c800bf00 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 shutdown_connections 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0d00ce130 0x7fe0d00b69b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe0bc06f430 0x7fe0bc071940 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d00b88b0 0x7fe0d00b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 --2- 172.21.15.83:0/53255810 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0d009ba00 0x7fe0d01f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 >> 172.21.15.83:0/53255810 conn(0x7fe0d00d3620 msgr2=0x7fe0d015b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 shutdown_connections 2024-05-17T22:07:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.150+0000 7fe0d5f8b640 1 -- 172.21.15.83:0/53255810 wait complete. 2024-05-17T22:07:41.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:41.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 Processor -- start 2024-05-17T22:07:41.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 -- start start 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72c80af7c0 0x7f72c8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72c815d8d0 0x7f72c815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c80700e0 con 0x7f72c806d7a0 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f72c80709f0 con 0x7f72c80af7c0 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cf767640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f72c80aff10 con 0x7f72c815d8d0 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cd503640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72c80af7c0 0x7f72c8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72ccd02640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f72c806d7a0 0x7f72c806dbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46882/0 (socket says 172.21.15.83:46882) 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72ccd02640 1 -- 172.21.15.83:0/1853699411 learned_addr learned my addr 172.21.15.83:0/1853699411 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cd503640 1 -- 172.21.15.83:0/1853699411 >> v2:172.21.15.155:3300/0 conn(0x7f72c815d8d0 msgr2=0x7f72c815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cd503640 1 --2- 172.21.15.83:0/1853699411 >> v2:172.21.15.155:3300/0 conn(0x7f72c815d8d0 0x7f72c815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cd503640 1 -- 172.21.15.83:0/1853699411 >> v1:172.21.15.83:6789/0 conn(0x7f72c806d7a0 legacy=0x7f72c806dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.322+0000 7f72cd503640 1 -- 172.21.15.83:0/1853699411 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c80703a0 con 0x7f72c80af7c0 2024-05-17T22:07:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cd503640 1 --2- 172.21.15.83:0/1853699411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c80af7c0 0x7f72c8070e80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f72c8068aa0 tx=0x7f72c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c40620efccb5a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72bffff640 1 -- 172.21.15.83:0/1853699411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40ab430 con 0x7f72c80af7c0 2024-05-17T22:07:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72bffff640 1 -- 172.21.15.83:0/1853699411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c40b0070 con 0x7f72c80af7c0 2024-05-17T22:07:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72bffff640 1 -- 172.21.15.83:0/1853699411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c40ab7f0 con 0x7f72c80af7c0 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 -- 172.21.15.83:0/1853699411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c80af7c0 msgr2=0x7f72c8070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 --2- 172.21.15.83:0/1853699411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c80af7c0 0x7f72c8070e80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f72c8068aa0 tx=0x7f72c401c450 comp rx=0 tx=0).stop 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 -- 172.21.15.83:0/1853699411 shutdown_connections 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 --2- 172.21.15.83:0/1853699411 >> v2:172.21.15.155:3300/0 conn(0x7f72c815d8d0 0x7f72c815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 --2- 172.21.15.83:0/1853699411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c80af7c0 0x7f72c8070e80 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 -- 172.21.15.83:0/1853699411 >> 172.21.15.83:0/1853699411 conn(0x7f72c8073670 msgr2=0x7f72c8073a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 -- 172.21.15.83:0/1853699411 shutdown_connections 2024-05-17T22:07:41.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.326+0000 7f72cf767640 1 -- 172.21.15.83:0/1853699411 wait complete. 2024-05-17T22:07:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 Processor -- start 2024-05-17T22:07:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- start start 2024-05-17T22:07:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 0x7f72c815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c80af7c0 0x7f72c809da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8071540 con 0x7f72c815d8d0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72c80713c0 con 0x7f72c80af7c0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72c8068b30 con 0x7f72c806d7a0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51878/0 (socket says 172.21.15.83:51878) 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 -- 172.21.15.83:0/2936294103 learned_addr learned my addr 172.21.15.83:0/2936294103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72ccd02640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c80af7c0 0x7f72c809da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cd503640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 0x7f72c815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 -- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 msgr2=0x7f72c815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 0x7f72c815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 -- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c80af7c0 msgr2=0x7f72c809da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c80af7c0 0x7f72c809da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 -- 172.21.15.83:0/2936294103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c8155a10 con 0x7f72c815d8d0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cd503640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 0x7f72c815cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cdd04640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f72c000b9e0 tx=0x7f72c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0019070 con 0x7f72c815d8d0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c000ca60 con 0x7f72c815d8d0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c8155c80 con 0x7f72c815d8d0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0014440 con 0x7f72c815d8d0 2024-05-17T22:07:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c815d160 con 0x7f72c815d8d0 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.330+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c8070e80 con 0x7f72c815d8d0 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.334+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f72c001c070 con 0x7f72c815d8d0 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.334+0000 7f72bdffb640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 0x7f72ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.334+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f72c008f0e0 con 0x7f72c815d8d0 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.334+0000 7f72cd503640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 0x7f72ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.334+0000 7f72cd503640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 0x7f72ac071910 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72c40af000 tx=0x7f72c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.338+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f72c005bed0 con 0x7f72c815d8d0 2024-05-17T22:07:41.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72c806dbb0 con 0x7f72c815d8d0 2024-05-17T22:07:41.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72bdffb640 1 -- 172.21.15.83:0/2936294103 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5175 (secure 0 0 0) 0x7f72c005b7d0 con 0x7f72c815d8d0 2024-05-17T22:07:41.528 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:41.528 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:41.149078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7980":{"gid":7980,"name":"c","rank":0,"incarnation":99,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-17T22:07:41.528 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 99 2024-05-17T22:07:41.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 msgr2=0x7f72ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 0x7f72ac071910 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72c40af000 tx=0x7f72c4008040 comp rx=0 tx=0).stop 2024-05-17T22:07:41.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 msgr2=0x7f72c809df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:41.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f72c000b9e0 tx=0x7f72c000bea0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 shutdown_connections 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72c815d8d0 0x7f72c809df40 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.534+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72ac06f400 0x7f72ac071910 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.538+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72c80af7c0 0x7f72c809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.538+0000 7f72cf767640 1 --2- 172.21.15.83:0/2936294103 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72c806d7a0 0x7f72c815cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.538+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 >> 172.21.15.83:0/2936294103 conn(0x7f72c8073670 msgr2=0x7f72c81534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.538+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 shutdown_connections 2024-05-17T22:07:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:41.538+0000 7f72cf767640 1 -- 172.21.15.83:0/2936294103 wait complete. 2024-05-17T22:07:41.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:07:41.149078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7980}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7980': {'gid': 7980, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/78757339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 78757339}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 78757339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:07:41.953 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7980:up:replay 2024-05-17T22:07:42.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 Processor -- start 2024-05-17T22:07:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 -- start start 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59c8072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c80d1950 con 0x7f59c809c0a0 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c80d1670 con 0x7f59c80cf2e0 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cf78b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59c8071280 con 0x7f59c809c570 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59ccd26640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f59c809c0a0 0x7f59c8071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46884/0 (socket says 172.21.15.83:46884) 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59ccd26640 1 -- 172.21.15.83:0/2859550372 learned_addr learned my addr 172.21.15.83:0/2859550372 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59cdd28640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59c8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609687780 0 0) 0x7f59c80d1950 con 0x7f59c809c0a0 2024-05-17T22:07:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.118+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b4003530 con 0x7f59c809c0a0 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1730296162 0 0) 0x7f59b4003530 con 0x7f59c809c0a0 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 msgr2=0x7f59c8072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59c8072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 >> v1:172.21.15.155:6789/0 conn(0x7f59c80cf2e0 legacy=0x7f59c809b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c8071560 con 0x7f59c809c0a0 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f59c4001700 con 0x7f59c809c0a0 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 >> v1:172.21.15.83:6789/0 conn(0x7f59c809c0a0 legacy=0x7f59c8071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4003a40 0x7f59b4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4004810 0x7f59b4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59b40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8071280 con 0x7f59b4004810 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59b400bc50 con 0x7f59b4003a40 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59b400be50 con 0x7f59c809c570 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4004810 0x7f59b4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cd527640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4003a40 0x7f59b4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cdd28640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59b40052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 -- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 msgr2=0x7f59b40052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c809c570 0x7f59b40052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 -- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4003a40 msgr2=0x7f59b4003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4003a40 0x7f59b4003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 -- 172.21.15.83:0/2859550372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b4003530 con 0x7f59b4004810 2024-05-17T22:07:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59ccd26640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4004810 0x7f59b4004c20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c210 tx=0x7f59c401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509fd86e99f674bd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4007d20 con 0x7f59b4004810 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c40a9020 con 0x7f59b4004810 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59bbfff640 1 -- 172.21.15.83:0/2859550372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c40a5ba0 con 0x7f59b4004810 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cf78b640 1 -- 172.21.15.83:0/2859550372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4004810 msgr2=0x7f59b4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cf78b640 1 --2- 172.21.15.83:0/2859550372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4004810 0x7f59b4004c20 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c210 tx=0x7f59c401cbc0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cdd28640 1 -- 172.21.15.83:0/2859550372 reap_dead start 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cf78b640 1 -- 172.21.15.83:0/2859550372 shutdown_connections 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.122+0000 7f59cf78b640 1 -- 172.21.15.83:0/2859550372 >> 172.21.15.83:0/2859550372 conn(0x7f59c80b5120 msgr2=0x7f59c80b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- 172.21.15.83:0/2859550372 shutdown_connections 2024-05-17T22:07:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- 172.21.15.83:0/2859550372 wait complete. 2024-05-17T22:07:43.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 Processor -- start 2024-05-17T22:07:43.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- start start 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 0x7f59c81526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c80c6cc0 0x7f59c8152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d1380 con 0x7f59c81f7150 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d1200 con 0x7f59b4004810 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cf78b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80b8de0 con 0x7f59c80c6cc0 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51892/0 (socket says 172.21.15.83:51892) 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 -- 172.21.15.83:0/4082733980 learned_addr learned my addr 172.21.15.83:0/4082733980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59ccd26640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c80c6cc0 0x7f59c8152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cd527640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 0x7f59c81526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 -- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c80c6cc0 msgr2=0x7f59c8152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c80c6cc0 0x7f59c8152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 -- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 msgr2=0x7f59c81526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 0x7f59c81526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 -- 172.21.15.83:0/4082733980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80c73a0 con 0x7f59c81f7150 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cd527640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 0x7f59c81526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.126+0000 7f59cdd28640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f59c80d1130 tx=0x7f59c000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c001c070 con 0x7f59c81f7150 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c0017400 con 0x7f59c81f7150 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c80a0df0 con 0x7f59c81f7150 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c00177f0 con 0x7f59c81f7150 2024-05-17T22:07:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c80a13b0 con 0x7f59c81f7150 2024-05-17T22:07:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c809b990 con 0x7f59c81f7150 2024-05-17T22:07:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59c001f070 con 0x7f59c81f7150 2024-05-17T22:07:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 0x7f59ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f59c008f510 con 0x7f59c81f7150 2024-05-17T22:07:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59cd527640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 0x7f59ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:43.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.130+0000 7f59cd527640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 0x7f59ac071910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0016a0 tx=0x7f59bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.134+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f59c0012070 con 0x7f59c81f7150 2024-05-17T22:07:43.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.326+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59c80c7130 con 0x7f59c81f7150 2024-05-17T22:07:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.326+0000 7f59b9ffb640 1 -- 172.21.15.83:0/4082733980 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5178 (secure 0 0 0) 0x7f59c005c3a0 con 0x7f59c81f7150 2024-05-17T22:07:43.321 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:43.322 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:42.167179+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7980":{"gid":7980,"name":"c","rank":0,"incarnation":99,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-17T22:07:43.322 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 101 2024-05-17T22:07:43.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 msgr2=0x7f59ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 0x7f59ac071910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0016a0 tx=0x7f59bc009040 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 msgr2=0x7f59c81f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f59c80d1130 tx=0x7f59c000f4a0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 shutdown_connections 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c81f7150 0x7f59c81f7560 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59c80c6cc0 0x7f59c8152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f59ac06f400 0x7f59ac071910 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 --2- 172.21.15.83:0/4082733980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59b4004810 0x7f59c81526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 >> 172.21.15.83:0/4082733980 conn(0x7f59c80b5120 msgr2=0x7f59c80b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 shutdown_connections 2024-05-17T22:07:43.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:43.330+0000 7f59cf78b640 1 -- 172.21.15.83:0/4082733980 wait complete. 2024-05-17T22:07:43.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:07:42.167179+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7980}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7980': {'gid': 7980, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:rejoin', 'state_seq': 11, 'addr': '172.21.15.83:6837/78757339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 78757339}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 78757339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:07:43.746 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7980:up:rejoin 2024-05-17T22:07:44.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 Processor -- start 2024-05-17T22:07:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 -- start start 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15841461c0 0x7f158415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f158415e6b0 0x7f158406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15841465d0 con 0x7f1584070510 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f158406ef30 con 0x7f158415e6b0 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f158a4b9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f158406f210 con 0x7f15841461c0 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f1588a56640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1584070510 0x7f15841605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46900/0 (socket says 172.21.15.83:46900) 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f1588a56640 1 -- 172.21.15.83:0/3288452631 learned_addr learned my addr 172.21.15.83:0/3288452631 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f15837fe640 1 --2- 172.21.15.83:0/3288452631 >> v2:172.21.15.83:3300/0 conn(0x7f158415e6b0 0x7f158406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f1583fff640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15841461c0 0x7f158415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f1582ffd640 1 -- 172.21.15.83:0/3288452631 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 303781565 0 0) 0x7f15841465d0 con 0x7f1584070510 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f1582ffd640 1 -- 172.21.15.83:0/3288452631 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f156c003530 con 0x7f1584070510 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f15837fe640 1 -- 172.21.15.83:0/3288452631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15841461c0 msgr2=0x7f158415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f15837fe640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15841461c0 0x7f158415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f15837fe640 1 -- 172.21.15.83:0/3288452631 >> v1:172.21.15.83:6789/0 conn(0x7f1584070510 legacy=0x7f15841605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:44.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.918+0000 7f15837fe640 1 -- 172.21.15.83:0/3288452631 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1584070920 con 0x7f158415e6b0 2024-05-17T22:07:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f15837fe640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f158415e6b0 0x7f158406ffd0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f15780097c0 tx=0x7f157801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaad4f880609d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f1582ffd640 1 -- 172.21.15.83:0/3288452631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15780a7070 con 0x7f158415e6b0 2024-05-17T22:07:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f1582ffd640 1 -- 172.21.15.83:0/3288452631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15780b0070 con 0x7f158415e6b0 2024-05-17T22:07:44.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f1582ffd640 1 -- 172.21.15.83:0/3288452631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15780ab890 con 0x7f158415e6b0 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 -- 172.21.15.83:0/3288452631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f158415e6b0 msgr2=0x7f158406ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f158415e6b0 0x7f158406ffd0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f15780097c0 tx=0x7f157801c470 comp rx=0 tx=0).stop 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 -- 172.21.15.83:0/3288452631 shutdown_connections 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f158415e6b0 0x7f158406ffd0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 --2- 172.21.15.83:0/3288452631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15841461c0 0x7f158415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 -- 172.21.15.83:0/3288452631 >> 172.21.15.83:0/3288452631 conn(0x7f158406cd90 msgr2=0x7f15840b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 -- 172.21.15.83:0/3288452631 shutdown_connections 2024-05-17T22:07:44.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.922+0000 7f158a4b9640 1 -- 172.21.15.83:0/3288452631 wait complete. 2024-05-17T22:07:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 Processor -- start 2024-05-17T22:07:44.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- start start 2024-05-17T22:07:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15841461c0 0x7f1584072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158415e6b0 0x7f1584073370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1584160e30 con 0x7f1584070510 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1584160cb0 con 0x7f15841461c0 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f15840d10a0 con 0x7f158415e6b0 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51904/0 (socket says 172.21.15.83:51904) 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 -- 172.21.15.83:0/2264250298 learned_addr learned my addr 172.21.15.83:0/2264250298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f15837fe640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15841461c0 0x7f1584072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1588a56640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158415e6b0 0x7f1584073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 -- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158415e6b0 msgr2=0x7f1584073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158415e6b0 0x7f1584073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 -- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15841461c0 msgr2=0x7f1584072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:44.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15841461c0 0x7f1584072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 -- 172.21.15.83:0/2264250298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15840aacd0 con 0x7f1584070510 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1583fff640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f157400d600 tx=0x7f157400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574004240 con 0x7f1584070510 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15740043e0 con 0x7f1584070510 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574003840 con 0x7f1584070510 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15840aaf70 con 0x7f1584070510 2024-05-17T22:07:44.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15840ab530 con 0x7f1584070510 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.926+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158415e170 con 0x7f1584070510 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.930+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15740039e0 con 0x7f1584070510 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.930+0000 7f1580ff9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 0x7f1570071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.930+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f157408ff30 con 0x7f1584070510 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.930+0000 7f15837fe640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 0x7f1570071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:44.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.930+0000 7f15837fe640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 0x7f1570071940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f15780af000 tx=0x7f15780a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:44.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:44.934+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f157405cf30 con 0x7f1584070510 2024-05-17T22:07:45.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15840b41a0 con 0x7f1584070510 2024-05-17T22:07:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f1580ff9640 1 -- 172.21.15.83:0/2264250298 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5185 (secure 0 0 0) 0x7f157405c830 con 0x7f1584070510 2024-05-17T22:07:45.124 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:45.124 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:44.164826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7980":{"gid":7980,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7980,"qdb_cluster":[7980]},"id":8}]} 2024-05-17T22:07:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 102 2024-05-17T22:07:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 msgr2=0x7f1570071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 0x7f1570071940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f15780af000 tx=0x7f15780a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 msgr2=0x7f1584158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.130+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f157400d600 tx=0x7f157400dac0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 shutdown_connections 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f158415e6b0 0x7f1584073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f157006f430 0x7f1570071940 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15841461c0 0x7f1584072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 --2- 172.21.15.83:0/2264250298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1584070510 0x7f1584158af0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 >> 172.21.15.83:0/2264250298 conn(0x7f158406cd90 msgr2=0x7f15840b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 shutdown_connections 2024-05-17T22:07:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.134+0000 7f158a4b9640 1 -- 172.21.15.83:0/2264250298 wait complete. 2024-05-17T22:07:45.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:07:44.164826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7980}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7980': {'gid': 7980, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.83:6837/78757339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 78757339}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 78757339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7980, 'qdb_cluster': [7980]} 2024-05-17T22:07:45.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:07:45.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-17T22:07:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 Processor -- start 2024-05-17T22:07:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 -- start start 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9040cf830 0x7fb9040b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb90415d700 0x7fb9041663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9040b5640 con 0x7fb9040cdf40 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb904070870 con 0x7fb9040cf830 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb90a4a9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fb9040b58e0 con 0x7fb90415d700 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9040cf830 0x7fb9040b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9040cf830 0x7fb9040b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51910/0 (socket says 172.21.15.83:51910) 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 -- 172.21.15.83:0/1528171772 learned_addr learned my addr 172.21.15.83:0/1528171772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 -- 172.21.15.83:0/1528171772 >> v2:172.21.15.155:3300/0 conn(0x7fb90415d700 msgr2=0x7fb9041663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 --2- 172.21.15.83:0/1528171772 >> v2:172.21.15.155:3300/0 conn(0x7fb90415d700 0x7fb9041663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 -- 172.21.15.83:0/1528171772 >> v1:172.21.15.155:6789/0 conn(0x7fb9040cdf40 legacy=0x7fb90415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.722+0000 7fb903fff640 1 -- 172.21.15.83:0/1528171772 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90406f060 con 0x7fb9040cf830 2024-05-17T22:07:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb903fff640 1 --2- 172.21.15.83:0/1528171772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cf830 0x7fb9040b3aa0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb90406d6e0 tx=0x7fb8f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa25acf6768241b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb9037fe640 1 -- 172.21.15.83:0/1528171772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b0070 con 0x7fb9040cf830 2024-05-17T22:07:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb9037fe640 1 -- 172.21.15.83:0/1528171772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40ab600 con 0x7fb9040cf830 2024-05-17T22:07:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb9037fe640 1 -- 172.21.15.83:0/1528171772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40ab9d0 con 0x7fb9040cf830 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- 172.21.15.83:0/1528171772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cf830 msgr2=0x7fb9040b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/1528171772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cf830 0x7fb9040b3aa0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fb90406d6e0 tx=0x7fb8f401c520 comp rx=0 tx=0).stop 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- 172.21.15.83:0/1528171772 shutdown_connections 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/1528171772 >> v2:172.21.15.155:3300/0 conn(0x7fb90415d700 0x7fb9041663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/1528171772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cf830 0x7fb9040b3aa0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- 172.21.15.83:0/1528171772 >> 172.21.15.83:0/1528171772 conn(0x7fb90406cd90 msgr2=0x7fb9040b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- 172.21.15.83:0/1528171772 shutdown_connections 2024-05-17T22:07:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- 172.21.15.83:0/1528171772 wait complete. 2024-05-17T22:07:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 Processor -- start 2024-05-17T22:07:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.726+0000 7fb90a4a9640 1 -- start start 2024-05-17T22:07:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9040cf830 0x7fb9040cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 0x7fb9040ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041692b0 con 0x7fb9040cdf40 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb904169130 con 0x7fb90415d700 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb904169430 con 0x7fb9040cf830 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51920/0 (socket says 172.21.15.83:51920) 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 -- 172.21.15.83:0/3950846599 learned_addr learned my addr 172.21.15.83:0/3950846599 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb908a46640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 0x7fb9040ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9040cf830 0x7fb9040cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 -- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9040cf830 msgr2=0x7fb9040cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9040cf830 0x7fb9040cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 -- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 msgr2=0x7fb9040ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 0x7fb9040ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 -- 172.21.15.83:0/3950846599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9040abb00 con 0x7fb9040cdf40 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb908a46640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 0x7fb9040ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb903fff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40ae040 tx=0x7fb8f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb9017fa640 1 -- 172.21.15.83:0/3950846599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b0040 con 0x7fb9040cdf40 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb9017fa640 1 -- 172.21.15.83:0/3950846599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40bc070 con 0x7fb9040cdf40 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb9017fa640 1 -- 172.21.15.83:0/3950846599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b6860 con 0x7fb9040cdf40 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9040abf80 con 0x7fb9040cdf40 2024-05-17T22:07:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9040ae8b0 con 0x7fb9040cdf40 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.730+0000 7fb9017fa640 1 -- 172.21.15.83:0/3950846599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb8f40b6a00 con 0x7fb9040cdf40 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb9017fa640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 0x7fb8e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb9017fa640 1 -- 172.21.15.83:0/3950846599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fb8f41295a0 con 0x7fb9040cdf40 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 0x7fb8e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 0x7fb8e8071910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb9041fc080 tx=0x7fb8ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 Processor -- start 2024-05-17T22:07:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 -- start start 2024-05-17T22:07:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ab6600 0x557046c06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 0x557046c07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557046aa4ca0 con 0x557046ac0fa0 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557046a17110 con 0x557046a9fd30 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557046a06cb0 con 0x557046ab6600 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51930/0 (socket says 172.21.15.83:51930) 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 -- 172.21.15.83:0/3907641488 learned_addr learned my addr 172.21.15.83:0/3907641488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb908a46640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 0x557046c07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ab6600 0x557046c06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 -- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ab6600 msgr2=0x557046c06df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ab6600 0x557046c06df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 -- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 msgr2=0x557046c07330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 0x557046c07330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 -- 172.21.15.83:0/3907641488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570469cb370 con 0x557046ac0fa0 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb908a46640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 0x557046c07330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb903fff640 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40f5cc0 tx=0x7fb8f4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efff9115285c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8f9ffb640 1 -- 172.21.15.83:0/3907641488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40f6430 con 0x557046ac0fa0 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8f9ffb640 1 -- 172.21.15.83:0/3907641488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f410c070 con 0x557046ac0fa0 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.734+0000 7fb8f9ffb640 1 -- 172.21.15.83:0/3907641488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40f5d30 con 0x557046ac0fa0 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/3907641488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 msgr2=0x557046c068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40f5cc0 tx=0x7fb8f4009240 comp rx=0 tx=0).stop 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/3907641488 shutdown_connections 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046a9fd30 0x557046c07330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ab6600 0x557046c06df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- 172.21.15.83:0/3907641488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046c068b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/3907641488 >> 172.21.15.83:0/3907641488 conn(0x5570466c8460 msgr2=0x557046bbbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/3907641488 shutdown_connections 2024-05-17T22:07:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/3907641488 wait complete. 2024-05-17T22:07:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- start start 2024-05-17T22:07:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 Processor -- start 2024-05-17T22:07:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046bc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046bc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570469c2d40 con 0x557046ac0fa0 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557046a0acf0 con 0x557046ab6600 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5570469afe90 con 0x557046a9fd30 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 10 client.0 _ll_register_callbacks cb 0x5570469e4fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51932/0 (socket says 172.21.15.83:51932) 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 -- 172.21.15.83:0/2735210875 learned_addr learned my addr 172.21.15.83:0/2735210875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046bc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb903fff640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046bc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 -- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 msgr2=0x557046bc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046bc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 -- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 msgr2=0x557046bc37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046bc37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 -- 172.21.15.83:0/2735210875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570469d56e0 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb908a46640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00da40 tx=0x7fb8fc00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694174796b878bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00bdc0 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fb8fc017420 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f410c070 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc021050 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 10 client.8253 fetch_fsmap learned FSMap version 102 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.738+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x5570469c5690 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f410c070 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7fb8fc024020 con 0x557046ac0fa0 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 10 client.8253 fetch_fsmap finished waiting for FSMap version 102 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 10 client.8253 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:07:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 20 client.8253 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x5570466ad7d0 0x557046c0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 4 client.8253 mds_command: new command op to 7980 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 --> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570469a2a00 con 0x5570466ad7d0 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb903fff640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x5570466ad7d0 0x557046c0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 10 client.8253 ms_handle_connect on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb903fff640 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x5570466ad7d0 0x557046c0d920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4c9614d2f27e94 server_cookie=2eb7e8b19d3ccb07 in_seq=0 out_seq=0 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/2735210875 <== mds.0 v2:172.21.15.83:6835/78757339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5570469a2a00 con 0x5570466ad7d0 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 10 client.8253 handle_command_reply: tid=0 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 client.8253 shutdown 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 msgr2=0x557046bc8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00da40 tx=0x7fb8fc00df00 comp rx=0 tx=0).stop 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 shutdown_connections 2024-05-17T22:07:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x5570466ad7d0 0x557046c0d920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ac0fa0 0x557046bc8460 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb8f2ffd640 0 client.8253 ms_handle_reset on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046bc37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 --2- 172.21.15.83:0/2735210875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046bc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 >> 172.21.15.83:0/2735210875 conn(0x5570466c8460 msgr2=0x557046bbbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 shutdown_connections 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 1 -- 172.21.15.83:0/2735210875 wait complete. 2024-05-17T22:07:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.742+0000 7fb911723000 20 client.8253 trim_cache size 0 max 16384 2024-05-17T22:07:45.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 Processor -- start 2024-05-17T22:07:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- start start 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046c1e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ac0fa0 0x557046bc2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9042045e0 con 0x557046a9fd30 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040ab0d0 con 0x557046ab6600 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fb90415a260 con 0x557046ac0fa0 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51946/0 (socket says 172.21.15.83:51946) 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 -- 172.21.15.83:0/2451683140 learned_addr learned my addr 172.21.15.83:0/2451683140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 -- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ac0fa0 msgr2=0x557046bc2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ac0fa0 0x557046bc2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 -- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 msgr2=0x557046c1e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046c1e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 -- 172.21.15.83:0/2451683140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557046a8dcb0 con 0x557046a9fd30 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb903fff640 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb8f41298c0 tx=0x7fb8f4129070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35617fc7d29301de server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb8f97fa640 1 -- 172.21.15.83:0/2451683140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40fa0e0 con 0x557046a9fd30 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb8f97fa640 1 -- 172.21.15.83:0/2451683140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f410c040 con 0x557046a9fd30 2024-05-17T22:07:45.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb8f97fa640 1 -- 172.21.15.83:0/2451683140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40feb90 con 0x557046a9fd30 2024-05-17T22:07:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- 172.21.15.83:0/2451683140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 msgr2=0x557046c1deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb8f41298c0 tx=0x7fb8f4129070 comp rx=0 tx=0).stop 2024-05-17T22:07:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- 172.21.15.83:0/2451683140 shutdown_connections 2024-05-17T22:07:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046ac0fa0 0x557046bc2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ab6600 0x557046c1e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 --2- 172.21.15.83:0/2451683140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046a9fd30 0x557046c1deb0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- 172.21.15.83:0/2451683140 >> 172.21.15.83:0/2451683140 conn(0x5570466c8460 msgr2=0x557046c05d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- 172.21.15.83:0/2451683140 shutdown_connections 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.762+0000 7fb911723000 1 -- 172.21.15.83:0/2451683140 wait complete. 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- start start 2024-05-17T22:07:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 Processor -- start 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046c78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 0x557046c798c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557046a0a000 con 0x557046ab6600 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x557046bc6c20 con 0x557046ac0fa0 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x557046a15ee0 con 0x557046a9fd30 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 10 client.0 _ll_register_callbacks cb 0x7fb8fc0076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51962/0 (socket says 172.21.15.83:51962) 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 -- 172.21.15.83:0/733056965 learned_addr learned my addr 172.21.15.83:0/733056965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb908a46640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 0x557046c798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb903fff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046c78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 -- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 msgr2=0x557046c78e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046c78e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 -- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 msgr2=0x557046c798c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 0x557046c798c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 -- 172.21.15.83:0/733056965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570469c64f0 con 0x557046ab6600 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb908a46640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 0x557046c798c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8fbfff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00ccb0 tx=0x7fb8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7894d19d0286790 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec023020 con 0x557046ab6600 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fb8ec01f630 con 0x557046ab6600 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f410c040 con 0x557046ab6600 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec02d050 con 0x557046ab6600 2024-05-17T22:07:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 10 client.8259 fetch_fsmap learned FSMap version 102 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x5570469d04f0 con 0x557046ab6600 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f410c040 con 0x557046ab6600 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7fb8ec030020 con 0x557046ab6600 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 10 client.8259 fetch_fsmap finished waiting for FSMap version 102 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 10 client.8259 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 20 client.8259 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x55704669b830 0x557046bbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 4 client.8259 mds_command: new command op to 7980 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 --> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] -- command(tid 0: {"prefix": "status"}) -- 0x557046a19060 con 0x55704669b830 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb903fff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x55704669b830 0x557046bbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb8f2ffd640 10 client.8259 ms_handle_connect on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.766+0000 7fb903fff640 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x55704669b830 0x557046bbdc10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228516b11515c8df server_cookie=5522a5d6d6e25313 in_seq=0 out_seq=0 2024-05-17T22:07:45.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb8f2ffd640 1 -- 172.21.15.83:0/733056965 <== mds.0 v2:172.21.15.83:6835/78757339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x557046a19060 con 0x55704669b830 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb8f2ffd640 10 client.8259 handle_command_reply: tid=0 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 client.8259 shutdown 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 msgr2=0x557046c79380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00ccb0 tx=0x7fb8ec0046c0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 shutdown_connections 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x55704669b830 0x557046bbdc10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557046ac0fa0 0x557046c798c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557046ab6600 0x557046c79380 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb8f2ffd640 0 client.8259 ms_handle_reset on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 --2- 172.21.15.83:0/733056965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x557046a9fd30 0x557046c78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 >> 172.21.15.83:0/733056965 conn(0x5570466c8460 msgr2=0x557046c1cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 shutdown_connections 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 1 -- 172.21.15.83:0/733056965 wait complete. 2024-05-17T22:07:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb911723000 20 client.8259 trim_cache size 0 max 16384 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7980, 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.6210783580000001, 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 102, 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 110, 2024-05-17T22:07:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 110, 2024-05-17T22:07:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 38.202943087000001 2024-05-17T22:07:45.769 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:45.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 msgr2=0x7fb8e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 0x7fb8e8071910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb9041fc080 tx=0x7fb8ec008040 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 msgr2=0x7fb9040cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40ae040 tx=0x7fb8f401ad60 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 shutdown_connections 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fb8e806f400 0x7fb8e8071910 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb90415d700 0x7fb9040ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9040cf830 0x7fb9040cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 --2- 172.21.15.83:0/3950846599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9040cdf40 0x7fb9040cbc60 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 >> 172.21.15.83:0/3950846599 conn(0x7fb90406cd90 msgr2=0x7fb90406f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 shutdown_connections 2024-05-17T22:07:45.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.770+0000 7fb90a4a9640 1 -- 172.21.15.83:0/3950846599 wait complete. 2024-05-17T22:07:45.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:07:45.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.930+0000 7fcecf58a640 1 Processor -- start 2024-05-17T22:07:45.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecf58a640 1 -- start start 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecf58a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcec81605a0 0x7fcec81609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecf58a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec81465d0 con 0x7fcec806ffd0 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecf58a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec8160ef0 con 0x7fcec81461c0 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecf58a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fcec806ef30 con 0x7fcec81605a0 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fceccb25640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcec806ffd0 0x7fcec80703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46902/0 (socket says 172.21.15.83:46902) 2024-05-17T22:07:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fceccb25640 1 -- 172.21.15.83:0/1139606426 learned_addr learned my addr 172.21.15.83:0/1139606426 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 --2- 172.21.15.83:0/1139606426 >> v2:172.21.15.155:3300/0 conn(0x7fcec81605a0 0x7fcec81609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768825566 0 0) 0x7fcec81465d0 con 0x7fcec806ffd0 2024-05-17T22:07:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb4003530 con 0x7fcec806ffd0 2024-05-17T22:07:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3622114976 0 0) 0x7fceb4003530 con 0x7fcec806ffd0 2024-05-17T22:07:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 >> v2:172.21.15.155:3300/0 conn(0x7fcec81605a0 msgr2=0x7fcec81609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 --2- 172.21.15.83:0/1139606426 >> v2:172.21.15.155:3300/0 conn(0x7fcec81605a0 0x7fcec81609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 >> v1:172.21.15.155:6789/0 conn(0x7fcec81461c0 legacy=0x7fcec815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec806f210 con 0x7fcec806ffd0 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fceb8001700 con 0x7fcec806ffd0 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 >> v1:172.21.15.83:6789/0 conn(0x7fcec806ffd0 legacy=0x7fcec80703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb4003a60 0x7fceb4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4004810 0x7fceb4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 0x7fceb400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcec806ef30 con 0x7fceb4005260 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fceb400c1f0 con 0x7fceb4003a60 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fceb400c3f0 con 0x7fceb4004810 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 0x7fceb400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fceccb25640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4004810 0x7fceb4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecd326640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb4003a60 0x7fceb4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 -- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4004810 msgr2=0x7fceb4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4004810 0x7fceb4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 -- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb4003a60 msgr2=0x7fceb4003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fceb4003a60 0x7fceb4003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.934+0000 7fcecdb27640 1 -- 172.21.15.83:0/1139606426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb4003530 con 0x7fceb4005260 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecdb27640 1 -- 172.21.15.83:0/1139606426 reap_dead start 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecdb27640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 0x7fceb400b9d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009fc0 tx=0x7fcec001eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe23a6593471b931 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0009c10 con 0x7fceb4005260 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec00039a0 con 0x7fceb4005260 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcebffff640 1 -- 172.21.15.83:0/1139606426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0003d90 con 0x7fceb4005260 2024-05-17T22:07:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 -- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 msgr2=0x7fceb400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 0x7fceb400b9d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009fc0 tx=0x7fcec001eed0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 -- 172.21.15.83:0/1139606426 shutdown_connections 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 --2- 172.21.15.83:0/1139606426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4005260 0x7fceb400b9d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 -- 172.21.15.83:0/1139606426 >> 172.21.15.83:0/1139606426 conn(0x7fcec806cd90 msgr2=0x7fcec80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 -- 172.21.15.83:0/1139606426 shutdown_connections 2024-05-17T22:07:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.938+0000 7fcecf58a640 1 -- 172.21.15.83:0/1139606426 wait complete. 2024-05-17T22:07:45.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 Processor -- start 2024-05-17T22:07:45.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- start start 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4005260 0x7fcec81f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 0x7fcec80b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcec806fab0 con 0x7fcec80aa310 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcec806f7d0 con 0x7fcec80b5f00 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fcec81609b0 con 0x7fceb4005260 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51980/0 (socket says 172.21.15.83:51980) 2024-05-17T22:07:45.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 -- 172.21.15.83:0/3015116520 learned_addr learned my addr 172.21.15.83:0/3015116520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecdb27640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 0x7fcec80b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecd326640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4005260 0x7fcec81f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 -- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4005260 msgr2=0x7fcec81f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4005260 0x7fcec81f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 -- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 msgr2=0x7fcec80b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 0x7fcec80b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 -- 172.21.15.83:0/3015116520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec80aa720 con 0x7fcec80aa310 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecdb27640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 0x7fcec80b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fceccb25640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fceb800e700 tx=0x7fceb800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8019070 con 0x7fcec80aa310 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb800c660 con 0x7fcec80aa310 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb800ca10 con 0x7fcec80aa310 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec80b5180 con 0x7fcec80aa310 2024-05-17T22:07:45.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.942+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec80b5420 con 0x7fcec80aa310 2024-05-17T22:07:45.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec80b6ab0 con 0x7fcec80aa310 2024-05-17T22:07:45.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fceb801c070 con 0x7fcec80aa310 2024-05-17T22:07:45.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcebdffb640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 0x7fceac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:45.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fceb808e900 con 0x7fcec80aa310 2024-05-17T22:07:45.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcecd326640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 0x7fceac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:45.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.946+0000 7fcecd326640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 0x7fceac071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000c10 tx=0x7fcec4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:45.950+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fcec80aa720 con 0x7fcec80aa310 2024-05-17T22:07:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.154+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcec80b31f0 con 0x7fcec80aa310 2024-05-17T22:07:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcebdffb640 1 -- 172.21.15.83:0/3015116520 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7fceb805b1d0 con 0x7fcec80aa310 2024-05-17T22:07:46.150 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 msgr2=0x7fceac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 0x7fceac071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcec4000c10 tx=0x7fcec4009040 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 msgr2=0x7fcec813daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fceb800e700 tx=0x7fceb800ebc0 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 shutdown_connections 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fceac06f400 0x7fceac071910 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec80b5f00 0x7fcec80b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec80aa310 0x7fcec813daf0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 --2- 172.21.15.83:0/3015116520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fceb4005260 0x7fcec81f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 >> 172.21.15.83:0/3015116520 conn(0x7fcec806cd90 msgr2=0x7fcec8143d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 shutdown_connections 2024-05-17T22:07:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46.158+0000 7fcecf58a640 1 -- 172.21.15.83:0/3015116520 wait complete. 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:07:46.174 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:46.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.175 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.180 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41878sec preferred_lft 41878sec 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:07:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:07:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.183 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:07:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:07:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:07:46.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.183 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:07:46.233 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:07:46.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.234 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:07:46.286 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:07:46.287 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:07:46.288 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:07:46.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:07:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:46.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.407 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:07:46.459 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:07:46.459 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:07:46.459 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:46.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:07:46.515 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:07:46.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:46.515 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:46.563 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:07:46.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:07:46.563 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:46.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:46.668 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:46.716 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:46.717 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:46.718 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:46.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.719 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:46.767 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:07:46.767 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:07:46.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:46.827 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:46.875 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:46.875 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:46.875 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:46.875 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:46.876 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:46.877 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:46.878 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:46.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:46.879 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:46.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:07:46.930+0000 7f2267688580 -1 init, newargv = 0x7f22580054f0 newargc=15 2024-05-17T22:07:46.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49172]: starting ceph client 2024-05-17T22:07:46.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49172]: starting fuse 2024-05-17T22:07:48.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:07:48.044 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:07:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:07:48.093 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:07:48.094 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:48.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:48.095 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:07:48.143 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:07:48.143 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:07:48.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:07:48.143 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:07:48.143 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:07:48.144 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:07:48.145 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:48.266 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.49172.asok 2024-05-17T22:07:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:48.278 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49172.asok 2024-05-17T22:07:48.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:48.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49172.asok status 2024-05-17T22:07:48.455 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-05-17T22:07:48.462+0000 7f0cb83c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:07:48.461 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:48.461 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:07:48.461 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "49172", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8274, 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:07:48.462 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8274 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4214381711 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4214381711 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8274 192.168.0.1:0/4214381711", 2024-05-17T22:07:48.463 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/4214381711", 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 102, 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 110, 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:07:48.464 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:48.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:48.478 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:07:48.478 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:48.484 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:07:48.484 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:48.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:48.484 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:48.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:48.537 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-17T22:07:48.598 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/charlie 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/alpha 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/unflushed 2024-05-17T22:07:48.599 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/unflushed/jfile 2024-05-17T22:07:48.599 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:48.599 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:48.599 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:48.599 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.600 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-17T22:07:48.600 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:48.705 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:07:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:48.715 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:48.715 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:48.715 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:48.715 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.716 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-17T22:07:48.716 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:48.822 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:07:48.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-17T22:07:48.832 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:48.934 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T22:07:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:48.943 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:48.944 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:48.944 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:48.944 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:48.944 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-17T22:07:48.944 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:49.049 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628280 2024-05-17T22:07:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:49.059 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:49.059 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:49.059 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:49.059 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:49.059 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-17T22:07:49.060 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:49.163 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2024-05-17T22:07:49.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-17T22:07:49.173 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:49.281 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627779 2024-05-17T22:07:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:49.290 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:07:49.291 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:07:49.291 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:07:49.291 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:07:49.291 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-17T22:07:49.291 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:07:49.398 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627780 2024-05-17T22:07:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:07:49.407 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:49.408 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:49.456 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:49.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 Processor -- start 2024-05-17T22:07:49.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 -- start start 2024-05-17T22:07:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 0x7f454006eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4540072e80 0x7f4540073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45400737d0 con 0x7f45400b4610 2024-05-17T22:07:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f45400b5130 con 0x7f4540072e80 2024-05-17T22:07:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4547d4c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45401463e0 con 0x7f4540145fd0 2024-05-17T22:07:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f45452e7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4540072e80 0x7f4540073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4545ae8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 0x7f454006eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f45462e9640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f45400b4610 0x7f45400b4a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:36756/0 (socket says 172.21.15.83:36756) 2024-05-17T22:07:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f45462e9640 1 -- 172.21.15.83:0/3326445503 learned_addr learned my addr 172.21.15.83:0/3326445503 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4545ae8640 1 -- 172.21.15.83:0/3326445503 >> v2:172.21.15.155:3300/0 conn(0x7f4540072e80 msgr2=0x7f4540073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4545ae8640 1 --2- 172.21.15.83:0/3326445503 >> v2:172.21.15.155:3300/0 conn(0x7f4540072e80 0x7f4540073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4545ae8640 1 -- 172.21.15.83:0/3326445503 >> v1:172.21.15.155:6789/0 conn(0x7f45400b4610 legacy=0x7f45400b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.666+0000 7f4545ae8640 1 -- 172.21.15.83:0/3326445503 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45400b5360 con 0x7f4540145fd0 2024-05-17T22:07:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.670+0000 7f4545ae8640 1 --2- 172.21.15.83:0/3326445503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 0x7f454006eec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097c0 tx=0x7f453c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad86928805f82fd server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.670+0000 7f4544ae6640 1 -- 172.21.15.83:0/3326445503 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c0aa600 con 0x7f4540145fd0 2024-05-17T22:07:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.670+0000 7f4544ae6640 1 -- 172.21.15.83:0/3326445503 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453c0af070 con 0x7f4540145fd0 2024-05-17T22:07:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.670+0000 7f4544ae6640 1 -- 172.21.15.83:0/3326445503 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c0aa9c0 con 0x7f4540145fd0 2024-05-17T22:07:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- 172.21.15.83:0/3326445503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 msgr2=0x7f454006eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- 172.21.15.83:0/3326445503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 0x7f454006eec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097c0 tx=0x7f453c01c4a0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- 172.21.15.83:0/3326445503 shutdown_connections 2024-05-17T22:07:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- 172.21.15.83:0/3326445503 >> v2:172.21.15.155:3300/0 conn(0x7f4540072e80 0x7f4540073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- 172.21.15.83:0/3326445503 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4540145fd0 0x7f454006eec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- 172.21.15.83:0/3326445503 >> 172.21.15.83:0/3326445503 conn(0x7f454006cd90 msgr2=0x7f45400b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- 172.21.15.83:0/3326445503 shutdown_connections 2024-05-17T22:07:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- 172.21.15.83:0/3326445503 wait complete. 2024-05-17T22:07:49.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 Processor -- start 2024-05-17T22:07:49.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- start start 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 0x7f4540142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4540145fd0 0x7f454015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45400af140 con 0x7f4540072e80 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f45400aefc0 con 0x7f4540145fd0 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4547d4c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f45400b0670 con 0x7f45400b4610 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4545ae8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4545ae8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55508/0 (socket says 172.21.15.83:55508) 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f4545ae8640 1 -- 172.21.15.83:0/4089613554 learned_addr learned my addr 172.21.15.83:0/4089613554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.674+0000 7f45452e7640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 0x7f4540142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45462e9640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4540145fd0 0x7f454015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 -- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 msgr2=0x7f4540142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 0x7f4540142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 -- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4540145fd0 msgr2=0x7f454015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4540145fd0 0x7f454015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 -- 172.21.15.83:0/4089613554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45401fcf00 con 0x7f4540072e80 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45452e7640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 0x7f4540142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4545ae8640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f453c01a0d0 tx=0x7f453c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c006e20 con 0x7f4540072e80 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453c0be610 con 0x7f4540072e80 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c0be9e0 con 0x7f4540072e80 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454013bfc0 con 0x7f4540072e80 2024-05-17T22:07:49.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454015c970 con 0x7f4540072e80 2024-05-17T22:07:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45400b0a50 con 0x7f4540072e80 2024-05-17T22:07:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f453c0bc020 con 0x7f4540072e80 2024-05-17T22:07:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 0x7f4524071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f453c129930 con 0x7f4540072e80 2024-05-17T22:07:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.678+0000 7f45452e7640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 0x7f4524071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.682+0000 7f45452e7640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 0x7f4524071940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f45400b1cf0 tx=0x7f4530008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:49.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.682+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f453c016090 con 0x7f4540072e80 2024-05-17T22:07:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.854+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4540070340 con 0x7f4540072e80 2024-05-17T22:07:49.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.854+0000 7f45367fc640 1 -- 172.21.15.83:0/4089613554 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v110) ==== 80+0+124 (secure 0 0 0) 0x7f453c0f68a0 con 0x7f4540072e80 2024-05-17T22:07:49.850 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6833/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:07:49.850 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6832/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:07:49.850 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 msgr2=0x7f4524071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 0x7f4524071940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f45400b1cf0 tx=0x7f4530008040 comp rx=0 tx=0).stop 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 msgr2=0x7f454013c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f453c01a0d0 tx=0x7f453c01ace0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 shutdown_connections 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f452406f430 0x7f4524071940 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4540145fd0 0x7f454015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f45400b4610 0x7f4540142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 --2- 172.21.15.83:0/4089613554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4540072e80 0x7f454013c790 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:49.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 >> 172.21.15.83:0/4089613554 conn(0x7f454006cd90 msgr2=0x7f454006f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:49.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 shutdown_connections 2024-05-17T22:07:49.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:49.858+0000 7f4547d4c640 1 -- 172.21.15.83:0/4089613554 wait complete. 2024-05-17T22:07:49.870 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:07:49.871 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:07:49.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:07:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 Processor -- start 2024-05-17T22:07:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 -- start start 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4bd00cf830 0x7f4bd00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4bd00cdf40 0x7f4bd015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd00b5640 con 0x7f4bd015d530 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd00b58e0 con 0x7f4bd00cdf40 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd653e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd006f060 con 0x7f4bd00cf830 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcf7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4bd00cdf40 0x7f4bd015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd4adb640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4bd015d530 0x7f4bd006f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57946/0 (socket says 172.21.15.83:57946) 2024-05-17T22:07:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bd4adb640 1 -- 172.21.15.83:0/3223134097 learned_addr learned my addr 172.21.15.83:0/3223134097 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcffff640 1 --2- 172.21.15.83:0/3223134097 >> v2:172.21.15.155:3300/0 conn(0x7f4bd00cf830 0x7f4bd00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3223134097 >> v2:172.21.15.155:3300/0 conn(0x7f4bd00cf830 msgr2=0x7f4bd00b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcf7fe640 1 --2- 172.21.15.83:0/3223134097 >> v2:172.21.15.155:3300/0 conn(0x7f4bd00cf830 0x7f4bd00b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3223134097 >> v1:172.21.15.83:6789/0 conn(0x7f4bd015d530 legacy=0x7f4bd006f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bceffd640 1 -- 172.21.15.83:0/3223134097 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273273350 0 0) 0x7f4bd00b5640 con 0x7f4bd015d530 2024-05-17T22:07:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.074+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3223134097 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd006f340 con 0x7f4bd00cdf40 2024-05-17T22:07:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bcf7fe640 1 --2- 172.21.15.83:0/3223134097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cdf40 0x7f4bd015cff0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4bd006d6e0 tx=0x7f4bb801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d598774cc326694 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bceffd640 1 -- 172.21.15.83:0/3223134097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b1070 con 0x7f4bd00cdf40 2024-05-17T22:07:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bceffd640 1 -- 172.21.15.83:0/3223134097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80ac5f0 con 0x7f4bd00cdf40 2024-05-17T22:07:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bceffd640 1 -- 172.21.15.83:0/3223134097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80ac9c0 con 0x7f4bd00cdf40 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 -- 172.21.15.83:0/3223134097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cdf40 msgr2=0x7f4bd015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3223134097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cdf40 0x7f4bd015cff0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4bd006d6e0 tx=0x7f4bb801c480 comp rx=0 tx=0).stop 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 -- 172.21.15.83:0/3223134097 shutdown_connections 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3223134097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cdf40 0x7f4bd015cff0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3223134097 >> v2:172.21.15.155:3300/0 conn(0x7f4bd00cf830 0x7f4bd00b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 -- 172.21.15.83:0/3223134097 >> 172.21.15.83:0/3223134097 conn(0x7f4bd006cd90 msgr2=0x7f4bd00b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 -- 172.21.15.83:0/3223134097 shutdown_connections 2024-05-17T22:07:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.078+0000 7f4bd653e640 1 -- 172.21.15.83:0/3223134097 wait complete. 2024-05-17T22:07:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 Processor -- start 2024-05-17T22:07:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- start start 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bd00cdf40 0x7f4bd00af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bd015d530 0x7f4bd0159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0169780 con 0x7f4bd00cf830 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0169600 con 0x7f4bd015d530 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd0169900 con 0x7f4bd00cdf40 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55538/0 (socket says 172.21.15.83:55538) 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3761956115 learned_addr learned my addr 172.21.15.83:0/3761956115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd4adb640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bd015d530 0x7f4bd0159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcffff640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bd00cdf40 0x7f4bd00af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bd00cdf40 msgr2=0x7f4bd00af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bd00cdf40 0x7f4bd00af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bd015d530 msgr2=0x7f4bd0159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bd015d530 0x7f4bd0159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 -- 172.21.15.83:0/3761956115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00b0100 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bcf7fe640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4bb801ab30 tx=0x7f4bb8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b1040 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80bb070 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b5870 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01fcb60 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01fd120 con 0x7f4bd00cf830 2024-05-17T22:07:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.082+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd01fc950 con 0x7f4bd00cf830 2024-05-17T22:07:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.086+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4bb80b5a10 con 0x7f4bd00cf830 2024-05-17T22:07:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.086+0000 7f4bccff9640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.086+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f4bb8128db0 con 0x7f4bd00cf830 2024-05-17T22:07:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.086+0000 7f4bcffff640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.086+0000 7f4bcffff640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 0x7f4bb0071940 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009e90 tx=0x7f4bc000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.090+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f4bb80a9070 con 0x7f4bd00cf830 2024-05-17T22:07:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.262+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4bd00afe90 con 0x7f4bd00cf830 2024-05-17T22:07:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.262+0000 7f4bccff9640 1 -- 172.21.15.83:0/3761956115 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v110) ==== 80+0+124 (secure 0 0 0) 0x7f4bb8016070 con 0x7f4bd00cf830 2024-05-17T22:07:50.257 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6833/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:07:50.257 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6832/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:07:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 msgr2=0x7f4bb0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 0x7f4bb0071940 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009e90 tx=0x7f4bc000b040 comp rx=0 tx=0).stop 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 msgr2=0x7f4bd014d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4bb801ab30 tx=0x7f4bb8019220 comp rx=0 tx=0).stop 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 shutdown_connections 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bd015d530 0x7f4bd0159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd00cf830 0x7f4bd014d670 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 --2- 172.21.15.83:0/3761956115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bd00cdf40 0x7f4bd00af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:50.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 >> 172.21.15.83:0/3761956115 conn(0x7f4bd006cd90 msgr2=0x7f4bd00b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:50.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 shutdown_connections 2024-05-17T22:07:50.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:50.266+0000 7f4bd653e640 1 -- 172.21.15.83:0/3761956115 wait complete. 2024-05-17T22:07:50.277 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:07:50.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:50.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:07:50.288 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49172]: fuse finished with error 0 and tester_r 0 2024-05-17T22:07:50.290 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:07:56.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:07:56.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:07:56.392 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:07:56.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:07:56.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 Processor -- start 2024-05-17T22:07:56.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 -- start start 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a501461c0 0x7f7a5015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a501465d0 con 0x7f7a500706e0 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5006ef30 con 0x7f7a5015e6b0 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5858e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5006f210 con 0x7f7a501461c0 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5632a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a501461c0 0x7f7a5015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a56b2b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7a500706e0 0x7f7a501605a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57954/0 (socket says 172.21.15.83:57954) 2024-05-17T22:07:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a56b2b640 1 -- 172.21.15.83:0/1350633815 learned_addr learned my addr 172.21.15.83:0/1350633815 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5632a640 1 -- 172.21.15.83:0/1350633815 >> v1:172.21.15.155:6789/0 conn(0x7f7a5015e6b0 legacy=0x7f7a5006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5632a640 1 -- 172.21.15.83:0/1350633815 >> v1:172.21.15.83:6789/0 conn(0x7f7a500706e0 legacy=0x7f7a501605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a55328640 1 -- 172.21.15.83:0/1350633815 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574565973 0 0) 0x7f7a501465d0 con 0x7f7a500706e0 2024-05-17T22:07:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.618+0000 7f7a5632a640 1 -- 172.21.15.83:0/1350633815 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5006f4f0 con 0x7f7a501461c0 2024-05-17T22:07:56.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5632a640 1 --2- 172.21.15.83:0/1350633815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a5015e170 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097c0 tx=0x7f7a4c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd2ac10c2f5f353 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a55328640 1 -- 172.21.15.83:0/1350633815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0a7070 con 0x7f7a501461c0 2024-05-17T22:07:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a55328640 1 -- 172.21.15.83:0/1350633815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0b0070 con 0x7f7a501461c0 2024-05-17T22:07:56.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a55328640 1 -- 172.21.15.83:0/1350633815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0ab890 con 0x7f7a501461c0 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 -- 172.21.15.83:0/1350633815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 msgr2=0x7f7a5015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1350633815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a5015e170 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097c0 tx=0x7f7a4c01c470 comp rx=0 tx=0).stop 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 -- 172.21.15.83:0/1350633815 shutdown_connections 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1350633815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a5015e170 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 -- 172.21.15.83:0/1350633815 >> 172.21.15.83:0/1350633815 conn(0x7f7a5006cd90 msgr2=0x7f7a500b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 -- 172.21.15.83:0/1350633815 shutdown_connections 2024-05-17T22:07:56.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.622+0000 7f7a5858e640 1 -- 172.21.15.83:0/1350633815 wait complete. 2024-05-17T22:07:56.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 Processor -- start 2024-05-17T22:07:56.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- start start 2024-05-17T22:07:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a500706e0 0x7f7a5013baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 0x7f7a500d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500d10a0 con 0x7f7a501461c0 2024-05-17T22:07:56.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50160ec0 con 0x7f7a500706e0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f7a500d1220 con 0x7f7a5015e6b0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55564/0 (socket says 172.21.15.83:55564) 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 -- 172.21.15.83:0/1420226819 learned_addr learned my addr 172.21.15.83:0/1420226819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5632a640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a500706e0 0x7f7a5013baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a56b2b640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 0x7f7a500d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 -- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 msgr2=0x7f7a500d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 0x7f7a500d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 -- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a500706e0 msgr2=0x7f7a5013baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a500706e0 0x7f7a5013baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 -- 172.21.15.83:0/1420226819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5015b8b0 con 0x7f7a501461c0 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a56b2b640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 0x7f7a500d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:56.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a55b29640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b410 tx=0x7f7a4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44015030 con 0x7f7a501461c0 2024-05-17T22:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a44004020 con 0x7f7a501461c0 2024-05-17T22:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5015bb90 con 0x7f7a501461c0 2024-05-17T22:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44004410 con 0x7f7a501461c0 2024-05-17T22:07:56.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5015c150 con 0x7f7a501461c0 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.626+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5015b600 con 0x7f7a501461c0 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.630+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a44018070 con 0x7f7a501461c0 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.630+0000 7f7a42ffd640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 0x7f7a34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.630+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f7a4408eec0 con 0x7f7a501461c0 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.630+0000 7f7a5632a640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 0x7f7a34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:56.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.630+0000 7f7a5632a640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 0x7f7a34071940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0af000 tx=0x7f7a4c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:56.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.634+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f7a4405bd50 con 0x7f7a501461c0 2024-05-17T22:07:56.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.822+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5015e170 con 0x7f7a501461c0 2024-05-17T22:07:56.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.822+0000 7f7a42ffd640 1 -- 172.21.15.83:0/1420226819 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5185 (secure 0 0 0) 0x7f7a4405b650 con 0x7f7a501461c0 2024-05-17T22:07:56.813 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:56.813 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:44.164826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7980},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7980":{"gid":7980,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":12,"addr":"172.21.15.83:6837/78757339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":78757339},{"type":"v1","addr":"172.21.15.83:6837","nonce":78757339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7980,"qdb_cluster":[7980]},"id":8}]} 2024-05-17T22:07:56.814 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 102 2024-05-17T22:07:56.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 msgr2=0x7f7a34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 0x7f7a34071940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0af000 tx=0x7f7a4c0a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 msgr2=0x7f7a500d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b410 tx=0x7f7a4400b8d0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 shutdown_connections 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a5015e6b0 0x7f7a500d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a501461c0 0x7f7a500d1510 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f7a3406f430 0x7f7a34071940 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 --2- 172.21.15.83:0/1420226819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a500706e0 0x7f7a5013baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:56.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 >> 172.21.15.83:0/1420226819 conn(0x7f7a5006cd90 msgr2=0x7f7a500ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 shutdown_connections 2024-05-17T22:07:56.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:56.826+0000 7f7a5858e640 1 -- 172.21.15.83:0/1420226819 wait complete. 2024-05-17T22:07:56.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-05-17T22:07:56.989 INFO:teuthology.orchestra.run.smithi083.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-17T22:07:56.989 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-17T22:07:56.989 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-17T22:07:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 Processor -- start 2024-05-17T22:07:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 -- start start 2024-05-17T22:07:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd21c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638fd13c1b0 con 0x5638fd237100 2024-05-17T22:07:57.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638fd197ef0 con 0x5638fd2ab460 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ae1378080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5638fd1eced0 con 0x5638fd236ac0 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42198/0 (socket says 172.21.15.83:42198) 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 -- 172.21.15.83:0/1548694618 learned_addr learned my addr 172.21.15.83:0/1548694618 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2adae5a640 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd21c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2adb65b640 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 -- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 msgr2=0x5638fd21c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd21c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 -- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 msgr2=0x5638fd2a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 -- 172.21.15.83:0/1548694618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fd116c10 con 0x5638fd237100 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ada659640 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0001ec0 tx=0x7f2ad000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f312e388ee95c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.066+0000 7f2ad9657640 1 -- 172.21.15.83:0/1548694618 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad000d510 con 0x5638fd237100 2024-05-17T22:07:57.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ad9657640 1 -- 172.21.15.83:0/1548694618 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad0008070 con 0x5638fd237100 2024-05-17T22:07:57.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ad9657640 1 -- 172.21.15.83:0/1548694618 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad000d910 con 0x5638fd237100 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- 172.21.15.83:0/1548694618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 msgr2=0x5638fd2a6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0001ec0 tx=0x7f2ad000b6b0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- 172.21.15.83:0/1548694618 shutdown_connections 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2a6e90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- 172.21.15.83:0/1548694618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd21c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- 172.21.15.83:0/1548694618 >> 172.21.15.83:0/1548694618 conn(0x5638fd1d8e40 msgr2=0x5638fd1f0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- 172.21.15.83:0/1548694618 shutdown_connections 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- 172.21.15.83:0/1548694618 wait complete. 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 Processor -- start 2024-05-17T22:07:57.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- start start 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd2a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2bf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2bf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638fd2a7cb0 con 0x5638fd237100 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5638fd2a7a10 con 0x5638fd2ab460 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ae1378080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5638fd2a7f50 con 0x5638fd236ac0 2024-05-17T22:07:57.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2bf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42200/0 (socket says 172.21.15.83:42200) 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 -- 172.21.15.83:0/2753013849 learned_addr learned my addr 172.21.15.83:0/2753013849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2adae5a640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd2a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2adb65b640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 -- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 msgr2=0x5638fd2a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5638fd236ac0 0x5638fd2a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 -- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 msgr2=0x5638fd2bf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2bf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 -- 172.21.15.83:0/2753013849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fd2bd340 con 0x5638fd237100 2024-05-17T22:07:57.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2adb65b640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638fd2ab460 0x5638fd2bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.070+0000 7f2ada659640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2bf010 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000bbb0 tx=0x7f2ad0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 -- 172.21.15.83:0/2753013849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad001b850 con 0x5638fd237100 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 -- 172.21.15.83:0/2753013849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad0020030 con 0x5638fd237100 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638fd2bd5e0 con 0x5638fd237100 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 -- 172.21.15.83:0/2753013849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad001bbf0 con 0x5638fd237100 2024-05-17T22:07:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fd2bdb20 con 0x5638fd237100 2024-05-17T22:07:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 -- 172.21.15.83:0/2753013849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ad0017040 con 0x5638fd237100 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ac806de10 0x7f2ac8070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ac3fff640 1 -- 172.21.15.83:0/2753013849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f2ad00901b0 con 0x5638fd237100 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adae5a640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ac806de10 0x7f2ac8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5638fd354250 0x5638fd2a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5638fd2a29d0 con 0x5638fd354250 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adb65b640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5638fd354250 0x5638fd2a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adae5a640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ac806de10 0x7f2ac8070320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0ac000 tx=0x7f2acc0a5000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adb65b640 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5638fd354250 0x5638fd2a2490 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adb65b640 1 -- 172.21.15.83:0/2753013849 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f2ad4011040 con 0x5638fd354250 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5638fd354250 msgr2=0x5638fd2a2490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5638fd354250 0x5638fd2a2490 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ac806de10 msgr2=0x7f2ac8070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f2ac806de10 0x7f2ac8070320 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0ac000 tx=0x7f2acc0a5000 comp rx=0 tx=0).stop 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 msgr2=0x5638fd2bf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 --2- 172.21.15.83:0/2753013849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638fd237100 0x5638fd2bf010 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000bbb0 tx=0x7f2ad0000ee0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2adb65b640 1 -- 172.21.15.83:0/2753013849 reap_dead start 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 shutdown_connections 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 >> 172.21.15.83:0/2753013849 conn(0x5638fd1d8e40 msgr2=0x5638fd2aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 shutdown_connections 2024-05-17T22:07:57.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.074+0000 7f2ae1378080 1 -- 172.21.15.83:0/2753013849 wait complete. 2024-05-17T22:07:57.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:07:57.073 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-17T22:07:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 Processor -- start 2024-05-17T22:07:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 -- start start 2024-05-17T22:07:57.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be35774110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be35703850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be356071b0 con 0x55be356b6700 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55be35662ef0 con 0x55be3578e920 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55be35702930 con 0x55be35702220 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42202/0 (socket says 172.21.15.83:42202) 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 -- 172.21.15.83:0/1618688907 learned_addr learned my addr 172.21.15.83:0/1618688907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75236d4640 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be35703850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75226d2640 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be35774110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 -- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 msgr2=0x55be35774110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be35774110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 -- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 msgr2=0x55be35703850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be35703850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 -- 172.21.15.83:0/1618688907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be355e1c10 con 0x55be356b6700 2024-05-17T22:07:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7514001680 tx=0x7f7514001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c389e94e19688f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75216d0640 1 -- 172.21.15.83:0/1618688907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514004530 con 0x55be356b6700 2024-05-17T22:07:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75216d0640 1 -- 172.21.15.83:0/1618688907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75140046d0 con 0x55be356b6700 2024-05-17T22:07:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75216d0640 1 -- 172.21.15.83:0/1618688907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514004a80 con 0x55be356b6700 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 -- 172.21.15.83:0/1618688907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 msgr2=0x55be35704780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.146+0000 7f75293f1080 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7514001680 tx=0x7f7514001b40 comp rx=0 tx=0).stop 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1618688907 shutdown_connections 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be35703850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be35774110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- 172.21.15.83:0/1618688907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35704780 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1618688907 >> 172.21.15.83:0/1618688907 conn(0x55be356a3e40 msgr2=0x55be3578ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1618688907 shutdown_connections 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1618688907 wait complete. 2024-05-17T22:07:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 Processor -- start 2024-05-17T22:07:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- start start 2024-05-17T22:07:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35777b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be3575d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be3575db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be35704cc0 con 0x55be356b6700 2024-05-17T22:07:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55be357915a0 con 0x55be3578e920 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55be35791840 con 0x55be35702220 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35777b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35777b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42208/0 (socket says 172.21.15.83:42208) 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 -- 172.21.15.83:0/1617854882 learned_addr learned my addr 172.21.15.83:0/1617854882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75226d2640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be3575d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75236d4640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be3575db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 -- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 msgr2=0x55be3575d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55be35702220 0x55be3575d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 -- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 msgr2=0x55be3575db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be3578e920 0x55be3575db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 -- 172.21.15.83:0/1617854882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be357786a0 con 0x55be356b6700 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f7522ed3640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35777b90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7514006c10 tx=0x7f7514004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f751bfff640 1 -- 172.21.15.83:0/1617854882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514024070 con 0x55be356b6700 2024-05-17T22:07:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f751bfff640 1 -- 172.21.15.83:0/1617854882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7514013620 con 0x55be356b6700 2024-05-17T22:07:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55be35778940 con 0x55be356b6700 2024-05-17T22:07:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f751bfff640 1 -- 172.21.15.83:0/1617854882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514018040 con 0x55be356b6700 2024-05-17T22:07:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.150+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3578b750 con 0x55be356b6700 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f751bfff640 1 -- 172.21.15.83:0/1617854882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7514027070 con 0x55be356b6700 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f751bfff640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f750406dde0 0x7f75040702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f751bfff640 1 -- 172.21.15.83:0/1617854882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f751408f0f0 con 0x55be356b6700 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75226d2640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f750406dde0 0x7f75040702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55be3578cc00 0x55be35772930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x55be35772e70 con 0x55be3578cc00 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75236d4640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55be3578cc00 0x55be35772930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75226d2640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f750406dde0 0x7f75040702f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f750c005fd0 tx=0x7f750c00c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75236d4640 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55be3578cc00 0x55be35772930 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75236d4640 1 -- 172.21.15.83:0/1617854882 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f751c00a040 con 0x55be3578cc00 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55be3578cc00 msgr2=0x55be35772930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55be3578cc00 0x55be35772930 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f750406dde0 msgr2=0x7f75040702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f750406dde0 0x7f75040702f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f750c005fd0 tx=0x7f750c00c040 comp rx=0 tx=0).stop 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 msgr2=0x55be35777b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 --2- 172.21.15.83:0/1617854882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be356b6700 0x55be35777b90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7514006c10 tx=0x7f7514004470 comp rx=0 tx=0).stop 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75236d4640 1 -- 172.21.15.83:0/1617854882 reap_dead start 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 shutdown_connections 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 >> 172.21.15.83:0/1617854882 conn(0x55be356a3e40 msgr2=0x55be35703850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 shutdown_connections 2024-05-17T22:07:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.154+0000 7f75293f1080 1 -- 172.21.15.83:0/1617854882 wait complete. 2024-05-17T22:07:57.151 INFO:teuthology.orchestra.run.smithi083.stdout:mytag123 2024-05-17T22:07:57.152 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-17T22:07:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 Processor -- start 2024-05-17T22:07:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 -- start start 2024-05-17T22:07:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579485c3240 0x5579485c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x55794869fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5579485101b0 con 0x5579486989d0 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55794856bef0 con 0x557948698de0 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b0a950080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5579485c1b90 con 0x5579485c3240 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42230/0 (socket says 172.21.15.83:42230) 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 -- 172.21.15.83:0/3536273717 learned_addr learned my addr 172.21.15.83:0/3536273717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 -- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579485c3240 msgr2=0x5579485c0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579485c3240 0x5579485c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 -- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 msgr2=0x55794869fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x55794869fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 -- 172.21.15.83:0/3536273717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579484eac10 con 0x5579486989d0 2024-05-17T22:07:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b03c31640 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00a500 tx=0x7f8aec003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1568b728c62a9b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b02c2f640 1 -- 172.21.15.83:0/3536273717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0bb070 con 0x5579486989d0 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b02c2f640 1 -- 172.21.15.83:0/3536273717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aec0aab10 con 0x5579486989d0 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.222+0000 7f8b02c2f640 1 -- 172.21.15.83:0/3536273717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0af040 con 0x5579486989d0 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/3536273717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 msgr2=0x55794869f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00a500 tx=0x7f8aec003030 comp rx=0 tx=0).stop 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/3536273717 shutdown_connections 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x55794869fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579486989d0 0x55794869f840 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- 172.21.15.83:0/3536273717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579485c3240 0x5579485c0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/3536273717 >> 172.21.15.83:0/3536273717 conn(0x5579485ace40 msgr2=0x5579485f1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/3536273717 shutdown_connections 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/3536273717 wait complete. 2024-05-17T22:07:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 Processor -- start 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- start start 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 0x557948678730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579486989d0 0x55794868fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x5579485c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557948677360 con 0x5579485c3240 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55794867c010 con 0x557948698de0 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5579485eebf0 con 0x5579486989d0 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 0x557948678730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 0x557948678730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42232/0 (socket says 172.21.15.83:42232) 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 -- 172.21.15.83:0/2703188366 learned_addr learned my addr 172.21.15.83:0/2703188366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b03c31640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579486989d0 0x55794868fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04c33640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x5579485c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 -- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579486989d0 msgr2=0x55794868fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579486989d0 0x55794868fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 -- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 msgr2=0x5579485c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x5579485c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 -- 172.21.15.83:0/2703188366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579486790e0 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b03c31640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5579486989d0 0x55794868fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04c33640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557948698de0 0x5579485c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b04432640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 0x557948678730 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00bd00 tx=0x7f8afc009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b01c2d640 1 -- 172.21.15.83:0/2703188366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f640 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b01c2d640 1 -- 172.21.15.83:0/2703188366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc008070 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557948679380 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b01c2d640 1 -- 172.21.15.83:0/2703188366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00fa40 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.226+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794872f780 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b01c2d640 1 -- 172.21.15.83:0/2703188366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8afc00fbe0 con 0x5579485c3240 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b01c2d640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ae406dde0 0x7f8ae40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b01c2d640 1 -- 172.21.15.83:0/2703188366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f8afc08e060 con 0x5579485c3240 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b03c31640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ae406dde0 0x7f8ae40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b0a950080 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x557948737be0 0x55794873a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x5579486902d0 con 0x557948737be0 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b03c31640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ae406dde0 0x7f8ae40702f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0199d0 tx=0x7f8aec0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b04c33640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x557948737be0 0x55794873a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.230+0000 7f8b04c33640 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x557948737be0 0x55794873a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b04c33640 1 -- 172.21.15.83:0/2703188366 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f8af400c690 con 0x557948737be0 2024-05-17T22:07:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x557948737be0 msgr2=0x55794873a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x557948737be0 0x55794873a000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ae406dde0 msgr2=0x7f8ae40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ae406dde0 0x7f8ae40702f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0199d0 tx=0x7f8aec0a7000 comp rx=0 tx=0).stop 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 msgr2=0x557948678730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 --2- 172.21.15.83:0/2703188366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579485c3240 0x557948678730 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00bd00 tx=0x7f8afc009570 comp rx=0 tx=0).stop 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b04c33640 1 -- 172.21.15.83:0/2703188366 reap_dead start 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 shutdown_connections 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 >> 172.21.15.83:0/2703188366 conn(0x5579485ace40 msgr2=0x5579486995f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 shutdown_connections 2024-05-17T22:07:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.234+0000 7f8b0a950080 1 -- 172.21.15.83:0/2703188366 wait complete. 2024-05-17T22:07:57.229 INFO:teuthology.orchestra.run.smithi083.stdout:mytag123 2024-05-17T22:07:57.229 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-17T22:07:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.298+0000 7ff204be6080 1 Processor -- start 2024-05-17T22:07:57.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.298+0000 7ff204be6080 1 -- start start 2024-05-17T22:07:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb956050 0x559dfb93bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 0x559dfb93c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559dfb85b1b0 con 0x559dfb953580 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559dfb8b6ef0 con 0x559dfb9ca950 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559dfb90f700 con 0x559dfb956050 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42256/0 (socket says 172.21.15.83:42256) 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/2455292112 learned_addr learned my addr 172.21.15.83:0/2455292112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1feec9640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 0x559dfb93c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb956050 msgr2=0x559dfb93bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fdec7640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb956050 0x559dfb93bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb956050 0x559dfb93bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 msgr2=0x559dfb93c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 0x559dfb93c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/2455292112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfb835c10 con 0x559dfb953580 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1feec9640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 0x559dfb93c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800a470 tx=0x7ff1f800a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605fc0badb6c0c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fcec5640 1 -- 172.21.15.83:0/2455292112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800b930 con 0x559dfb953580 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fcec5640 1 -- 172.21.15.83:0/2455292112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f800f030 con 0x559dfb953580 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff1fcec5640 1 -- 172.21.15.83:0/2455292112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800bd30 con 0x559dfb953580 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- 172.21.15.83:0/2455292112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 msgr2=0x559dfb9cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800a470 tx=0x7ff1f800a930 comp rx=0 tx=0).stop 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- 172.21.15.83:0/2455292112 shutdown_connections 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb9ca950 0x559dfb93c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb956050 0x559dfb93bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 --2- 172.21.15.83:0/2455292112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb9cbe30 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- 172.21.15.83:0/2455292112 >> 172.21.15.83:0/2455292112 conn(0x559dfb8f7e40 msgr2=0x559dfb9cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- 172.21.15.83:0/2455292112 shutdown_connections 2024-05-17T22:07:57.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- 172.21.15.83:0/2455292112 wait complete. 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 Processor -- start 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.302+0000 7ff204be6080 1 -- start start 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb94fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb956050 0x559dfb9e2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb9ca950 0x559dfb9c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559dfb93c7b0 con 0x559dfb953580 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x559dfb93ca50 con 0x559dfb956050 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x559dfb94da90 con 0x559dfb9ca950 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb94fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb94fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42266/0 (socket says 172.21.15.83:42266) 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/1845704141 learned_addr learned my addr 172.21.15.83:0/1845704141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1feec9640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb9ca950 0x559dfb9c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fdec7640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb956050 0x559dfb9e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb9ca950 msgr2=0x559dfb9c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb9ca950 0x559dfb9c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb956050 msgr2=0x559dfb9e2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb956050 0x559dfb9e2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 -- 172.21.15.83:0/1845704141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfb9e1910 con 0x559dfb953580 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fdec7640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559dfb956050 0x559dfb9e2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1feec9640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x559dfb9ca950 0x559dfb9c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fe6c8640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb94fb60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005410 tx=0x7ff1f8002f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 -- 172.21.15.83:0/1845704141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8024070 con 0x559dfb953580 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 -- 172.21.15.83:0/1845704141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f80149b0 con 0x559dfb953580 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559dfb9e1bb0 con 0x559dfb953580 2024-05-17T22:07:57.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 -- 172.21.15.83:0/1845704141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8014d80 con 0x559dfb953580 2024-05-17T22:07:57.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfb9500a0 con 0x559dfb953580 2024-05-17T22:07:57.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 -- 172.21.15.83:0/1845704141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff1f8027070 con 0x559dfb953580 2024-05-17T22:07:57.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1e006dde0 0x7ff1e00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1f77fe640 1 -- 172.21.15.83:0/1845704141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7ff1f808f5a0 con 0x559dfb953580 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fdec7640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1e006dde0 0x7ff1e00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559dfb94ea70 0x559dfb951cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x559dfb952210 con 0x559dfb94ea70 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.306+0000 7ff1fdec7640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1e006dde0 0x7ff1e00702f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80075f0 tx=0x7ff1e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff1feec9640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559dfb94ea70 0x559dfb951cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff1feec9640 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559dfb94ea70 0x559dfb951cd0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff1feec9640 1 -- 172.21.15.83:0/1845704141 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7ff1f00aa960 con 0x559dfb94ea70 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559dfb94ea70 msgr2=0x559dfb951cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x559dfb94ea70 0x559dfb951cd0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1e006dde0 msgr2=0x7ff1e00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff1e006dde0 0x7ff1e00702f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80075f0 tx=0x7ff1e8009040 comp rx=0 tx=0).stop 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 msgr2=0x559dfb94fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 --2- 172.21.15.83:0/1845704141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dfb953580 0x559dfb94fb60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005410 tx=0x7ff1f8002f80 comp rx=0 tx=0).stop 2024-05-17T22:07:57.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff1feec9640 1 -- 172.21.15.83:0/1845704141 reap_dead start 2024-05-17T22:07:57.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 shutdown_connections 2024-05-17T22:07:57.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 >> 172.21.15.83:0/1845704141 conn(0x559dfb8f7e40 msgr2=0x559dfb9cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 shutdown_connections 2024-05-17T22:07:57.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.310+0000 7ff204be6080 1 -- 172.21.15.83:0/1845704141 wait complete. 2024-05-17T22:07:57.306 INFO:teuthology.orchestra.run.smithi083.stdout:mytag123 2024-05-17T22:07:57.307 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-17T22:07:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.462+0000 7fafc252b640 1 Processor -- start 2024-05-17T22:07:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.462+0000 7fafc252b640 1 -- start start 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc252b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fafbc06eec0 0x7fafbc0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc252b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc06f2d0 con 0x7fafbc0b44c0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc252b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc06f5a0 con 0x7fafbc15be70 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc252b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc06f880 con 0x7fafbc06eec0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc0ac8640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fafbc0b44c0 0x7fafbc072e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37486/0 (socket says 172.21.15.83:37486) 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc0ac8640 1 -- 172.21.15.83:0/3559237947 learned_addr learned my addr 172.21.15.83:0/3559237947 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3559237947 >> v2:172.21.15.155:3300/0 conn(0x7fafbc06eec0 0x7fafbc0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243927307 0 0) 0x7fafbc06f2d0 con 0x7fafbc0b44c0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa8003530 con 0x7fafbc0b44c0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2110051394 0 0) 0x7fafa8003530 con 0x7fafbc0b44c0 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 >> v2:172.21.15.155:3300/0 conn(0x7fafbc06eec0 msgr2=0x7fafbc0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 --2- 172.21.15.83:0/3559237947 >> v2:172.21.15.155:3300/0 conn(0x7fafbc06eec0 0x7fafbc0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 >> v1:172.21.15.155:6789/0 conn(0x7fafbc15be70 legacy=0x7fafbc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc06fb60 con 0x7fafbc0b44c0 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fafb4001700 con 0x7fafbc0b44c0 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 >> v1:172.21.15.83:6789/0 conn(0x7fafbc0b44c0 legacy=0x7fafbc072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa8003a60 0x7fafa8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8005360 0x7fafa800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc06f880 con 0x7fafa8004810 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafa800d3a0 con 0x7fafa8005360 2024-05-17T22:07:57.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafa800c2e0 con 0x7fafa8003a60 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8005360 0x7fafa800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa8003a60 msgr2=0x7fafa8003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa8003a60 0x7fafa8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafa8003a60 0x7fafa8003ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8005360 msgr2=0x7fafa800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafa8005360 0x7fafa800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafc0ac8640 1 -- 172.21.15.83:0/3559237947 reap_dead start 2024-05-17T22:07:57.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.466+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3559237947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa8003530 con 0x7fafa8004810 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fafbc072e30 tx=0x7fafac01c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43801d99922f66b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac001ae0 con 0x7fafa8004810 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac003b80 con 0x7fafa8004810 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafbaffd640 1 -- 172.21.15.83:0/3559237947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0050d0 con 0x7fafa8004810 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 msgr2=0x7fafa8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fafbc072e30 tx=0x7fafac01c6c0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- 172.21.15.83:0/3559237947 shutdown_connections 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 --2- 172.21.15.83:0/3559237947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafa8004c20 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- 172.21.15.83:0/3559237947 >> 172.21.15.83:0/3559237947 conn(0x7fafbc06cd90 msgr2=0x7fafbc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- 172.21.15.83:0/3559237947 shutdown_connections 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- 172.21.15.83:0/3559237947 wait complete. 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 Processor -- start 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.470+0000 7fafc252b640 1 -- start start 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc146810 0x7fafbc1e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafbc1e9a40 0x7fafbc1e9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc160190 con 0x7fafa8004810 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc15c280 con 0x7fafbc146810 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc160310 con 0x7fafbc1e9a40 2024-05-17T22:07:57.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42288/0 (socket says 172.21.15.83:42288) 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 -- 172.21.15.83:0/3054766165 learned_addr learned my addr 172.21.15.83:0/3054766165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafbc1e9a40 0x7fafbc1e9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc146810 0x7fafbc1e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 -- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafbc1e9a40 msgr2=0x7fafbc1e9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafbc1e9a40 0x7fafbc1e9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 -- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc146810 msgr2=0x7fafbc1e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc146810 0x7fafbc1e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 -- 172.21.15.83:0/3054766165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1ea390 con 0x7fafa8004810 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fafa4010900 tx=0x7fafa4010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafb8ff9640 1 -- 172.21.15.83:0/3054766165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400f040 con 0x7fafa8004810 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafb8ff9640 1 -- 172.21.15.83:0/3054766165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa400eb00 con 0x7fafa8004810 2024-05-17T22:07:57.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1e89a0 con 0x7fafa8004810 2024-05-17T22:07:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafb8ff9640 1 -- 172.21.15.83:0/3054766165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa401c400 con 0x7fafa8004810 2024-05-17T22:07:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1e8f60 con 0x7fafa8004810 2024-05-17T22:07:57.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.474+0000 7fafb8ff9640 1 -- 172.21.15.83:0/3054766165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fafa401c5a0 con 0x7fafa8004810 2024-05-17T22:07:57.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafb8ff9640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 0x7faf9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 0x7faf9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafb8ff9640 1 -- 172.21.15.83:0/3054766165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fafa408f910 con 0x7fafa8004810 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 0x7faf9c071940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fafac0ad000 tx=0x7fafac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 Processor -- start 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 -- start start 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dd31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dd328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c6dbd0a90 con 0x562c6dbb8880 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c6db44120 con 0x562c6dbb9090 2024-05-17T22:07:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562c6db33cc0 con 0x562c6dbcbf80 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42300/0 (socket says 172.21.15.83:42300) 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 -- 172.21.15.83:0/1822054131 learned_addr learned my addr 172.21.15.83:0/1822054131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dd328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbbfff640 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dd31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 -- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 msgr2=0x562c6dd31e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dd31e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 -- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 msgr2=0x562c6dd328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dd328a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 -- 172.21.15.83:0/1822054131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6daf0110 con 0x562c6dbb8880 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fafac005040 tx=0x7fafac0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21143426e6876bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafb1ffb640 1 -- 172.21.15.83:0/1822054131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0ca070 con 0x562c6dbb8880 2024-05-17T22:07:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafb1ffb640 1 -- 172.21.15.83:0/1822054131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac005620 con 0x562c6dbb8880 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.478+0000 7fafb1ffb640 1 -- 172.21.15.83:0/1822054131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0a9470 con 0x562c6dbb8880 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/1822054131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 msgr2=0x562c6dd32360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fafac005040 tx=0x7fafac0014b0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/1822054131 shutdown_connections 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dd328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb8880 0x562c6dd32360 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- 172.21.15.83:0/1822054131 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dd31e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/1822054131 >> 172.21.15.83:0/1822054131 conn(0x562c6dc07010 msgr2=0x562c6dce75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/1822054131 shutdown_connections 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/1822054131 wait complete. 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- start start 2024-05-17T22:07:57.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 Processor -- start 2024-05-17T22:07:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dceea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dcf3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c6daefec0 con 0x562c6dbb9090 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562c6db37d00 con 0x562c6dbb8880 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562c6d7fd680 con 0x562c6dbcbf80 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 10 client.0 _ll_register_callbacks cb 0x562c6dbeb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42306/0 (socket says 172.21.15.83:42306) 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3101460991 learned_addr learned my addr 172.21.15.83:0/3101460991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dceea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dcf3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 msgr2=0x562c6dcf3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dcf3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 msgr2=0x562c6dceea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dceea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 -- 172.21.15.83:0/3101460991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6daf2890 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fafac0c8040 tx=0x7fafac0194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5d92c029254df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac0ca070 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fafac0057d0 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafac005620 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fafac0bb940 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 10 client.8340 fetch_fsmap learned FSMap version 102 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x562c6daf4650 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac005620 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7fafac0c4030 con 0x562c6dbb9090 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 10 client.8340 fetch_fsmap finished waiting for FSMap version 102 2024-05-17T22:07:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 10 client.8340 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 20 client.8340 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7da930 0x562c6dd38e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 4 client.8340 mds_command: new command op to 7980 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.482+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 --> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c6dacfc40 con 0x562c6d7da930 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7da930 0x562c6dd38e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafbbfff640 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7da930 0x562c6dd38e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e4f1ec9bfa9e08 server_cookie=ab1dcc5936315970 in_seq=0 out_seq=0 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7faf9affd640 10 client.8340 ms_handle_connect on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7faf9affd640 1 -- 172.21.15.83:0/3101460991 <== mds.0 v2:172.21.15.83:6835/78757339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c6dacfc40 con 0x562c6d7da930 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7faf9affd640 10 client.8340 handle_command_reply: tid=0 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 client.8340 shutdown 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 msgr2=0x562c6dceef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fafac0c8040 tx=0x7fafac0194b0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 shutdown_connections 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7da930 0x562c6dd38e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbcbf80 0x562c6dcf3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dceef70 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7faf9affd640 0 client.8340 ms_handle_reset on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 --2- 172.21.15.83:0/3101460991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dceea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 >> 172.21.15.83:0/3101460991 conn(0x562c6dc07010 msgr2=0x562c6dce75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 shutdown_connections 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 1 -- 172.21.15.83:0/3101460991 wait complete. 2024-05-17T22:07:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.486+0000 7fafc97a5000 20 client.8340 trim_cache size 0 max 16384 2024-05-17T22:07:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 Processor -- start 2024-05-17T22:07:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- start start 2024-05-17T22:07:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dc07010 0x562c6dda0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dda10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc202060 con 0x562c6dbb9090 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc2010c0 con 0x562c6dbb8880 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc15e730 con 0x562c6dc07010 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42310/0 (socket says 172.21.15.83:42310) 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 -- 172.21.15.83:0/2818826095 learned_addr learned my addr 172.21.15.83:0/2818826095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 -- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dc07010 msgr2=0x562c6dda0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dc07010 0x562c6dda0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 -- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 msgr2=0x562c6dda10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dda10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 -- 172.21.15.83:0/2818826095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6dad38c0 con 0x562c6dbb9090 2024-05-17T22:07:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c10 tx=0x7fafb4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ade689df3d46b11 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafb17fa640 1 -- 172.21.15.83:0/2818826095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4010040 con 0x562c6dbb9090 2024-05-17T22:07:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafb17fa640 1 -- 172.21.15.83:0/2818826095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb4011b50 con 0x562c6dbb9090 2024-05-17T22:07:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafb17fa640 1 -- 172.21.15.83:0/2818826095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4015410 con 0x562c6dbb9090 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- 172.21.15.83:0/2818826095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 msgr2=0x562c6dda1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fafb4000c10 tx=0x7fafb4009040 comp rx=0 tx=0).stop 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- 172.21.15.83:0/2818826095 shutdown_connections 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dbb9090 0x562c6dda1630 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb8880 0x562c6dda10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 --2- 172.21.15.83:0/2818826095 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dc07010 0x562c6dda0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.502+0000 7fafc97a5000 1 -- 172.21.15.83:0/2818826095 >> 172.21.15.83:0/2818826095 conn(0x562c6dceb400 msgr2=0x562c6dced9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- 172.21.15.83:0/2818826095 shutdown_connections 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- 172.21.15.83:0/2818826095 wait complete. 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- start start 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 Processor -- start 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbb8880 0x562c6dda4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dda5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c6db37010 con 0x562c6dc07010 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc15ed80 con 0x562c6dbb9090 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x562c6db33cc0 con 0x562c6dbb8880 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 10 client.0 _ll_register_callbacks cb 0x7fafac0c8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42318/0 (socket says 172.21.15.83:42318) 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 -- 172.21.15.83:0/831021013 learned_addr learned my addr 172.21.15.83:0/831021013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafbb7fe640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dda5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 -- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbb8880 msgr2=0x562c6dda4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbb8880 0x562c6dda4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 -- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 msgr2=0x562c6dda5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dda5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 -- 172.21.15.83:0/831021013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6daeefa0 con 0x562c6dc07010 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc0ac8640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0cea50 tx=0x7fafb4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fea64094b2cf39 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4015b50 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7fafb4015cf0 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb4011b50 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fafb402d050 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 10 client.8346 fetch_fsmap learned FSMap version 102 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x562c6d7ee220 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb4011b50 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 102) ==== 3494+0+0 (secure 0 0 0) 0x7fafb4030020 con 0x562c6dc07010 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 10 client.8346 fetch_fsmap finished waiting for FSMap version 102 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 10 client.8346 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 20 client.8346 populate_metadata read hostname 'smithi083' 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7f3fe0 0x562c6dce9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 4 client.8346 mds_command: new command op to 7980 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 --> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c6db46070 con 0x562c6d7f3fe0 2024-05-17T22:07:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.506+0000 7fafbbfff640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7f3fe0 0x562c6dce9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.510+0000 7faf9affd640 10 client.8346 ms_handle_connect on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:57.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.510+0000 7fafbbfff640 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7f3fe0 0x562c6dce9770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f053a3d04526ae1 server_cookie=49579bc5880a696b in_seq=0 out_seq=0 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7faf9affd640 1 -- 172.21.15.83:0/831021013 <== mds.0 v2:172.21.15.83:6835/78757339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c6db46070 con 0x562c6d7f3fe0 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7faf9affd640 10 client.8346 handle_command_reply: tid=0 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 client.8346 shutdown 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 msgr2=0x562c6dce9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0cea50 tx=0x7fafb4001a00 comp rx=0 tx=0).stop 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 shutdown_connections 2024-05-17T22:07:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:6835/78757339,v1:172.21.15.83:6837/78757339] conn(0x562c6d7f3fe0 0x562c6dce9770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c6dc07010 0x562c6dce9230 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c6dbb9090 0x562c6dda5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7faf9affd640 0 client.8346 ms_handle_reset on v2:172.21.15.83:6835/78757339 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 --2- 172.21.15.83:0/831021013 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562c6dbb8880 0x562c6dda4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 >> 172.21.15.83:0/831021013 conn(0x562c6dceb400 msgr2=0x562c6dcee3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 shutdown_connections 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 1 -- 172.21.15.83:0/831021013 wait complete. 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.518+0000 7fafc97a5000 20 client.8346 trim_cache size 0 max 16384 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0 2024-05-17T22:07:57.511 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:07:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 msgr2=0x7faf9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 0x7faf9c071940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fafac0ad000 tx=0x7fafac009040 comp rx=0 tx=0).stop 2024-05-17T22:07:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 msgr2=0x7fafbc146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fafa4010900 tx=0x7fafa4010dc0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 shutdown_connections 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fafbc1e9a40 0x7fafbc1e9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7faf9c06f430 0x7faf9c071940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc146810 0x7fafbc1e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 --2- 172.21.15.83:0/3054766165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8004810 0x7fafbc146060 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 >> 172.21.15.83:0/3054766165 conn(0x7fafbc06cd90 msgr2=0x7fafbc072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 shutdown_connections 2024-05-17T22:07:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.522+0000 7fafc252b640 1 -- 172.21.15.83:0/3054766165 wait complete. 2024-05-17T22:07:57.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:07:57.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.678+0000 7f56ae398640 1 Processor -- start 2024-05-17T22:07:57.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.678+0000 7f56ae398640 1 -- start start 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f56a80b3d00 0x7f56a80d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a8072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56a80733c0 0x7f56a806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f56a80b4110 con 0x7f56a80733c0 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f56a8070770 con 0x7f56a80b3d00 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ae398640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80737d0 con 0x7f56a8143f20 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56a80733c0 0x7f56a806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56a80733c0 0x7f56a806eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42328/0 (socket says 172.21.15.83:42328) 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 -- 172.21.15.83:0/2655486460 learned_addr learned my addr 172.21.15.83:0/2655486460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56a77fe640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a8072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56a7fff640 1 --2- 172.21.15.83:0/2655486460 >> v2:172.21.15.155:3300/0 conn(0x7f56a80b3d00 0x7f56a80d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 -- 172.21.15.83:0/2655486460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 msgr2=0x7f56a8072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a8072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 -- 172.21.15.83:0/2655486460 >> v2:172.21.15.155:3300/0 conn(0x7f56a80b3d00 msgr2=0x7f56a80d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 --2- 172.21.15.83:0/2655486460 >> v2:172.21.15.155:3300/0 conn(0x7f56a80b3d00 0x7f56a80d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 -- 172.21.15.83:0/2655486460 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a80d10d0 con 0x7f56a80733c0 2024-05-17T22:07:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56ac935640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a806eec0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f56a806d6e0 tx=0x7f56a001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc9218e5c50a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56a6ffd640 1 -- 172.21.15.83:0/2655486460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a5840 con 0x7f56a80733c0 2024-05-17T22:07:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56a6ffd640 1 -- 172.21.15.83:0/2655486460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a00a59e0 con 0x7f56a80733c0 2024-05-17T22:07:57.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.682+0000 7f56a6ffd640 1 -- 172.21.15.83:0/2655486460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00a5db0 con 0x7f56a80733c0 2024-05-17T22:07:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- 172.21.15.83:0/2655486460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 msgr2=0x7f56a806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a806eec0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f56a806d6e0 tx=0x7f56a001c9c0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- 172.21.15.83:0/2655486460 shutdown_connections 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a806eec0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- 172.21.15.83:0/2655486460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a8072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- 172.21.15.83:0/2655486460 >> v2:172.21.15.155:3300/0 conn(0x7f56a80b3d00 0x7f56a80d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- 172.21.15.83:0/2655486460 >> 172.21.15.83:0/2655486460 conn(0x7f56a806cd90 msgr2=0x7f56a80b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- 172.21.15.83:0/2655486460 shutdown_connections 2024-05-17T22:07:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- 172.21.15.83:0/2655486460 wait complete. 2024-05-17T22:07:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 Processor -- start 2024-05-17T22:07:57.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- start start 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 0x7f56a81431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a80b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8070c00 con 0x7f56a80733c0 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8144330 con 0x7f56a80b3d00 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ae398640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f56a80b5990 con 0x7f56a8143f20 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42344/0 (socket says 172.21.15.83:42344) 2024-05-17T22:07:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 -- 172.21.15.83:0/3563139837 learned_addr learned my addr 172.21.15.83:0/3563139837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a77fe640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 0x7f56a81431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 -- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 msgr2=0x7f56a80b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56ac935640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a80b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a80b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 -- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 msgr2=0x7f56a81431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.686+0000 7f56a7fff640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 0x7f56a81431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a7fff640 1 -- 172.21.15.83:0/3563139837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a8139e70 con 0x7f56a80733c0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a77fe640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 0x7f56a81431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a7fff640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f569800c820 tx=0x7f569800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698015070 con 0x7f56a80733c0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5698007a30 con 0x7f56a80733c0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698007e20 con 0x7f56a80733c0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a80aeae0 con 0x7f56a80733c0 2024-05-17T22:07:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a80af0a0 con 0x7f56a80733c0 2024-05-17T22:07:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a806fc10 con 0x7f56a80733c0 2024-05-17T22:07:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5698018070 con 0x7f56a80733c0 2024-05-17T22:07:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 0x7f5694071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5795+0+0 (secure 0 0 0) 0x7f569808f1a0 con 0x7f56a80733c0 2024-05-17T22:07:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a77fe640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 0x7f5694071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.690+0000 7f56a77fe640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 0x7f5694071940 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56a8145eb0 tx=0x7f569c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.694+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f569805c030 con 0x7f56a80733c0 2024-05-17T22:07:57.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.878+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f56a81467a0 con 0x7f56a80733c0 2024-05-17T22:07:57.869 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:07:57.878+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:07:57.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.882+0000 7f56a4ff9640 1 -- 172.21.15.83:0/3563139837 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) ==== 161+0+0 (secure 0 0 0) 0x7f569805b930 con 0x7f56a80733c0 2024-05-17T22:07:57.872 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:07:57.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 msgr2=0x7f5694071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 0x7f5694071940 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56a8145eb0 tx=0x7f569c008040 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 msgr2=0x7f56a8139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f569800c820 tx=0x7f569800cce0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 shutdown_connections 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56a8143f20 0x7f56a80b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f569406f430 0x7f5694071940 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56a80b3d00 0x7f56a81431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 --2- 172.21.15.83:0/3563139837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56a80733c0 0x7f56a8139930 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 >> 172.21.15.83:0/3563139837 conn(0x7f56a806cd90 msgr2=0x7f56a8075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 shutdown_connections 2024-05-17T22:07:57.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.886+0000 7f56ae398640 1 -- 172.21.15.83:0/3563139837 wait complete. 2024-05-17T22:07:57.880 INFO:tasks.ceph.mds.c.smithi083.stderr: -1952> 2024-05-17T22:07:48.462+0000 7f0cb83c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:07:57.896 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-17T22:07:57.942 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:07:57.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 -- start start 2024-05-17T22:07:57.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 Processor -- start 2024-05-17T22:07:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b7680 0x55ad472cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 0x55ad472ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e4600 con 0x55ad465b7200 2024-05-17T22:07:57.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e4480 con 0x55ad465b8400 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ad472e4300 con 0x55ad465b7680 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c041b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c041b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42382/0 (socket says 172.21.15.83:42382) 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.966+0000 7ff4c041b640 1 -- 172.21.15.83:0/553215780 learned_addr learned my addr 172.21.15.83:0/553215780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 -- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b7680 msgr2=0x55ad472cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 0x55ad472ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b7680 0x55ad472cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b7680 0x55ad472cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 -- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 msgr2=0x55ad472ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 0x55ad472ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 -- 172.21.15.83:0/553215780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad472c6960 con 0x55ad465b7200 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 0x55ad472ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c041b640 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55ad4735f8f0 tx=0x55ad4660e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8d68e5f18ec7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bd415640 1 -- 172.21.15.83:0/553215780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652e4e0 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bd415640 1 -- 172.21.15.83:0/553215780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ad46542540 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bd415640 1 -- 172.21.15.83:0/553215780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652eb60 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad472c6d20 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ad472c6f00 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 4 waiting for MDS map... 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bd415640 1 -- 172.21.15.83:0/553215780 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x55ad472d0e00 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bd415640 1 -- 172.21.15.83:0/553215780 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 103) ==== 2885+0+0 (secure 0 0 0) 0x55ad4654f180 con 0x55ad465b7200 2024-05-17T22:07:57.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 4 Got MDS map 103 2024-05-17T22:07:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 10 main: JournalTool::main 2024-05-17T22:07:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 4 main: JournalTool: connecting to RADOS... 2024-05-17T22:07:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 Processor -- start 2024-05-17T22:07:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- start start 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6900 0x55ad472cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e4180 con 0x55ad465b6000 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ad46559380 con 0x55ad465b6900 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ad46559800 con 0x55ad465b6480 2024-05-17T22:07:57.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42384/0 (socket says 172.21.15.83:42384) 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 -- 172.21.15.83:0/813076127 learned_addr learned my addr 172.21.15.83:0/813076127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6900 0x55ad472cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 -- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 msgr2=0x55ad472cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 -- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6900 msgr2=0x55ad472cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6900 0x55ad472cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 -- 172.21.15.83:0/813076127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad472c6d20 con 0x55ad465b6000 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bf419640 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x55ad4735eed0 tx=0x55ad4660ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5413db2c7723c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bb411640 1 -- 172.21.15.83:0/813076127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652e9c0 con 0x55ad465b6000 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bb411640 1 -- 172.21.15.83:0/813076127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ad46542c40 con 0x55ad465b6000 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.970+0000 7ff4bb411640 1 -- 172.21.15.83:0/813076127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652e4e0 con 0x55ad465b6000 2024-05-17T22:07:57.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/813076127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 msgr2=0x55ad472ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x55ad4735eed0 tx=0x55ad4660ea80 comp rx=0 tx=0).stop 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/813076127 shutdown_connections 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6000 0x55ad472ce100 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- 172.21.15.83:0/813076127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6900 0x55ad472cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/813076127 >> 172.21.15.83:0/813076127 conn(0x55ad465b6d80 msgr2=0x55ad424f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/813076127 shutdown_connections 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/813076127 wait complete. 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 Processor -- start 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- start start 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6000 0x55ad472cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e5200 con 0x55ad465b6900 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e4180 con 0x55ad465b6000 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ad472e5380 con 0x55ad465b6480 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42394/0 (socket says 172.21.15.83:42394) 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 -- 172.21.15.83:0/3530696751 learned_addr learned my addr 172.21.15.83:0/3530696751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6000 0x55ad472cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 -- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 msgr2=0x55ad472cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 -- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6000 msgr2=0x55ad472cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6000 0x55ad472cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 -- 172.21.15.83:0/3530696751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad472c7a40 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x55ad474395f0 tx=0x55ad4660ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4b9c0e640 1 -- 172.21.15.83:0/3530696751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652eb60 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4b9c0e640 1 -- 172.21.15.83:0/3530696751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ad46542a80 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ad472c7c20 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4b9c0e640 1 -- 172.21.15.83:0/3530696751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652e000 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad472c61e0 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.974+0000 7ff4c145b400 4 main: JournalTool: resolving pool 16 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4b9c0e640 1 -- 172.21.15.83:0/3530696751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55ad46820c00 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4b9c0e640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 0x55ad472cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4b9c0e640 1 -- 172.21.15.83:0/3530696751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x55ad46834400 con 0x55ad465b6900 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 0x55ad472cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 4 main: JournalTool: creating IoCtx.. 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 4 main: Executing for rank 0 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 0x55ad472cf700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55ad4735f7a0 tx=0x55ad4743f260 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 -- start start 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 Processor -- start 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b9600 0x55ad472cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b9a80 0x55ad472ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e5680 con 0x55ad465b8d00 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ad472e5800 con 0x55ad465b9a80 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c145b400 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55ad46559800 con 0x55ad465b9600 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42402/0 (socket says 172.21.15.83:42402) 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 -- 172.21.15.83:0/3293890944 learned_addr learned my addr 172.21.15.83:0/3293890944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b9600 msgr2=0x55ad472cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b9600 0x55ad472cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b9a80 msgr2=0x55ad472ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b9a80 0x55ad472ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.978+0000 7ff4c041b640 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad4698a5a0 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c041b640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x55ad468d5dd0 tx=0x55ad4660e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eaf2ef9e3e2e84 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4b6c08640 1 -- 172.21.15.83:0/3293890944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad4652e340 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4b6c08640 1 -- 172.21.15.83:0/3293890944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ad46543500 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4b6c08640 1 -- 172.21.15.83:0/3293890944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ad46863d40 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad4698a780 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ad4698a960 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 4 waiting for MDS map... 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4b6c08640 1 -- 172.21.15.83:0/3293890944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x55ad4699a200 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4b6c08640 1 -- 172.21.15.83:0/3293890944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 103) ==== 2885+0+0 (secure 0 0 0) 0x55ad4654f500 con 0x55ad465b8d00 2024-05-17T22:07:57.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 4 Got MDS map 103 2024-05-17T22:07:57.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55ad469a8000 0x55ad469b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4000 con 0x55ad469a8000 2024-05-17T22:07:57.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55ad469a8000 0x55ad469b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.982+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55ad469a8000 0x55ad469b0000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=22b82ccac82938f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4bfc1a640 1 -- 172.21.15.83:0/3293890944 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v111'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc000 con 0x55ad469a8000 2024-05-17T22:07:57.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4400 con 0x55ad469a8000 2024-05-17T22:07:57.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4bfc1a640 1 -- 172.21.15.83:0/3293890944 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v111'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc000 con 0x55ad469a8000 2024-05-17T22:07:57.980 INFO:teuthology.orchestra.run.smithi083.stdout:writing EResetJournal entry 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4800 con 0x55ad469a8000 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 0x55ad469b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4c00 con 0x55ad469a8480 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55ad469a8900 0x55ad469b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b5000 con 0x55ad469a8900 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4000 con 0x55ad469a8480 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55ad469a8d80 0x55ad469b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4400 con 0x55ad469a8d80 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 0x55ad469b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.986+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b4800 con 0x55ad469a9200 2024-05-17T22:07:57.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4c041b640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55ad469a8900 0x55ad469b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55ad469a8d80 0x55ad469b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 0x55ad469b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 0x55ad469b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bfc1a640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55ad469a8d80 0x55ad469b1080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=5beb2647e6914db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4c041b640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55ad469a8900 0x55ad469b0b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=ad07b778f720b2a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 0x55ad469b0580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=3f8dbce466a24c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bfc1a640 1 -- 172.21.15.83:0/3293890944 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(3 200.00000001 [delete] v111'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc000 con 0x55ad469a8000 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bf419640 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 0x55ad469b1600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=d2d795edc40d898f server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:57.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.990+0000 7ff4bfc1a640 1 -- 172.21.15.83:0/3293890944 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(7 200.00000005 [delete] v111'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc000 con 0x55ad469a8d80 2024-05-17T22:07:57.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.994+0000 7ff4bf419640 1 -- 172.21.15.83:0/3293890944 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(4 200.00000002 [delete] v111'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc500 con 0x55ad469a8480 2024-05-17T22:07:57.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.994+0000 7ff4b5405640 1 -- 172.21.15.83:0/3293890944 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55ad469b5400 con 0x55ad469a8000 2024-05-17T22:07:57.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.994+0000 7ff4c041b640 1 -- 172.21.15.83:0/3293890944 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(5 200.00000003 [delete] v111'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc780 con 0x55ad469a8900 2024-05-17T22:07:57.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.994+0000 7ff4bf419640 1 -- 172.21.15.83:0/3293890944 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(6 200.00000004 [delete] v111'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc500 con 0x55ad469a8480 2024-05-17T22:07:57.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.994+0000 7ff4bfc1a640 1 -- 172.21.15.83:0/3293890944 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v111'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc000 con 0x55ad469a8000 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4bf419640 1 -- 172.21.15.83:0/3293890944 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(8 200.00000006 [delete] v111'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55ad465fc500 con 0x55ad469a9200 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55ad469a8000 msgr2=0x55ad469b0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55ad469a8000 0x55ad469b0000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55ad469a8900 msgr2=0x55ad469b0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55ad469a8900 0x55ad469b0b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55ad469a8d80 msgr2=0x55ad469b1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:57.998+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55ad469a8d80 0x55ad469b1080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4bf419640 1 -- 172.21.15.83:0/3293890944 reap_dead start 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 msgr2=0x55ad469b0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 0x55ad469b0580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 msgr2=0x55ad469b1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 0x55ad469b1600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 msgr2=0x55ad472cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x55ad468d5dd0 tx=0x55ad4660e840 comp rx=0 tx=0).stop 2024-05-17T22:07:57.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 shutdown_connections 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55ad469a9200 0x55ad469b1600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55ad469a8480 0x55ad469b0580 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3293890944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b8d00 0x55ad472cd600 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 >> 172.21.15.83:0/3293890944 conn(0x55ad465b8880 msgr2=0x55ad46934800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 shutdown_connections 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3293890944 wait complete. 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 msgr2=0x55ad472cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 0x55ad472cf700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55ad4735f7a0 tx=0x55ad4743f260 comp rx=0 tx=0).stop 2024-05-17T22:07:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 msgr2=0x55ad472ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x55ad474395f0 tx=0x55ad4660ea20 comp rx=0 tx=0).stop 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 shutdown_connections 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b6900 0x55ad472ce680 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b6480 0x55ad472cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55ad465b9180 0x55ad472cf700 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/3530696751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b6000 0x55ad472cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 >> 172.21.15.83:0/3530696751 conn(0x55ad465b6d80 msgr2=0x55ad424f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 shutdown_connections 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/3530696751 wait complete. 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 msgr2=0x55ad472cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x55ad4735f8f0 tx=0x55ad4660e720 comp rx=0 tx=0).stop 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 shutdown_connections 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad465b7200 0x55ad472cd080 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ad465b8400 0x55ad472ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 --2- 172.21.15.83:0/553215780 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55ad465b7680 0x55ad472cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 >> 172.21.15.83:0/553215780 conn(0x55ad465b7f80 msgr2=0x55ad424f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 shutdown_connections 2024-05-17T22:07:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.002+0000 7ff4c145b400 1 -- 172.21.15.83:0/553215780 wait complete. 2024-05-17T22:07:57.998 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.101947, result: writing EResetJournal entry 2024-05-17T22:07:57.998 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-17T22:07:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.114+0000 7f6ae970a8c0 1 -- start start 2024-05-17T22:07:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.114+0000 7f6ae970a8c0 1 Processor -- start 2024-05-17T22:07:58.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d449f80 0x55642e1aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d449200 0x55642e1af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55642e1ba180 con 0x55642d449680 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55642e1ba300 con 0x55642d449200 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55642e1ba480 con 0x55642d449f80 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42416/0 (socket says 172.21.15.83:42416) 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/2656216079 learned_addr learned my addr 172.21.15.83:0/2656216079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d449f80 0x55642e1aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d449200 0x55642e1af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d449f80 msgr2=0x55642e1aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d449f80 0x55642e1aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d449200 msgr2=0x55642e1af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d449200 0x55642e1af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/2656216079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564293ca1e0 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x55642e239ce0 tx=0x55642d4ec900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29ca15cc7dd06e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae56c4640 1 -- 172.21.15.83:0/2656216079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40a4e0 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae56c4640 1 -- 172.21.15.83:0/2656216079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55642d41ee00 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae56c4640 1 -- 172.21.15.83:0/2656216079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40ab60 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564293cb2c0 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5564293cba40 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 4 waiting for MDS map... 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae56c4640 1 -- 172.21.15.83:0/2656216079 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x55642e302a00 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae56c4640 1 -- 172.21.15.83:0/2656216079 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 103) ==== 2885+0+0 (secure 0 0 0) 0x55642d42b180 con 0x55642d449680 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 4 Got MDS map 103 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 4 datascan.main: Using metadata pool output 2024-05-17T22:07:58.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:07:58.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 Processor -- start 2024-05-17T22:07:58.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.118+0000 7f6ae970a8c0 1 -- start start 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d448900 0x55642e1af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55642e1ba600 con 0x55642d44a880 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55642e1ba780 con 0x55642d44ad00 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55642e1ba900 con 0x55642d448900 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42432/0 (socket says 172.21.15.83:42432) 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/4135209269 learned_addr learned my addr 172.21.15.83:0/4135209269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d448900 0x55642e1af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d448900 msgr2=0x55642e1af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d448900 0x55642e1af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 msgr2=0x55642e1b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/4135209269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564293cbc20 con 0x55642d44a880 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x55642e2f30b0 tx=0x55642d4e9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e471d112483403 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae36c0640 1 -- 172.21.15.83:0/4135209269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40a340 con 0x55642d44a880 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae36c0640 1 -- 172.21.15.83:0/4135209269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55642d41ec40 con 0x55642d44a880 2024-05-17T22:07:58.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae36c0640 1 -- 172.21.15.83:0/4135209269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40a4e0 con 0x55642d44a880 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/4135209269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 msgr2=0x55642e1afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x55642e2f30b0 tx=0x55642d4e9f20 comp rx=0 tx=0).stop 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/4135209269 shutdown_connections 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d44a880 0x55642e1afb80 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/4135209269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d448900 0x55642e1af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/4135209269 >> 172.21.15.83:0/4135209269 conn(0x55642d448d80 msgr2=0x55642d6fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/4135209269 shutdown_connections 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/4135209269 wait complete. 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 Processor -- start 2024-05-17T22:07:58.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- start start 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d44a880 0x55642e1b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55642e1bbc80 con 0x55642d448900 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55642e1bbb00 con 0x55642d44ad00 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae970a8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55642e1bb980 con 0x55642d44a880 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae86ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae86ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42436/0 (socket says 172.21.15.83:42436) 2024-05-17T22:07:58.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.122+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 learned_addr learned my addr 172.21.15.83:0/1530195916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d44a880 msgr2=0x55642e1b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d44a880 0x55642e1b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 msgr2=0x55642e1b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d44ad00 0x55642e1b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564293ca1e0 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x55642d714060 tx=0x55642e312e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 -- 172.21.15.83:0/1530195916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40a340 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 -- 172.21.15.83:0/1530195916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55642d41e540 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5564293cb2c0 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 -- 172.21.15.83:0/1530195916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55642d40a4e0 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564293cbe00 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 4 datascan.init: resolving metadata pool 16 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 -- 172.21.15.83:0/1530195916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x55642d70cc00 con 0x55642d448900 2024-05-17T22:07:58.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 0x55642e1b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae1ebd640 1 -- 172.21.15.83:0/1530195916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x55642d716a00 con 0x55642d448900 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 0x55642e1b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55642d44b180 0x55642e1b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718800 con 0x55642d44b180 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 0x55642e1b1700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55642e2f30b0 tx=0x55642e312c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55642d44b180 0x55642e1b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.126+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55642d44b180 0x55642e1b0100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(1 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55642d818000 0x55642d820000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719000 con 0x55642d818000 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55642d818000 0x55642d820000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55642d818000 0x55642d820000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(2 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae36c0640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55642d818480 0x55642d820580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719400 con 0x55642d818480 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55642d818480 0x55642d820580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55642d818480 0x55642d820580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(3 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55642d818900 0x55642d820b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719800 con 0x55642d818900 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55642d818900 0x55642d820b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55642d818900 0x55642d820b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(4 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5c00 con 0x55642d44b180 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(5 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719c00 con 0x55642d818480 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(6 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5400 con 0x55642d818900 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(7 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.130+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718800 con 0x55642d44b180 2024-05-17T22:07:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 3 ==== osd_op_reply(8 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838000 con 0x55642d818900 2024-05-17T22:07:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(9 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719800 con 0x55642d818480 2024-05-17T22:07:58.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(10 [pgnls start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838400 con 0x55642d818480 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 4 ==== osd_op_reply(11 [pgnls start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5800 con 0x55642d818900 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(12 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838800 con 0x55642d44b180 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 4 ==== osd_op_reply(13 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae2ebf640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55642d818d80 0x55642d821080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5c00 con 0x55642d818d80 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55642d818d80 0x55642d821080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.134+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55642d818d80 0x55642d821080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(14 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838000 con 0x55642d818000 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(15 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5400 con 0x55642d818900 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 5 ==== osd_op_reply(16 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838800 con 0x55642d818d80 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(17 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae2ebf640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55642d819200 0x55642d821600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5800 con 0x55642d819200 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55642d819200 0x55642d821600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae7ec9640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55642d819200 0x55642d821600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(18 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838c00 con 0x55642d818480 2024-05-17T22:07:58.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 5 ==== osd_op_reply(19 [pgnls start_epoch 111 out=81b] v110'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x55642d718c00 con 0x55642d818480 2024-05-17T22:07:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae970a8c0 10 datascan.operator(): handling object 1099511627778.0 2024-05-17T22:07:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.138+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55642d719400 con 0x55642d818480 2024-05-17T22:07:58.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 7 ==== osd_op_reply(21 10000000002.00000000 [call] v111'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719c00 con 0x55642d44b180 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 5 ==== osd_op_reply(22 [pgnls start_epoch 111 out=49b] v109'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839000 con 0x55642d44b180 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 6 ==== osd_op_reply(23 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718800 con 0x55642d818480 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 8 ==== osd_op_reply(24 [pgnls start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.142+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838400 con 0x55642d818900 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 6 ==== osd_op_reply(25 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838800 con 0x55642d818000 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(26 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae36c0640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55642d819f80 0x55642d822680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838000 con 0x55642d819f80 2024-05-17T22:07:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55642d819f80 0x55642d822680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55642d819f80 0x55642d822680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(27 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839400 con 0x55642d818900 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 7 ==== osd_op_reply(28 [pgnls start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5800 con 0x55642d819200 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(29 [pgnls start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839800 con 0x55642d818900 2024-05-17T22:07:58.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 8 ==== osd_op_reply(30 [pgnls start_epoch 111 out=81b] v109'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x55642d719800 con 0x55642d818900 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae970a8c0 10 datascan.operator(): handling object 1099511628279.0 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.146+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55642d839000 con 0x55642d818900 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v111'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55642d44b600 0x55642e1afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838400 con 0x55642d44b600 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55642d44b600 0x55642e1afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae86ca640 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55642d44b600 0x55642e1afb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(33 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d44b600 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839c00 con 0x55642d819f80 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(34 [pgnls start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5c00 con 0x55642d818000 2024-05-17T22:07:58.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(35 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838000 con 0x55642d818d80 2024-05-17T22:07:58.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(36 [pgnls start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719000 con 0x55642d818900 2024-05-17T22:07:58.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 11 ==== osd_op_reply(37 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.150+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839400 con 0x55642d818900 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 12 ==== osd_op_reply(38 [pgnls start_epoch 111 out=49b] v109'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856000 con 0x55642d818480 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 9 ==== osd_op_reply(39 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839800 con 0x55642d818000 2024-05-17T22:07:58.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(40 [pgnls start_epoch 111 out=81b] v109'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x55642d719c00 con 0x55642d818000 2024-05-17T22:07:58.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae970a8c0 10 datascan.operator(): handling object 1099511627780.1 2024-05-17T22:07:58.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5400 con 0x55642d819f80 2024-05-17T22:07:58.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(42 10000000004.00000000 [call] v111'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838800 con 0x55642d818480 2024-05-17T22:07:58.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 10 ==== osd_op_reply(43 [pgnls start_epoch 111 out=49b] v109'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.154+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856000 con 0x55642d818d80 2024-05-17T22:07:58.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(44 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856400 con 0x55642d819200 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(45 [pgnls start_epoch 111 out=81b] v110'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x55642d838400 con 0x55642d819200 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae970a8c0 10 datascan.operator(): handling object 1099511628280.0 2024-05-17T22:07:58.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.158+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55642d838000 con 0x55642d819200 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v111'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839800 con 0x55642d819200 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(48 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856800 con 0x55642d819f80 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(49 [pgnls start_epoch 111 out=81b] v111'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x55642d838c00 con 0x55642d819f80 2024-05-17T22:07:58.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.162+0000 7f6ae970a8c0 10 datascan.operator(): handling object 1099511627780.0 2024-05-17T22:07:58.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55642d839000 con 0x55642d819f80 2024-05-17T22:07:58.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 6 ==== osd_op_reply(51 10000000004.00000000 [call] v111'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d839c00 con 0x55642d818000 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(52 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856c00 con 0x55642d819200 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.5 v2:172.21.15.155:6808/758721566 7 ==== osd_op_reply(53 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d819200 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.166+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857800 con 0x55642d818d80 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(54 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857000 con 0x55642d44b180 2024-05-17T22:07:58.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 7 ==== osd_op_reply(55 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5c00 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 8 ==== osd_op_reply(56 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857400 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 9 ==== osd_op_reply(57 [pgnls start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5564293a5800 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 10 ==== osd_op_reply(58 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.170+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856800 con 0x55642d818d80 2024-05-17T22:07:58.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 6 ==== osd_op_reply(59 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718c00 con 0x55642d818d80 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 7 ==== osd_op_reply(60 [pgnls start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857c00 con 0x55642d818480 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae7ec9640 1 -- 172.21.15.83:0/1530195916 <== osd.3 v2:172.21.15.83:6824/2257217219 11 ==== osd_op_reply(61 [pgnls start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8780 con 0x55642d818480 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719400 con 0x55642d44b180 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.7 v2:172.21.15.155:6816/3020759009 11 ==== osd_op_reply(62 [pgnls start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d44b180 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838c00 con 0x55642d818900 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 13 ==== osd_op_reply(63 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718800 con 0x55642d818d80 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 8 ==== osd_op_reply(64 [pgnls start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856400 con 0x55642d818d80 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 9 ==== osd_op_reply(65 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d719c00 con 0x55642d44b600 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(66 [pgnls start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d44b600 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.174+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856c00 con 0x55642d818900 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 14 ==== osd_op_reply(67 [pgnls start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d718c00 con 0x55642d818d80 2024-05-17T22:07:58.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 10 ==== osd_op_reply(68 [pgnls start_epoch 111 out=49b] v109'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857c00 con 0x55642d819f80 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 7 ==== osd_op_reply(69 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856000 con 0x55642d818900 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 15 ==== osd_op_reply(70 [pgnls start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838800 con 0x55642d818000 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(71 [pgnls start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818000 2024-05-17T22:07:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d857800 con 0x55642d818d80 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae86ca640 1 -- 172.21.15.83:0/1530195916 <== osd.4 v2:172.21.15.155:6824/505763805 11 ==== osd_op_reply(72 [pgnls start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8500 con 0x55642d818d80 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae36c0640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d838400 con 0x55642d819f80 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.6 v2:172.21.15.155:6800/1025803281 8 ==== osd_op_reply(73 [pgnls start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d819f80 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae2ebf640 1 -- 172.21.15.83:0/1530195916 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x55642d856800 con 0x55642d818900 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 <== osd.0 v2:172.21.15.83:6816/622135753 16 ==== osd_op_reply(74 [pgnls start_epoch 111 out=49b] v109'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55642d4d8000 con 0x55642d818900 2024-05-17T22:07:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55642d818900 msgr2=0x55642d820b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x55642d818900 0x55642d820b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55642d818000 msgr2=0x55642d820000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x55642d818000 0x55642d820000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55642d44b600 msgr2=0x55642e1afb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x55642d44b600 0x55642e1afb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55642d818480 msgr2=0x55642d820580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 reap_dead start 2024-05-17T22:07:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x55642d818480 0x55642d820580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55642d818d80 msgr2=0x55642d821080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x55642d818d80 0x55642d821080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55642d819200 msgr2=0x55642d821600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x55642d819200 0x55642d821600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55642d819f80 msgr2=0x55642d822680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x55642d819f80 0x55642d822680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55642d44b180 msgr2=0x55642e1b0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x55642d44b180 0x55642e1b0100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae76c8640 1 -- 172.21.15.83:0/1530195916 reap_dead start 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 msgr2=0x55642e1b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 0x55642e1b1700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55642e2f30b0 tx=0x55642e312c00 comp rx=0 tx=0).stop 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 msgr2=0x55642e1b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.178+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x55642d714060 tx=0x55642e312e40 comp rx=0 tx=0).stop 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 shutdown_connections 2024-05-17T22:07:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x55642d44ba80 0x55642e1b1700 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/1530195916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d448900 0x55642e1b1180 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 >> 172.21.15.83:0/1530195916 conn(0x55642d448d80 msgr2=0x55642d6fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 shutdown_connections 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/1530195916 wait complete. 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 msgr2=0x55642e1ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x55642e239ce0 tx=0x55642d4ec900 comp rx=0 tx=0).stop 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 shutdown_connections 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55642d449200 0x55642e1af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55642d449f80 0x55642e1aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 --2- 172.21.15.83:0/2656216079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642d449680 0x55642e1ae000 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 >> 172.21.15.83:0/2656216079 conn(0x55642d44a400 msgr2=0x5564293d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 shutdown_connections 2024-05-17T22:07:58.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.182+0000 7f6ae970a8c0 1 -- 172.21.15.83:0/2656216079 wait complete. 2024-05-17T22:07:58.182 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.183719, result: 2024-05-17T22:07:58.182 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-17T22:07:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- start start 2024-05-17T22:07:58.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 Processor -- start 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028d680 0x555b60ff2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028df80 0x555b60ff2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b602fa180 con 0x555b6028db00 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555b602fa300 con 0x555b6028df80 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555b602fa480 con 0x555b6028d680 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42456/0 (socket says 172.21.15.83:42456) 2024-05-17T22:07:58.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 -- 172.21.15.83:0/644095948 learned_addr learned my addr 172.21.15.83:0/644095948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 -- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028d680 msgr2=0x555b60ff2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d61df7640 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028d680 0x555b60ff2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d625f8640 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028df80 0x555b60ff2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028d680 0x555b60ff2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 -- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028df80 msgr2=0x555b60ff2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028df80 0x555b60ff2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 -- 172.21.15.83:0/644095948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b5c20e1e0 con 0x555b6028db00 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d615f6640 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x555b61079c50 tx=0x555b6032e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba6d16d7b675602 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d5f5f2640 1 -- 172.21.15.83:0/644095948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024e4e0 con 0x555b6028db00 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d5f5f2640 1 -- 172.21.15.83:0/644095948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b60262540 con 0x555b6028db00 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d5f5f2640 1 -- 172.21.15.83:0/644095948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024eb60 con 0x555b6028db00 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5c20f4a0 con 0x555b6028db00 2024-05-17T22:07:58.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555b5c20fa40 con 0x555b6028db00 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 4 waiting for MDS map... 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d5f5f2640 1 -- 172.21.15.83:0/644095948 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x555b6113a600 con 0x555b6028db00 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d5f5f2640 1 -- 172.21.15.83:0/644095948 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 103) ==== 2885+0+0 (secure 0 0 0) 0x555b6026f180 con 0x555b6028db00 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 4 Got MDS map 103 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 4 datascan.main: Using metadata pool output 2024-05-17T22:07:58.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.250+0000 7f5d636388c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 Processor -- start 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- start start 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b602fa600 con 0x555b6028e880 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555b602fa780 con 0x555b6028cd80 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555b602fa900 con 0x555b6028c900 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42464/0 (socket says 172.21.15.83:42464) 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 -- 172.21.15.83:0/3724984035 learned_addr learned my addr 172.21.15.83:0/3724984035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d625f8640 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 -- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028c900 msgr2=0x555b60ff3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d61df7640 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 -- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 msgr2=0x555b60ff3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 -- 172.21.15.83:0/3724984035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b5c20fc20 con 0x555b6028e880 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d615f6640 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x555b6114b0e0 tx=0x555b6032e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18faac42fe9c6fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/3724984035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024e340 con 0x555b6028e880 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/3724984035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b60262e00 con 0x555b6028e880 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/3724984035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024e4e0 con 0x555b6028e880 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- 172.21.15.83:0/3724984035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 msgr2=0x555b60ff3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x555b6114b0e0 tx=0x555b6032e6c0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- 172.21.15.83:0/3724984035 shutdown_connections 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 --2- 172.21.15.83:0/3724984035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- 172.21.15.83:0/3724984035 >> 172.21.15.83:0/3724984035 conn(0x555b6028d200 msgr2=0x555b6053a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- 172.21.15.83:0/3724984035 shutdown_connections 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- 172.21.15.83:0/3724984035 wait complete. 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 Processor -- start 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.254+0000 7f5d636388c0 1 -- start start 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555b602fbc80 con 0x555b6028c900 2024-05-17T22:07:58.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x555b602fbb00 con 0x555b6028cd80 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x555b602fb980 con 0x555b6028e880 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 0x555b60ff3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42476/0 (socket says 172.21.15.83:42476) 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 learned_addr learned my addr 172.21.15.83:0/4017824766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028e880 msgr2=0x555b60ff3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 msgr2=0x555b60ff3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b5c20e1e0 con 0x555b6028c900 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028e880 0x555b60ff3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028cd80 0x555b60ff3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:07:58.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 0x555b60ff3600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x555b610792f0 tx=0x555b6032fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 -- 172.21.15.83:0/4017824766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024e9c0 con 0x555b6028c900 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 -- 172.21.15.83:0/4017824766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555b60262c40 con 0x555b6028c900 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b5c20f4a0 con 0x555b6028c900 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 -- 172.21.15.83:0/4017824766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555b6024e340 con 0x555b6028c900 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5c20fa40 con 0x555b6028c900 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 4 datascan.init: resolving metadata pool 16 2024-05-17T22:07:58.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 -- 172.21.15.83:0/4017824766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x555b60550c00 con 0x555b6028c900 2024-05-17T22:07:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x555b6028fa80 0x555b60ff4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d5bdeb640 1 -- 172.21.15.83:0/4017824766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x555b60558600 con 0x555b6028c900 2024-05-17T22:07:58.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x555b6028fa80 0x555b60ff4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 4 datascan.main: opening data pool 'cephfs_data' 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x555b6028ed00 0x555b60ff4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b61139c00 con 0x555b6028ed00 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x555b6028ed00 0x555b60ff4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.258+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x555b6028fa80 0x555b60ff4100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x555b60331020 tx=0x555b61153320 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x555b6028ed00 0x555b60ff4680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x555b6028f180 0x555b60ff4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b60656c00 con 0x555b6028f180 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x555b6028f180 0x555b60ff4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x555b6028f180 0x555b60ff4c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x555b6028f600 0x555b60ff5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657000 con 0x555b6028f600 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x555b6028f600 0x555b60ff5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.262+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x555b6028f600 0x555b60ff5180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5cded640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x555b60670000 0x555b60678000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657800 con 0x555b60670000 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x555b60670000 0x555b60678000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x555b60670000 0x555b60678000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5d5ee640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x555b60670480 0x555b60678580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b60670480 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x555b60670480 0x555b60678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x555b60670480 0x555b60678580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6067c000 con 0x555b6028ed00 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139000 con 0x555b6028f600 2024-05-17T22:07:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6067c400 con 0x555b60670480 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139400 con 0x555b6028ed00 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.266+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6067c000 con 0x555b6028f600 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139000 con 0x555b6028ed00 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657800 con 0x555b60670480 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656800 con 0x555b60670480 2024-05-17T22:07:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139c00 con 0x555b6028ed00 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657000 con 0x555b6028f600 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5cded640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x555b60670900 0x555b60678b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656c00 con 0x555b60670900 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x555b60670900 0x555b60678b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.270+0000 7f5d61df7640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x555b60670900 0x555b60678b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686000 con 0x555b60670000 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b6028ed00 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686400 con 0x555b60670900 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657800 con 0x555b6028f180 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 111 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686800 con 0x555b60670480 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 111 out=81b] v111'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657400 con 0x555b6028f600 2024-05-17T22:07:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686c00 con 0x555b60670000 2024-05-17T22:07:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.274+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656800 con 0x555b60670480 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686000 con 0x555b6028ed00 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686c00 con 0x555b6028f600 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b60670480 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687000 con 0x555b6028ed00 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139400 con 0x555b6028f600 2024-05-17T22:07:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687400 con 0x555b6028ed00 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139000 con 0x555b60670480 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687800 con 0x555b60670480 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139c00 con 0x555b6028ed00 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.278+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687c00 con 0x555b6028f600 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657800 con 0x555b60670900 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686800 con 0x555b60670000 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656800 con 0x555b6028ed00 2024-05-17T22:07:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687800 con 0x555b60670900 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b6028f180 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686400 con 0x555b60670480 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 111 out=49b] v111'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.282+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657000 con 0x555b6028f600 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 111 out=49b] v109'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686c00 con 0x555b6028f600 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656c00 con 0x555b60670480 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657400 con 0x555b6028ed00 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139000 con 0x555b60670000 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5cded640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x555b60670d80 0x555b60679080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686000 con 0x555b60670d80 2024-05-17T22:07:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x555b60670d80 0x555b60679080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x555b60670d80 0x555b60679080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670d80 2024-05-17T22:07:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687000 con 0x555b6028ed00 2024-05-17T22:07:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 13 ==== osd_op_reply(47 [pgnls-filter start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.286+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656800 con 0x555b6028f180 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687400 con 0x555b6028ed00 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 14 ==== osd_op_reply(49 [pgnls-filter start_epoch 111 out=81b] v111'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 10 datascan.operator(): handling object 100000001f7.0 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c400 con 0x555b6028ed00 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 15 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-17T22:07:58.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c800 con 0x555b60670480 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x555b60671200 0x555b60ff5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b6068cc00 con 0x555b60671200 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x555b60671200 0x555b60ff5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x555b60671200 0x555b60ff5700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv14 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x555b60366500 con 0x555b60671200 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-17T22:07:58.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.get_frag_of: frag is * 2024-05-17T22:07:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d000 con 0x555b60670480 2024-05-17T22:07:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 10 datascan.find_or_create_dirfrag: read_version = 7 2024-05-17T22:07:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-17T22:07:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d400 con 0x555b60670480 2024-05-17T22:07:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-17T22:07:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.read_dentry: bravo_head not found in result 2024-05-17T22:07:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-17T22:07:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d800 con 0x555b60670480 2024-05-17T22:07:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v111'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-17T22:07:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.290+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068dc00 con 0x555b60671200 2024-05-17T22:07:58.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60671200 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686800 con 0x555b60670d80 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670d80 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657400 con 0x555b60670000 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687c00 con 0x555b60670900 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657800 con 0x555b6028ed00 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 16 ==== osd_op_reply(60 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687800 con 0x555b6028ed00 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 17 ==== osd_op_reply(61 [pgnls-filter start_epoch 111 out=49b] v109'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657000 con 0x555b60670480 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686c00 con 0x555b60670000 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 111 out=49b] v109'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139c00 con 0x555b60670480 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 111 out=49b] v109'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.294+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657400 con 0x555b60670900 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139400 con 0x555b6028f180 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 111 out=49b] v111'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b6028f180 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c400 con 0x555b60670d80 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 111 out=49b] v111'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670d80 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686000 con 0x555b60670000 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c000 con 0x555b6028f180 2024-05-17T22:07:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.298+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.5 v2:172.21.15.155:6808/758721566 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b6028f180 2024-05-17T22:07:58.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687000 con 0x555b60670900 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c800 con 0x555b6028f600 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687400 con 0x555b6028f600 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068cc00 con 0x555b6028f600 2024-05-17T22:07:58.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686800 con 0x555b6028f600 2024-05-17T22:07:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.302+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d000 con 0x555b60670900 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60686400 con 0x555b60670900 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 111 out=49b] v109'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656c00 con 0x555b60670480 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.3 v2:172.21.15.83:6824/2257217219 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 111 out=49b] v109'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60670480 2024-05-17T22:07:58.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139000 con 0x555b6028f600 2024-05-17T22:07:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.7 v2:172.21.15.155:6816/3020759009 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 111 out=49b] v109'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b6028f600 2024-05-17T22:07:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60656800 con 0x555b6028ed00 2024-05-17T22:07:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 18 ==== osd_op_reply(80 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657400 con 0x555b60670900 2024-05-17T22:07:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068c800 con 0x555b60670900 2024-05-17T22:07:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.306+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657000 con 0x555b60671200 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d625f8640 1 -- 172.21.15.83:0/4017824766 <== osd.2 v2:172.21.15.83:6808/3743214759 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 111 out=49b] v109'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366500 con 0x555b60671200 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068cc00 con 0x555b6028ed00 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 19 ==== osd_op_reply(84 [pgnls-filter start_epoch 111 out=49b] v109'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60657c00 con 0x555b60670900 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 111 out=49b] v109'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d400 con 0x555b60670d80 2024-05-17T22:07:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670d80 2024-05-17T22:07:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139c00 con 0x555b6028ed00 2024-05-17T22:07:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 20 ==== osd_op_reply(87 [pgnls-filter start_epoch 111 out=49b] v109'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068d800 con 0x555b60670000 2024-05-17T22:07:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.1 v2:172.21.15.83:6800/3404427378 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 111 out=49b] v109'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670000 2024-05-17T22:07:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b61139400 con 0x555b60670900 2024-05-17T22:07:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 <== osd.4 v2:172.21.15.155:6824/505763805 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 111 out=49b] v109'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366780 con 0x555b60670900 2024-05-17T22:07:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5cded640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b6068dc00 con 0x555b60670d80 2024-05-17T22:07:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 111 out=49b] v109'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b60670d80 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.310+0000 7f5d5d5ee640 1 -- 172.21.15.83:0/4017824766 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 111 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x555b60687c00 con 0x555b6028ed00 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d615f6640 1 -- 172.21.15.83:0/4017824766 <== osd.0 v2:172.21.15.83:6816/622135753 21 ==== osd_op_reply(91 [pgnls-filter start_epoch 111 out=49b] v109'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x555b60366000 con 0x555b6028ed00 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x555b6028ed00 msgr2=0x555b60ff4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x555b6028ed00 0x555b60ff4680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x555b60670000 msgr2=0x555b60678000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x555b60670000 0x555b60678000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x555b60671200 msgr2=0x555b60ff5700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x555b60671200 0x555b60ff5700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x555b60670480 msgr2=0x555b60678580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x555b60670480 0x555b60678580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x555b60670900 msgr2=0x555b60678b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x555b60670900 0x555b60678b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 reap_dead start 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x555b6028f180 msgr2=0x555b60ff4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x555b6028f180 0x555b60ff4c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 reap_dead start 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x555b60670d80 msgr2=0x555b60679080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x555b60670d80 0x555b60679080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x555b6028f600 msgr2=0x555b60ff5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x555b6028f600 0x555b60ff5180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x555b6028fa80 msgr2=0x555b60ff4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x555b6028fa80 0x555b60ff4100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x555b60331020 tx=0x555b61153320 comp rx=0 tx=0).stop 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 msgr2=0x555b60ff3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/4017824766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028c900 0x555b60ff3600 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x555b610792f0 tx=0x555b6032fec0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d61df7640 1 -- 172.21.15.83:0/4017824766 reap_dead start 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 shutdown_connections 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 >> 172.21.15.83:0/4017824766 conn(0x555b6028d200 msgr2=0x555b6053a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 shutdown_connections 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/4017824766 wait complete. 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 msgr2=0x555b60ff2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x555b61079c50 tx=0x555b6032e720 comp rx=0 tx=0).stop 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 shutdown_connections 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b6028df80 0x555b60ff2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555b6028db00 0x555b60ff2000 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 --2- 172.21.15.83:0/644095948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x555b6028d680 0x555b60ff2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 >> 172.21.15.83:0/644095948 conn(0x555b6028e400 msgr2=0x555b5c217800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 shutdown_connections 2024-05-17T22:07:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.314+0000 7f5d636388c0 1 -- 172.21.15.83:0/644095948 wait complete. 2024-05-17T22:07:58.310 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.127387, result: 2024-05-17T22:07:58.310 DEBUG:teuthology.orchestra.run.smithi083:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-17T22:07:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- start start 2024-05-17T22:07:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 Processor -- start 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7573680 0x5627b82c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7573f80 0x5627b82c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b8404180 con 0x5627b7573b00 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b8404300 con 0x5627b7573680 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b8404480 con 0x5627b7573f80 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42494/0 (socket says 172.21.15.83:42494) 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 -- 172.21.15.83:0/157007790 learned_addr learned my addr 172.21.15.83:0/157007790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7573f80 0x5627b82c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 -- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7573f80 msgr2=0x5627b82c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7573680 0x5627b82c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7573f80 0x5627b82c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 -- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7573680 msgr2=0x5627b82c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7573680 0x5627b82c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 -- 172.21.15.83:0/157007790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b34f41e0 con 0x5627b7573b00 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x5627b834fc80 tx=0x5627b7612480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42114199edc358a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fa0e7640 1 -- 172.21.15.83:0/157007790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b75344e0 con 0x5627b7573b00 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fa0e7640 1 -- 172.21.15.83:0/157007790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b75481c0 con 0x5627b7573b00 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fa0e7640 1 -- 172.21.15.83:0/157007790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b7534b60 con 0x5627b7573b00 2024-05-17T22:07:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b34f52c0 con 0x5627b7573b00 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5627b34f5a40 con 0x5627b7573b00 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 4 waiting for MDS map... 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fa0e7640 1 -- 172.21.15.83:0/157007790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x5627b841e600 con 0x5627b7573b00 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fa0e7640 1 -- 172.21.15.83:0/157007790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 103) ==== 2885+0+0 (secure 0 0 0) 0x5627b7555180 con 0x5627b7573b00 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 4 Got MDS map 103 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 4 datascan.main: Using metadata pool output 2024-05-17T22:07:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.374+0000 7f54fe12d8c0 4 datascan.main: connecting to RADOS... 2024-05-17T22:07:58.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 Processor -- start 2024-05-17T22:07:58.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- start start 2024-05-17T22:07:58.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b8404600 con 0x5627b7572900 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b8404780 con 0x5627b7572d80 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b8404900 con 0x5627b7574880 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42496/0 (socket says 172.21.15.83:42496) 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 -- 172.21.15.83:0/4283102015 learned_addr learned my addr 172.21.15.83:0/4283102015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 -- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7574880 msgr2=0x5627b82c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 -- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7572d80 msgr2=0x5627b82c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 -- 172.21.15.83:0/4283102015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b34f5c20 con 0x5627b7572900 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x5627b834f2c0 tx=0x5627b7613f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05125a63dbc3a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54f80e3640 1 -- 172.21.15.83:0/4283102015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b75349c0 con 0x5627b7572900 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54f80e3640 1 -- 172.21.15.83:0/4283102015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b7548e00 con 0x5627b7572900 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54f80e3640 1 -- 172.21.15.83:0/4283102015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b75344e0 con 0x5627b7572900 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/4283102015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 msgr2=0x5627b82c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x5627b834f2c0 tx=0x5627b7613f20 comp rx=0 tx=0).stop 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/4283102015 shutdown_connections 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/4283102015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/4283102015 >> 172.21.15.83:0/4283102015 conn(0x5627b7573200 msgr2=0x5627b7818000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/4283102015 shutdown_connections 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/4283102015 wait complete. 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 Processor -- start 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.378+0000 7f54fe12d8c0 1 -- start start 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627b8405b00 con 0x5627b7572d80 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5627b8405980 con 0x5627b7574880 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x5627b8405800 con 0x5627b7572900 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42506/0 (socket says 172.21.15.83:42506) 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 learned_addr learned my addr 172.21.15.83:0/3707872098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7572900 msgr2=0x5627b82c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7572900 0x5627b82c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7574880 msgr2=0x5627b82c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7574880 0x5627b82c3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627b34f54a0 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x5627b8433230 tx=0x5627b7612960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 -- 172.21.15.83:0/3707872098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b7534b60 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 -- 172.21.15.83:0/3707872098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627b7548c40 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627b34f43c0 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 -- 172.21.15.83:0/3707872098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5627b7534000 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627b34f5c20 con 0x5627b7572d80 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 4 datascan.init: resolving metadata pool 16 2024-05-17T22:07:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 -- 172.21.15.83:0/3707872098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x5627b7834c00 con 0x5627b7572d80 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 0x5627b82c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54f68e0640 1 -- 172.21.15.83:0/3707872098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x5627b783c600 con 0x5627b7572d80 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 0x5627b82c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 4 datascan.main: resolving metadata pool 16 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627b7574d00 0x5627b82c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627b7574d00 0x5627b82c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.382+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 0x5627b82c4100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5627b834fbf0 tx=0x5627b84350e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627b7574d00 0x5627b82c4680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== osd_op_reply(1 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7942000 0x5627b7946000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7942000 0x5627b7946000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7942000 0x5627b7946000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== osd_op_reply(2 [pgnls start_epoch 111 out=177b] v110'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== osd_op_reply(3 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 3 ==== osd_op_reply(4 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627b7942480 0x5627b7946580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942480 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627b7942480 0x5627b7946580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627b7942480 0x5627b7946580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== osd_op_reply(5 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 4 ==== osd_op_reply(6 [pgnls start_epoch 111 out=149b] v111'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627b7942900 0x5627b7946b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942900 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627b7942900 0x5627b7946b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.386+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627b7942900 0x5627b7946b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== osd_op_reply(7 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== osd_op_reply(8 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5627b7942d80 0x5627b7947080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942d80 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5627b7942d80 0x5627b7947080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5627b7942d80 0x5627b7947080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== osd_op_reply(9 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942d80 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627b7943200 0x5627b7947600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943200 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627b7943200 0x5627b7947600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.390+0000 7f54fc8ec640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627b7943200 0x5627b7947600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== osd_op_reply(10 [pgnls start_epoch 111 out=81b] v110'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 3 ==== osd_op_reply(11 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942480 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== osd_op_reply(12 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943200 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== osd_op_reply(13 [pgnls start_epoch 111 out=77b] v110'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942480 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 3 ==== osd_op_reply(14 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 4 ==== osd_op_reply(15 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627b7943680 0x5627b7947b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943680 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627b7943680 0x5627b7947b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627b7943680 0x5627b7947b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== osd_op_reply(16 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7943680 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.394+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 5 ==== osd_op_reply(17 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942900 2024-05-17T22:07:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== osd_op_reply(18 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627b7943b00 0x5627b7948100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943b00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627b7943b00 0x5627b7948100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627b7943b00 0x5627b7948100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== osd_op_reply(19 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942d80 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== osd_op_reply(20 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942d80 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 5 ==== osd_op_reply(21 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 6 ==== osd_op_reply(22 [pgnls start_epoch 111 out=97b] v110'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943200 2024-05-17T22:07:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 3 ==== osd_op_reply(23 [pgnls start_epoch 111 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 6 ==== osd_op_reply(24 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.398+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942900 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 3 ==== osd_op_reply(25 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943b00 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== osd_op_reply(26 [pgnls start_epoch 111 out=98b] v110'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7943b00 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 3 ==== osd_op_reply(27 [pgnls start_epoch 111 out=76b] v111'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942900 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 4 ==== osd_op_reply(28 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942480 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 4 ==== osd_op_reply(29 [pgnls start_epoch 111 out=105b] v111'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942000 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 7 ==== osd_op_reply(30 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 7 ==== osd_op_reply(31 [pgnls start_epoch 111 out=73b] v111'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7942900 2024-05-17T22:07:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 5 ==== osd_op_reply(32 [pgnls start_epoch 111 out=101b] v111'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-17T22:07:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.402+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940c00 con 0x5627b7942000 2024-05-17T22:07:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941000 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941400 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv14 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940c00 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-17T22:07:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941000 con 0x5627b7574d00 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941400 con 0x5627b7942000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941800 con 0x5627b7943200 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941c00 con 0x5627b7942000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978000 con 0x5627b7574d00 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-17T22:07:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978400 con 0x5627b7942000 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.406+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978800 con 0x5627b7943b00 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978c00 con 0x5627b7942480 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7979000 con 0x5627b7942480 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 20 datascan.scan_links: injected ino detected 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7979400 con 0x5627b7942900 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 10 ==== osd_op_reply(48 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 16 ==== osd_op_reply(49 [pgnls start_epoch 111 out=177b] v110'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 11 ==== osd_op_reply(50 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 12 ==== osd_op_reply(51 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942480 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 7 ==== osd_op_reply(52 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 13 ==== osd_op_reply(53 [pgnls start_epoch 111 out=149b] v111'43 uv43 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.410+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942900 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 7 ==== osd_op_reply(54 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 17 ==== osd_op_reply(55 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942d80 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.4 v2:172.21.15.155:6824/505763805 3 ==== osd_op_reply(56 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942d80 2024-05-17T22:07:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943200 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 5 ==== osd_op_reply(57 [pgnls start_epoch 111 out=81b] v110'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 18 ==== osd_op_reply(58 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942480 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 8 ==== osd_op_reply(59 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943200 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 6 ==== osd_op_reply(60 [pgnls start_epoch 111 out=77b] v110'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942480 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 9 ==== osd_op_reply(61 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 19 ==== osd_op_reply(62 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.414+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943680 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== osd_op_reply(63 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7943680 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 20 ==== osd_op_reply(64 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942900 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 8 ==== osd_op_reply(65 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943b00 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 5 ==== osd_op_reply(66 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942d80 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.4 v2:172.21.15.155:6824/505763805 4 ==== osd_op_reply(67 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942d80 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 14 ==== osd_op_reply(68 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 15 ==== osd_op_reply(69 [pgnls start_epoch 111 out=97b] v110'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.418+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943200 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 7 ==== osd_op_reply(70 [pgnls start_epoch 111 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 21 ==== osd_op_reply(71 [pgnls start_epoch 111 out=73b] v110'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942900 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 9 ==== osd_op_reply(72 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943b00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 6 ==== osd_op_reply(73 [pgnls start_epoch 111 out=98b] v110'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7943b00 2024-05-17T22:07:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 7 ==== osd_op_reply(74 [pgnls start_epoch 111 out=76b] v111'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942900 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 10 ==== osd_op_reply(75 [pgnls start_epoch 111 out=49b] v111'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942480 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 10 ==== osd_op_reply(76 [pgnls start_epoch 111 out=105b] v111'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942000 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 22 ==== osd_op_reply(77 [pgnls start_epoch 111 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7574d00 2024-05-17T22:07:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 16 ==== osd_op_reply(78 [pgnls start_epoch 111 out=73b] v111'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.422+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 111 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940800 con 0x5627b7942900 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 11 ==== osd_op_reply(79 [pgnls start_epoch 111 out=101b] v111'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940c00 con 0x5627b7942000 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-17T22:07:58.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941400 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941c00 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv14 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b841d400 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b841d000 con 0x5627b7942000 2024-05-17T22:07:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-17T22:07:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b841dc00 con 0x5627b7574d00 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978800 con 0x5627b7942000 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-17T22:07:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978400 con 0x5627b7943200 2024-05-17T22:07:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-17T22:07:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-17T22:07:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7940c00 con 0x5627b7942000 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.426+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941400 con 0x5627b7574d00 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.0 v2:172.21.15.83:6816/622135753 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602000 con 0x5627b7574d00 2024-05-17T22:07:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-17T22:07:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-17T22:07:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-17T22:07:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-17T22:07:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-17T22:07:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941c00 con 0x5627b7942000 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.2 v2:172.21.15.83:6808/3743214759 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602500 con 0x5627b7942000 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7941000 con 0x5627b7943b00 2024-05-17T22:07:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b841d400 con 0x5627b7942480 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-17T22:07:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b841d000 con 0x5627b7942480 2024-05-17T22:07:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 20 datascan.scan_links: injected ino detected 2024-05-17T22:07:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-17T22:07:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978c00 con 0x5627b7942900 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fc0eb640 1 -- 172.21.15.83:0/3707872098 <== osd.5 v2:172.21.15.155:6808/758721566 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5627b7602000 con 0x5627b7942900 2024-05-17T22:07:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: processing 1 injected_inos 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7979000 con 0x5627b7942480 2024-05-17T22:07:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 20 datascan.scan_links: first is now 2 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.430+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978400 con 0x5627b7942480 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.3 v2:172.21.15.83:6824/2257217219 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v111'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627b7602780 con 0x5627b7942480 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 10 datascan.scan_links: updating inotable 2024-05-17T22:07:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7978000 con 0x5627b7943b00 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x5627b7979800 con 0x5627b7943b00 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fd0ed640 1 -- 172.21.15.83:0/3707872098 <== osd.1 v2:172.21.15.83:6800/3404427378 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v111'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5627b7602500 con 0x5627b7943b00 2024-05-17T22:07:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 10 datascan.scan_links: updating snaptable 2024-05-17T22:07:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5627b7979c00 con 0x5627b7943200 2024-05-17T22:07:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 <== osd.6 v2:172.21.15.155:6800/1025803281 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5627b7602780 con 0x5627b7943200 2024-05-17T22:07:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627b7574d00 msgr2=0x5627b82c4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x5627b7574d00 0x5627b82c4680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627b7943b00 msgr2=0x5627b7948100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x5627b7943b00 0x5627b7948100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7942000 msgr2=0x5627b7946000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x5627b7942000 0x5627b7946000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.434+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627b7942480 msgr2=0x5627b7946580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x5627b7942480 0x5627b7946580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5627b7942d80 msgr2=0x5627b7947080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x5627b7942d80 0x5627b7947080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 reap_dead start 2024-05-17T22:07:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627b7942900 msgr2=0x5627b7946b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x5627b7942900 0x5627b7946b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 reap_dead start 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627b7943200 msgr2=0x5627b7947600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x5627b7943200 0x5627b7947600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627b7943680 msgr2=0x5627b7947b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x5627b7943680 0x5627b7947b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fc8ec640 1 -- 172.21.15.83:0/3707872098 reap_dead start 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 msgr2=0x5627b82c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 0x5627b82c4100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x5627b834fbf0 tx=0x5627b84350e0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 msgr2=0x5627b82c3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x5627b8433230 tx=0x5627b7612960 comp rx=0 tx=0).stop 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 shutdown_connections 2024-05-17T22:07:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x5627b7575a80 0x5627b82c4100 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/3707872098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7572d80 0x5627b82c3080 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 >> 172.21.15.83:0/3707872098 conn(0x5627b7573200 msgr2=0x5627b7818000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 shutdown_connections 2024-05-17T22:07:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/3707872098 wait complete. 2024-05-17T22:07:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 msgr2=0x5627b82c2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x5627b834fc80 tx=0x5627b7612480 comp rx=0 tx=0).stop 2024-05-17T22:07:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 shutdown_connections 2024-05-17T22:07:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5627b7573f80 0x5627b82c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627b7573b00 0x5627b82c2000 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 --2- 172.21.15.83:0/157007790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5627b7573680 0x5627b82c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 >> 172.21.15.83:0/157007790 conn(0x5627b7574400 msgr2=0x5627b34fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 shutdown_connections 2024-05-17T22:07:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.438+0000 7f54fe12d8c0 1 -- 172.21.15.83:0/157007790 wait complete. 2024-05-17T22:07:58.449 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.139165, result: 2024-05-17T22:07:58.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:58.453 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-05-17T22:07:58.474 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.475 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:58.476 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-05-17T22:07:58.477 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-05-17T22:07:58.478 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:58.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:58.480 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:58.481 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.482 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-05-17T22:07:58.483 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-05-17T22:07:58.484 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = False 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.485 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.487 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:58.488 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.489 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.491 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-05-17T22:07:58.492 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-05-17T22:07:58.493 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-05-17T22:07:58.494 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:58.495 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:58.496 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-05-17T22:07:58.497 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-05-17T22:07:58.498 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = False 2024-05-17T22:07:58.499 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-05-17T22:07:58.499 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-05-17T22:07:58.499 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:58.499 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.499 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:58.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-17T22:07:58.553 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-05-17T22:07:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-05-17T22:07:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.554 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:58.555 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-05-17T22:07:58.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.557 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-05-17T22:07:58.558 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2024-05-17T22:07:58.559 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:58.560 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-05-17T22:07:58.561 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:58.562 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:58.563 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-05-17T22:07:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-05-17T22:07:58.565 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-05-17T22:07:58.566 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-05-17T22:07:58.566 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.566 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2024-05-17T22:07:58.571 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2024-05-17T22:07:58.572 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.573 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-17T22:07:58.574 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.575 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.83,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2024-05-17T22:07:58.576 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug trim objects = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.577 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bdev async discard = True 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bdev enable discard = True 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore allocator = bitmap 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 1/20 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 1/20 2024-05-17T22:07:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 4/10 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 0.95 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = bluestore 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:58.579 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2024-05-17T22:07:58.580 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.581 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: mon down mkfs grace = 300 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2024-05-17T22:07:58.582 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2024-05-17T22:07:58.583 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2024-05-17T22:07:58.584 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2024-05-17T22:07:58.584 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2024-05-17T22:07:58.584 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2024-05-17T22:07:58.584 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2024-05-17T22:07:58.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-17T22:07:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 Processor -- start 2024-05-17T22:07:58.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 -- start start 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00b4810 0x7f74b00d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f74b00d0290 0x7f74b015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74b00d4170 con 0x7f74b00d2000 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f74b00d4450 con 0x7f74b00d0290 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b86bb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74b00d4730 con 0x7f74b00b4810 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b5c56640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f74b00d0290 0x7f74b015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b6c58640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f74b00d2000 0x7f74b00d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37528/0 (socket says 172.21.15.83:37528) 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b6c58640 1 -- 172.21.15.83:0/795217748 learned_addr learned my addr 172.21.15.83:0/795217748 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:07:58.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b6457640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00b4810 0x7f74b00d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b5c56640 1 -- 172.21.15.83:0/795217748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00b4810 msgr2=0x7f74b00d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:07:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b5c56640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00b4810 0x7f74b00d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b5c56640 1 -- 172.21.15.83:0/795217748 >> v1:172.21.15.83:6789/0 conn(0x7f74b00d2000 legacy=0x7f74b00d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.770+0000 7f74b5c56640 1 -- 172.21.15.83:0/795217748 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b00d4a10 con 0x7f74b00d0290 2024-05-17T22:07:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b5c56640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b015fb10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f74a00097c0 tx=0x7f74a001cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983ea499105846ef server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b5455640 1 -- 172.21.15.83:0/795217748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00aa620 con 0x7f74b00d0290 2024-05-17T22:07:58.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b5455640 1 -- 172.21.15.83:0/795217748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a00af070 con 0x7f74b00d0290 2024-05-17T22:07:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b5455640 1 -- 172.21.15.83:0/795217748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00aa9e0 con 0x7f74b00d0290 2024-05-17T22:07:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 -- 172.21.15.83:0/795217748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 msgr2=0x7f74b015fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b015fb10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f74a00097c0 tx=0x7f74a001cf00 comp rx=0 tx=0).stop 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 -- 172.21.15.83:0/795217748 shutdown_connections 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b015fb10 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 --2- 172.21.15.83:0/795217748 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00b4810 0x7f74b00d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 -- 172.21.15.83:0/795217748 >> 172.21.15.83:0/795217748 conn(0x7f74b006e010 msgr2=0x7f74b006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 -- 172.21.15.83:0/795217748 shutdown_connections 2024-05-17T22:07:58.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.774+0000 7f74b86bb640 1 -- 172.21.15.83:0/795217748 wait complete. 2024-05-17T22:07:58.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 Processor -- start 2024-05-17T22:07:58.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- start start 2024-05-17T22:07:58.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b00b4810 0x7f74b0153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00d2000 0x7f74b01f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74b00b9670 con 0x7f74b00d0290 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f74b00b94f0 con 0x7f74b00b4810 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f74b00b97f0 con 0x7f74b00d2000 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42514/0 (socket says 172.21.15.83:42514) 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 -- 172.21.15.83:0/2012500175 learned_addr learned my addr 172.21.15.83:0/2012500175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b6c58640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00d2000 0x7f74b01f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 -- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00d2000 msgr2=0x7f74b01f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00d2000 0x7f74b01f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b6457640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b00b4810 0x7f74b0153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 -- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b00b4810 msgr2=0x7f74b0153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b00b4810 0x7f74b0153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 -- 172.21.15.83:0/2012500175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b0153da0 con 0x7f74b00d0290 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b5c56640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f74a001a0d0 tx=0x7f74a001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00aadf0 con 0x7f74b00d0290 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a00be410 con 0x7f74b00d0290 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00be800 con 0x7f74b00d0290 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b0154220 con 0x7f74b00d0290 2024-05-17T22:07:58.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.778+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b01547e0 con 0x7f74b00d0290 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b0153f70 con 0x7f74b00d0290 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f74a00bc020 con 0x7f74b00d0290 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74a6ffd640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 0x7f7494071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f74a01297f0 con 0x7f74b00d0290 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74b6457640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 0x7f7494071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:07:58.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.782+0000 7f74b6457640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 0x7f7494071940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b7c0 tx=0x7f74ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:07:58.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.786+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f74a0016090 con 0x7f74b00d0290 2024-05-17T22:07:58.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:58.982+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f74b00d5210 con 0x7f74b00d0290 2024-05-17T22:07:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74a6ffd640 1 -- 172.21.15.83:0/2012500175 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v105) ==== 162+0+0 (secure 0 0 0) 0x7f74a00f67a0 con 0x7f74b00d0290 2024-05-17T22:07:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 msgr2=0x7f7494071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 0x7f7494071940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b7c0 tx=0x7f74ac008040 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 msgr2=0x7f74b01fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f74a001a0d0 tx=0x7f74a001ace0 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 shutdown_connections 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b00d2000 0x7f74b01f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74b00d0290 0x7f74b01fc230 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f749406f430 0x7f7494071940 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 --2- 172.21.15.83:0/2012500175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b00b4810 0x7f74b0153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 >> 172.21.15.83:0/2012500175 conn(0x7f74b006e010 msgr2=0x7f74b0162560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 shutdown_connections 2024-05-17T22:07:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:07:59.898+0000 7f74b86bb640 1 -- 172.21.15.83:0/2012500175 wait complete. 2024-05-17T22:07:59.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:00.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 Processor -- start 2024-05-17T22:08:00.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 -- start start 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d34165850 0x7f0d34167c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d340b0ec0 con 0x7f0d3415a540 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d340b11a0 con 0x7f0d341604d0 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d3acca640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3415a950 con 0x7f0d34165850 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d39267640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d34165850 0x7f0d34167c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d38a66640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0d3415a540 0x7f0d340b4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37536/0 (socket says 172.21.15.83:37536) 2024-05-17T22:08:00.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d38a66640 1 -- 172.21.15.83:0/1423431899 learned_addr learned my addr 172.21.15.83:0/1423431899 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:00.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d39267640 1 -- 172.21.15.83:0/1423431899 >> v1:172.21.15.155:6789/0 conn(0x7f0d341604d0 legacy=0x7f0d341608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:00.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d39267640 1 -- 172.21.15.83:0/1423431899 >> v1:172.21.15.83:6789/0 conn(0x7f0d3415a540 legacy=0x7f0d340b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d337fe640 1 -- 172.21.15.83:0/1423431899 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2254888420 0 0) 0x7f0d340b0ec0 con 0x7f0d3415a540 2024-05-17T22:08:00.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.070+0000 7f0d39267640 1 -- 172.21.15.83:0/1423431899 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d340b5610 con 0x7f0d34165850 2024-05-17T22:08:00.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d39267640 1 --2- 172.21.15.83:0/1423431899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d34167c70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009d90 tx=0x7f0d2c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6077e3663c0698ae server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:00.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d337fe640 1 -- 172.21.15.83:0/1423431899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0b2070 con 0x7f0d34165850 2024-05-17T22:08:00.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d337fe640 1 -- 172.21.15.83:0/1423431899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c0a57a0 con 0x7f0d34165850 2024-05-17T22:08:00.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d337fe640 1 -- 172.21.15.83:0/1423431899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0a5b70 con 0x7f0d34165850 2024-05-17T22:08:00.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 -- 172.21.15.83:0/1423431899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 msgr2=0x7f0d34167c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 --2- 172.21.15.83:0/1423431899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d34167c70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009d90 tx=0x7f0d2c01c8e0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 -- 172.21.15.83:0/1423431899 shutdown_connections 2024-05-17T22:08:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 --2- 172.21.15.83:0/1423431899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d34167c70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 -- 172.21.15.83:0/1423431899 >> 172.21.15.83:0/1423431899 conn(0x7f0d3409b8b0 msgr2=0x7f0d340b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 -- 172.21.15.83:0/1423431899 shutdown_connections 2024-05-17T22:08:00.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.074+0000 7f0d3acca640 1 -- 172.21.15.83:0/1423431899 wait complete. 2024-05-17T22:08:00.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 Processor -- start 2024-05-17T22:08:00.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- start start 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3415a540 0x7f0d34167150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d341604d0 0x7f0d341486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d340b7030 con 0x7f0d34165850 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d340b0c30 con 0x7f0d341604d0 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3416a9d0 con 0x7f0d3415a540 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42538/0 (socket says 172.21.15.83:42538) 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 -- 172.21.15.83:0/2088450693 learned_addr learned my addr 172.21.15.83:0/2088450693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:00.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d33fff640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d341604d0 0x7f0d341486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d38a66640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3415a540 0x7f0d34167150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 -- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3415a540 msgr2=0x7f0d34167150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3415a540 0x7f0d34167150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 -- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d341604d0 msgr2=0x7f0d341486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d341604d0 0x7f0d341486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 -- 172.21.15.83:0/2088450693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3414fa40 con 0x7f0d34165850 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d39267640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0b0040 tx=0x7f0d2c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0b2040 con 0x7f0d34165850 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c0bf070 con 0x7f0d34165850 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3414fd20 con 0x7f0d34165850 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0b88b0 con 0x7f0d34165850 2024-05-17T22:08:00.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d34166950 con 0x7f0d34165850 2024-05-17T22:08:00.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.078+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d341608e0 con 0x7f0d34165850 2024-05-17T22:08:00.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.082+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0d2c0b8a50 con 0x7f0d34165850 2024-05-17T22:08:00.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.082+0000 7f0d317fa640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 0x7f0d20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:00.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.082+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f0d2c12a4d0 con 0x7f0d34165850 2024-05-17T22:08:00.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.082+0000 7f0d38a66640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 0x7f0d20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:00.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.082+0000 7f0d38a66640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 0x7f0d20071910 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:00.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.086+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0d2c016560 con 0x7f0d34165850 2024-05-17T22:08:00.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.278+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d34076820 con 0x7f0d34165850 2024-05-17T22:08:00.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.278+0000 7f0d317fa640 1 -- 172.21.15.83:0/2088450693 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5184 (secure 0 0 0) 0x7f0d2c0f7310 con 0x7f0d34165850 2024-05-17T22:08:00.267 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:00.267 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:07:59.903181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:replay","state_seq":1,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-17T22:08:00.268 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 106 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 msgr2=0x7f0d20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 0x7f0d20071910 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007fd0 tx=0x7f0d24009040 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 msgr2=0x7f0d3414ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0b0040 tx=0x7f0d2c019680 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 shutdown_connections 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d34165850 0x7f0d3414ef60 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d2006f400 0x7f0d20071910 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d341604d0 0x7f0d341486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 --2- 172.21.15.83:0/2088450693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d3415a540 0x7f0d34167150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 >> 172.21.15.83:0/2088450693 conn(0x7f0d3409b8b0 msgr2=0x7f0d3414de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 shutdown_connections 2024-05-17T22:08:00.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:00.282+0000 7f0d3acca640 1 -- 172.21.15.83:0/2088450693 wait complete. 2024-05-17T22:08:00.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:07:59.903181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8358}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8358': {'gid': 8358, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/1541446976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 1541446976}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1541446976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:08:00.694 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8358:up:replay 2024-05-17T22:08:01.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 Processor -- start 2024-05-17T22:08:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 -- start start 2024-05-17T22:08:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd700d1040 0x7fbd70144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7015ea40 con 0x7fbd7015e080 2024-05-17T22:08:01.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7015ed20 con 0x7fbd70073540 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7627f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7015f000 con 0x7fbd700d1040 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd7015e080 0x7fbd70072e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:37540/0 (socket says 172.21.15.83:37540) 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6f7fe640 1 -- 172.21.15.83:0/2032297802 learned_addr learned my addr 172.21.15.83:0/2032297802 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/2032297802 >> v2:172.21.15.155:3300/0 conn(0x7fbd700d1040 0x7fbd70144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798841089 0 0) 0x7fbd7015ea40 con 0x7fbd7015e080 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd58003530 con 0x7fbd7015e080 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 727139822 0 0) 0x7fbd58003530 con 0x7fbd7015e080 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 >> v2:172.21.15.155:3300/0 conn(0x7fbd700d1040 msgr2=0x7fbd70144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 --2- 172.21.15.83:0/2032297802 >> v2:172.21.15.155:3300/0 conn(0x7fbd700d1040 0x7fbd70144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 >> v1:172.21.15.155:6789/0 conn(0x7fbd70073540 legacy=0x7fbd7006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7006ef30 con 0x7fbd7015e080 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fbd64001700 con 0x7fbd7015e080 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 >> v1:172.21.15.83:6789/0 conn(0x7fbd7015e080 legacy=0x7fbd70072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:01.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd58003a60 0x7fbd58003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 0x7fbd58004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58005260 0x7fbd5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7015f000 con 0x7fbd58004810 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5800c1f0 con 0x7fbd58005260 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5800c3f0 con 0x7fbd58003a60 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 0x7fbd58004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd7481c640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58005260 0x7fbd5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd58003a60 0x7fbd58003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6f7fe640 1 -- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd58003a60 msgr2=0x7fbd58003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.870+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd58003a60 0x7fbd58003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6f7fe640 1 -- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58005260 msgr2=0x7fbd5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58005260 0x7fbd5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd7481c640 1 -- 172.21.15.83:0/2032297802 reap_dead start 2024-05-17T22:08:01.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6f7fe640 1 -- 172.21.15.83:0/2032297802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd58003530 con 0x7fbd58004810 2024-05-17T22:08:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 0x7fbd58004c20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fbd70073ed0 tx=0x7fbd6401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c314d4160f3466 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64007cf0 con 0x7fbd58004810 2024-05-17T22:08:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd64005ae0 con 0x7fbd58004810 2024-05-17T22:08:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.874+0000 7fbd6effd640 1 -- 172.21.15.83:0/2032297802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64007070 con 0x7fbd58004810 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 msgr2=0x7fbd58004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 0x7fbd58004c20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fbd70073ed0 tx=0x7fbd6401f2d0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- 172.21.15.83:0/2032297802 shutdown_connections 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 --2- 172.21.15.83:0/2032297802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58004810 0x7fbd58004c20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- 172.21.15.83:0/2032297802 >> 172.21.15.83:0/2032297802 conn(0x7fbd7006cd90 msgr2=0x7fbd700b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- 172.21.15.83:0/2032297802 shutdown_connections 2024-05-17T22:08:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- 172.21.15.83:0/2032297802 wait complete. 2024-05-17T22:08:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 Processor -- start 2024-05-17T22:08:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- start start 2024-05-17T22:08:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 0x7fbd701579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 0x7fbd70159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7015e610 con 0x7fbd70159640 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7015e490 con 0x7fbd58004810 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.878+0000 7fbd7627f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7015e790 con 0x7fbd700cdf50 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42564/0 (socket says 172.21.15.83:42564) 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 -- 172.21.15.83:0/4172208348 learned_addr learned my addr 172.21.15.83:0/4172208348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 0x7fbd70159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 0x7fbd701579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 -- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 msgr2=0x7fbd70159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 0x7fbd70159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 -- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 msgr2=0x7fbd701579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 0x7fbd701579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 -- 172.21.15.83:0/4172208348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70158060 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6f7fe640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 0x7fbd70159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 0x7fbd701579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7481c640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ce920 tx=0x7fbd6800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68006b60 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68006d20 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003980 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd70158340 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd70158900 con 0x7fbd70159640 2024-05-17T22:08:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd70144080 con 0x7fbd70159640 2024-05-17T22:08:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd6801a020 con 0x7fbd70159640 2024-05-17T22:08:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 0x7fbd5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 0x7fbd5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7fbd6808e9c0 con 0x7fbd70159640 2024-05-17T22:08:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.882+0000 7fbd6ffff640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 0x7fbd5c071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009aa0 tx=0x7fbd60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:01.886+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fbd68090e20 con 0x7fbd70159640 2024-05-17T22:08:01.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:01.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:02.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd70070700 con 0x7fbd70159640 2024-05-17T22:08:02.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd6cff9640 1 -- 172.21.15.83:0/4172208348 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5184 (secure 0 0 0) 0x7fbd6805b800 con 0x7fbd70159640 2024-05-17T22:08:02.072 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:02.072 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:00.916746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-17T22:08:02.072 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 108 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 msgr2=0x7fbd5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 0x7fbd5c071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009aa0 tx=0x7fbd60008040 comp rx=0 tx=0).stop 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 msgr2=0x7fbd70154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fbd700ce920 tx=0x7fbd6800eb90 comp rx=0 tx=0).stop 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 shutdown_connections 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd70159640 0x7fbd70154bb0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd700cdf50 0x7fbd70159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fbd5c06f400 0x7fbd5c071910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:02.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 --2- 172.21.15.83:0/4172208348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58004810 0x7fbd701579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:02.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 >> 172.21.15.83:0/4172208348 conn(0x7fbd7006cd90 msgr2=0x7fbd700b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:02.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 shutdown_connections 2024-05-17T22:08:02.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:02.082+0000 7fbd7627f640 1 -- 172.21.15.83:0/4172208348 wait complete. 2024-05-17T22:08:02.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:08:00.916746+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8358}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8358': {'gid': 8358, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.83:6837/1541446976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 1541446976}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1541446976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-17T22:08:02.495 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8358:up:rejoin 2024-05-17T22:08:03.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 Processor -- start 2024-05-17T22:08:03.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 -- start start 2024-05-17T22:08:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 0x7fde381601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde381606f0 0x7fde380b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde38160f50 con 0x7fde3806f3c0 2024-05-17T22:08:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fde3806e690 con 0x7fde381606f0 2024-05-17T22:08:03.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3e893640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde3806e8e0 con 0x7fde380717e0 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde3ce30640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde381606f0 0x7fde380b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde37fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fde3806f3c0 0x7fde380710d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:51774/0 (socket says 172.21.15.83:51774) 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde377fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 0x7fde381601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde37fff640 1 -- 172.21.15.83:0/3126395842 learned_addr learned my addr 172.21.15.83:0/3126395842 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde377fe640 1 -- 172.21.15.83:0/3126395842 >> v2:172.21.15.155:3300/0 conn(0x7fde381606f0 msgr2=0x7fde380b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde377fe640 1 --2- 172.21.15.83:0/3126395842 >> v2:172.21.15.155:3300/0 conn(0x7fde381606f0 0x7fde380b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde377fe640 1 -- 172.21.15.83:0/3126395842 >> v1:172.21.15.155:6789/0 conn(0x7fde3806f3c0 legacy=0x7fde380710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.670+0000 7fde377fe640 1 -- 172.21.15.83:0/3126395842 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3806ebc0 con 0x7fde380717e0 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde377fe640 1 --2- 172.21.15.83:0/3126395842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 0x7fde381601b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097c0 tx=0x7fde2c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec5e66b8a1dddf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde36ffd640 1 -- 172.21.15.83:0/3126395842 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0b0070 con 0x7fde380717e0 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde36ffd640 1 -- 172.21.15.83:0/3126395842 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c0ab600 con 0x7fde380717e0 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde36ffd640 1 -- 172.21.15.83:0/3126395842 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38160d10 con 0x7fde380717e0 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 -- 172.21.15.83:0/3126395842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 msgr2=0x7fde381601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 --2- 172.21.15.83:0/3126395842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 0x7fde381601b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097c0 tx=0x7fde2c01c3e0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 -- 172.21.15.83:0/3126395842 shutdown_connections 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 --2- 172.21.15.83:0/3126395842 >> v2:172.21.15.155:3300/0 conn(0x7fde381606f0 0x7fde380b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 --2- 172.21.15.83:0/3126395842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde380717e0 0x7fde381601b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 -- 172.21.15.83:0/3126395842 >> 172.21.15.83:0/3126395842 conn(0x7fde38068730 msgr2=0x7fde38068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 -- 172.21.15.83:0/3126395842 shutdown_connections 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.674+0000 7fde3e893640 1 -- 172.21.15.83:0/3126395842 wait complete. 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 Processor -- start 2024-05-17T22:08:03.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- start start 2024-05-17T22:08:03.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3806f3c0 0x7fde380cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde381560c0 0x7fde380b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde3816c930 con 0x7fde381606f0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde38157060 con 0x7fde381560c0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde3816cab0 con 0x7fde3806f3c0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42572/0 (socket says 172.21.15.83:42572) 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 -- 172.21.15.83:0/3300885942 learned_addr learned my addr 172.21.15.83:0/3300885942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde37fff640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3806f3c0 0x7fde380cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3ce30640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde381560c0 0x7fde380b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 -- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3806f3c0 msgr2=0x7fde380cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3806f3c0 0x7fde380cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 -- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde381560c0 msgr2=0x7fde380b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde381560c0 0x7fde380b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 -- 172.21.15.83:0/3300885942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380ce510 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde377fe640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fde2c01a010 tx=0x7fde2c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0b0040 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c0ba070 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381466b0 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c006e20 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.678+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde380cd4f0 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde381601b0 con 0x7fde381606f0 2024-05-17T22:08:03.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde2c0b4610 con 0x7fde381606f0 2024-05-17T22:08:03.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde34ff9640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 0x7fde24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:03.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7fde2c128d80 con 0x7fde381606f0 2024-05-17T22:08:03.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde37fff640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 0x7fde24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:03.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.682+0000 7fde37fff640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 0x7fde24071910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fde28000f70 tx=0x7fde28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:03.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.686+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fde2c0c4020 con 0x7fde381606f0 2024-05-17T22:08:03.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde380b6190 con 0x7fde381606f0 2024-05-17T22:08:03.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde34ff9640 1 -- 172.21.15.83:0/3300885942 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5191 (secure 0 0 0) 0x7fde2c016090 con 0x7fde381606f0 2024-05-17T22:08:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:03.876 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:02.924755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8358,"qdb_cluster":[8358]},"id":8}]} 2024-05-17T22:08:03.876 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 109 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 msgr2=0x7fde24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 0x7fde24071910 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fde28000f70 tx=0x7fde28009040 comp rx=0 tx=0).stop 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 msgr2=0x7fde38146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fde2c01a010 tx=0x7fde2c019180 comp rx=0 tx=0).stop 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 shutdown_connections 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde2406f400 0x7fde24071910 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde381560c0 0x7fde380b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde381606f0 0x7fde38146170 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 --2- 172.21.15.83:0/3300885942 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3806f3c0 0x7fde380cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:03.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.886+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 >> 172.21.15.83:0/3300885942 conn(0x7fde38068730 msgr2=0x7fde380d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:03.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.890+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 shutdown_connections 2024-05-17T22:08:03.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:03.890+0000 7fde3e893640 1 -- 172.21.15.83:0/3300885942 wait complete. 2024-05-17T22:08:04.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-17T22:07:12.387539+0000', 'modified': '2024-05-17T22:08:02.924755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8358}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8358': {'gid': 8358, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.83:6837/1541446976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 1541446976}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1541446976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8358, 'qdb_cluster': [8358]} 2024-05-17T22:08:04.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-17T22:08:04.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-17T22:08:04.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 Processor -- start 2024-05-17T22:08:04.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 -- start start 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f75740b3be0 0x7f75740af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b4bb0 0x7f75740cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75740ce450 con 0x7f75740af8b0 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75740ce730 con 0x7f75740b4bb0 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f757bbd7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f75740cea10 con 0x7f75740b3be0 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b4bb0 0x7f75740cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f75740b4bb0 0x7f75740cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42580/0 (socket says 172.21.15.83:42580) 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 -- 172.21.15.83:0/146618850 learned_addr learned my addr 172.21.15.83:0/146618850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579973640 1 --2- 172.21.15.83:0/146618850 >> v2:172.21.15.155:3300/0 conn(0x7f75740b3be0 0x7f75740af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 -- 172.21.15.83:0/146618850 >> v2:172.21.15.155:3300/0 conn(0x7f75740b3be0 msgr2=0x7f75740af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 --2- 172.21.15.83:0/146618850 >> v2:172.21.15.155:3300/0 conn(0x7f75740b3be0 0x7f75740af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 -- 172.21.15.83:0/146618850 >> v1:172.21.15.155:6789/0 conn(0x7f75740af8b0 legacy=0x7f757406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.478+0000 7f7579172640 1 -- 172.21.15.83:0/146618850 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740cecf0 con 0x7f75740b4bb0 2024-05-17T22:08:04.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f7579172640 1 --2- 172.21.15.83:0/146618850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cf4f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f75740d0040 tx=0x7f756801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cdd646a026b108 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f7578971640 1 -- 172.21.15.83:0/146618850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680a5690 con 0x7f75740b4bb0 2024-05-17T22:08:04.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f7578971640 1 -- 172.21.15.83:0/146618850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75680a5830 con 0x7f75740b4bb0 2024-05-17T22:08:04.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f7578971640 1 -- 172.21.15.83:0/146618850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680a5c00 con 0x7f75740b4bb0 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 -- 172.21.15.83:0/146618850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 msgr2=0x7f75740cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 --2- 172.21.15.83:0/146618850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cf4f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f75740d0040 tx=0x7f756801c900 comp rx=0 tx=0).stop 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 -- 172.21.15.83:0/146618850 shutdown_connections 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 --2- 172.21.15.83:0/146618850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cf4f0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 --2- 172.21.15.83:0/146618850 >> v2:172.21.15.155:3300/0 conn(0x7f75740b3be0 0x7f75740af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 -- 172.21.15.83:0/146618850 >> 172.21.15.83:0/146618850 conn(0x7f75740b4090 msgr2=0x7f75740b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 -- 172.21.15.83:0/146618850 shutdown_connections 2024-05-17T22:08:04.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.482+0000 7f757bbd7640 1 -- 172.21.15.83:0/146618850 wait complete. 2024-05-17T22:08:04.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 Processor -- start 2024-05-17T22:08:04.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 -- start start 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75740af8b0 0x7f7574165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 0x7f75740ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75740b7d90 con 0x7f75740b4bb0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f757406d0b0 con 0x7f75740af8b0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757bbd7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75740b7f10 con 0x7f75740b3be0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42586/0 (socket says 172.21.15.83:42586) 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 -- 172.21.15.83:0/128925080 learned_addr learned my addr 172.21.15.83:0/128925080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f7579973640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75740af8b0 0x7f7574165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f7579172640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 0x7f75740ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 -- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 msgr2=0x7f75740ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 0x7f75740ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 -- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75740af8b0 msgr2=0x7f7574165f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75740af8b0 0x7f7574165f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 -- 172.21.15.83:0/128925080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757409f050 con 0x7f75740b4bb0 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f7579172640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 0x7f75740ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.486+0000 7f757a174640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f756400b440 tx=0x7f756400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 -- 172.21.15.83:0/128925080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564015020 con 0x7f75740b4bb0 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 -- 172.21.15.83:0/128925080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7564011850 con 0x7f75740b4bb0 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757409cc40 con 0x7f75740b4bb0 2024-05-17T22:08:04.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 -- 172.21.15.83:0/128925080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564011c40 con 0x7f75740b4bb0 2024-05-17T22:08:04.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757409f310 con 0x7f75740b4bb0 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 -- 172.21.15.83:0/128925080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7564018020 con 0x7f75740b4bb0 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 0x7f7560071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f75727fc640 1 -- 172.21.15.83:0/128925080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f756408f260 con 0x7f75740b4bb0 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f7579973640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 0x7f7560071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f7579973640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 0x7f7560071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f756c00a8d0 tx=0x7f756c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f7582e51000 1 Processor -- start 2024-05-17T22:08:04.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.490+0000 7f7582e51000 1 -- start start 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba187df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba188330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0ba025ca0 con 0x55f0ba041fa0 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0b9f98110 con 0x55f0ba037600 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f0b9f87cb0 con 0x55f0ba020d30 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42598/0 (socket says 172.21.15.83:42598) 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 -- 172.21.15.83:0/1396890182 learned_addr learned my addr 172.21.15.83:0/1396890182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f757a174640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba188330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579172640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba187df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 -- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 msgr2=0x55f0ba188330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba188330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 -- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 msgr2=0x55f0ba187df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba187df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 -- 172.21.15.83:0/1396890182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0b9f4c370 con 0x55f0ba041fa0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579172640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba187df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7579973640 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f756c00eb70 tx=0x7f756c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89558deffa5f2fca server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f755b7fe640 1 -- 172.21.15.83:0/1396890182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c01a050 con 0x55f0ba041fa0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f755b7fe640 1 -- 172.21.15.83:0/1396890182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c024070 con 0x55f0ba041fa0 2024-05-17T22:08:04.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f755b7fe640 1 -- 172.21.15.83:0/1396890182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c01f440 con 0x55f0ba041fa0 2024-05-17T22:08:04.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- 172.21.15.83:0/1396890182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 msgr2=0x55f0ba1878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f756c00eb70 tx=0x7f756c011000 comp rx=0 tx=0).stop 2024-05-17T22:08:04.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- 172.21.15.83:0/1396890182 shutdown_connections 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba188330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba187df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 --2- 172.21.15.83:0/1396890182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1878b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- 172.21.15.83:0/1396890182 >> 172.21.15.83:0/1396890182 conn(0x55f0b9c49460 msgr2=0x55f0ba13ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- 172.21.15.83:0/1396890182 shutdown_connections 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- 172.21.15.83:0/1396890182 wait complete. 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 -- start start 2024-05-17T22:08:04.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.494+0000 7f7582e51000 1 Processor -- start 2024-05-17T22:08:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 0x55f0ba1442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba144810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0b9f43d40 con 0x55f0ba041fa0 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0b9f8bcf0 con 0x55f0ba020d30 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f0b9f30e90 con 0x55f0ba037600 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 10 client.0 _ll_register_callbacks cb 0x55f0b9f65fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42614/0 (socket says 172.21.15.83:42614) 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 -- 172.21.15.83:0/468837381 learned_addr learned my addr 172.21.15.83:0/468837381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579973640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 0x55f0ba1442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579172640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 -- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 msgr2=0x55f0ba144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 -- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 msgr2=0x55f0ba1442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 0x55f0ba1442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 -- 172.21.15.83:0/468837381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b9f566e0 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579973640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 0x55f0ba1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579172640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f757a174640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f756400b410 tx=0x7f756408f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c596452702d9614 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756408b060 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f756405ac00 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756c024070 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f7564074070 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 10 client.8451 fetch_fsmap learned FSMap version 109 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55f0b9f46690 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c024070 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f756408db00 con 0x55f0ba041fa0 2024-05-17T22:08:04.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 10 client.8451 fetch_fsmap finished waiting for FSMap version 109 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 10 client.8451 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 20 client.8451 populate_metadata read hostname 'smithi083' 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c2e7d0 0x55f0ba18e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 4 client.8451 mds_command: new command op to 8358 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 --> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0b9f23a00 con 0x55f0b9c2e7d0 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579973640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c2e7d0 0x55f0ba18e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 10 client.8451 ms_handle_connect on v2:172.21.15.83:6835/1541446976 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7579973640 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c2e7d0 0x55f0ba18e920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95baa99eccc3ca17 server_cookie=ad83c8fb97f38a5f in_seq=0 out_seq=0 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 1 -- 172.21.15.83:0/468837381 <== mds.0 v2:172.21.15.83:6835/1541446976 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f0b9f23a00 con 0x55f0b9c2e7d0 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 10 client.8451 handle_command_reply: tid=0 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 client.8451 shutdown 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 msgr2=0x55f0ba149460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f756400b410 tx=0x7f756408f1f0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 shutdown_connections 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c2e7d0 0x55f0ba18e920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba149460 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7558ff9640 0 client.8451 ms_handle_reset on v2:172.21.15.83:6835/1541446976 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 --2- 172.21.15.83:0/468837381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba020d30 0x55f0ba1442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 >> 172.21.15.83:0/468837381 conn(0x55f0b9c49460 msgr2=0x55f0ba13ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 shutdown_connections 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.498+0000 7f7582e51000 1 -- 172.21.15.83:0/468837381 wait complete. 2024-05-17T22:08:04.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.502+0000 7f7582e51000 20 client.8451 trim_cache size 0 max 16384 2024-05-17T22:08:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.518+0000 7f7582e51000 1 Processor -- start 2024-05-17T22:08:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.518+0000 7f7582e51000 1 -- start start 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba041fa0 0x55f0ba19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7574202710 con 0x55f0ba020d30 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75740708a0 con 0x55f0ba041fa0 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f75742020c0 con 0x55f0ba037600 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42628/0 (socket says 172.21.15.83:42628) 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 -- 172.21.15.83:0/3852475328 learned_addr learned my addr 172.21.15.83:0/3852475328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 -- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 msgr2=0x55f0ba19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 -- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba041fa0 msgr2=0x55f0ba19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba041fa0 0x55f0ba19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 -- 172.21.15.83:0/3852475328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0ba00ecb0 con 0x55f0ba020d30 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7579973640 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e5a0 tx=0x7f756c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee0c1b4d01b0204 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f755affd640 1 -- 172.21.15.83:0/3852475328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c036070 con 0x55f0ba020d30 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f755affd640 1 -- 172.21.15.83:0/3852475328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c024030 con 0x55f0ba020d30 2024-05-17T22:08:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f755affd640 1 -- 172.21.15.83:0/3852475328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c01a070 con 0x55f0ba020d30 2024-05-17T22:08:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- 172.21.15.83:0/3852475328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 msgr2=0x55f0ba1446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f756c00e5a0 tx=0x7f756c01d000 comp rx=0 tx=0).stop 2024-05-17T22:08:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- 172.21.15.83:0/3852475328 shutdown_connections 2024-05-17T22:08:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba041fa0 0x55f0ba19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba037600 0x55f0ba19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 --2- 172.21.15.83:0/3852475328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba020d30 0x55f0ba1446b0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- 172.21.15.83:0/3852475328 >> 172.21.15.83:0/3852475328 conn(0x55f0b9c49460 msgr2=0x55f0ba186d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- 172.21.15.83:0/3852475328 shutdown_connections 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- 172.21.15.83:0/3852475328 wait complete. 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 -- start start 2024-05-17T22:08:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.522+0000 7f7582e51000 1 Processor -- start 2024-05-17T22:08:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba1fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba1fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0b9f56c10 con 0x55f0ba041fa0 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f0ba147c20 con 0x55f0ba037600 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x55f0ba00e820 con 0x55f0ba020d30 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 10 client.0 _ll_register_callbacks cb 0x7f756405b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42632/0 (socket says 172.21.15.83:42632) 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 -- 172.21.15.83:0/322547251 learned_addr learned my addr 172.21.15.83:0/322547251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579973640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba1fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579172640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba1fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 -- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 msgr2=0x55f0ba1fe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba1fe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 -- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 msgr2=0x55f0ba1fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba1fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 -- 172.21.15.83:0/322547251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0b9f474f0 con 0x55f0ba041fa0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579172640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba1fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579973640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba1fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f757a174640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f756408f010 tx=0x7f756400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96c2169ee5db207 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756405b2f0 con 0x55f0ba041fa0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f756408db00 con 0x55f0ba041fa0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756c024030 con 0x55f0ba041fa0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f756405fd50 con 0x55f0ba041fa0 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 10 client.8457 fetch_fsmap learned FSMap version 109 2024-05-17T22:08:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55f0b9f514f0 con 0x55f0ba041fa0 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c024030 con 0x55f0ba041fa0 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 109) ==== 3494+0+0 (secure 0 0 0) 0x7f756405ac00 con 0x55f0ba041fa0 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 10 client.8457 fetch_fsmap finished waiting for FSMap version 109 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 10 client.8457 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 20 client.8457 populate_metadata read hostname 'smithi083' 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c1c830 0x55f0ba1b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 4 client.8457 mds_command: new command op to 8358 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 --> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] -- command(tid 0: {"prefix": "status"}) -- 0x55f0b9f9a060 con 0x55f0b9c1c830 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579973640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c1c830 0x55f0ba1b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 10 client.8457 ms_handle_connect on v2:172.21.15.83:6835/1541446976 2024-05-17T22:08:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7579973640 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c1c830 0x55f0ba1b9ca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f49ca30edf78f9 server_cookie=21815e0be285b77d in_seq=0 out_seq=0 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 1 -- 172.21.15.83:0/322547251 <== mds.0 v2:172.21.15.83:6835/1541446976 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f0b9f9a060 con 0x55f0b9c1c830 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 10 client.8457 handle_command_reply: tid=0 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 client.8457 shutdown 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 msgr2=0x55f0ba1fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f756408f010 tx=0x7f756400e000 comp rx=0 tx=0).stop 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 shutdown_connections 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:6835/1541446976,v1:172.21.15.83:6837/1541446976] conn(0x55f0b9c1c830 0x55f0ba1b9ca0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0ba041fa0 0x55f0ba1fee60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f0ba037600 0x55f0ba1fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7558ff9640 0 client.8457 ms_handle_reset on v2:172.21.15.83:6835/1541446976 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 --2- 172.21.15.83:0/322547251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55f0ba020d30 0x55f0ba1fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 >> 172.21.15.83:0/322547251 conn(0x55f0b9c49460 msgr2=0x55f0ba13e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 shutdown_connections 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 1 -- 172.21.15.83:0/322547251 wait complete. 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.526+0000 7f7582e51000 20 client.8457 trim_cache size 0 max 16384 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce", 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8358, 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-05-17T22:08:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 4.6209806279999999, 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 109, 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 111, 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 111, 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 6.5751331830000002 2024-05-17T22:08:04.525 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:04.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 msgr2=0x7f7560071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 0x7f7560071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f756c00a8d0 tx=0x7f756c008040 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 msgr2=0x7f75740cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f756400b440 tx=0x7f756400b900 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 shutdown_connections 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75740b4bb0 0x7f75740cab30 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75740b3be0 0x7f75740ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f756006f430 0x7f7560071940 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 --2- 172.21.15.83:0/128925080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75740af8b0 0x7f7574165f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 >> 172.21.15.83:0/128925080 conn(0x7f75740b4090 msgr2=0x7f7574070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 shutdown_connections 2024-05-17T22:08:04.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.530+0000 7f757bbd7640 1 -- 172.21.15.83:0/128925080 wait complete. 2024-05-17T22:08:04.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-17T22:08:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 Processor -- start 2024-05-17T22:08:04.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 -- start start 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95d806cea0 0x7f95d80af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95d809bcc0 0x7f95d80afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8160850 con 0x7f95d81602e0 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f95d8160b30 con 0x7f95d806cea0 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dfc2a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f95d8160e10 con 0x7f95d809bcc0 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95d806cea0 0x7f95d80af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95d806cea0 0x7f95d80af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42646/0 (socket says 172.21.15.83:42646) 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 -- 172.21.15.83:0/83350690 learned_addr learned my addr 172.21.15.83:0/83350690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95de1c7640 1 --2- 172.21.15.83:0/83350690 >> v2:172.21.15.155:3300/0 conn(0x7f95d809bcc0 0x7f95d80afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 -- 172.21.15.83:0/83350690 >> v2:172.21.15.155:3300/0 conn(0x7f95d809bcc0 msgr2=0x7f95d80afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 --2- 172.21.15.83:0/83350690 >> v2:172.21.15.155:3300/0 conn(0x7f95d809bcc0 0x7f95d80afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 -- 172.21.15.83:0/83350690 >> v1:172.21.15.155:6789/0 conn(0x7f95d81602e0 legacy=0x7f95d806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:04.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.690+0000 7f95dd1c5640 1 -- 172.21.15.83:0/83350690 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81610f0 con 0x7f95d806cea0 2024-05-17T22:08:04.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dd1c5640 1 --2- 172.21.15.83:0/83350690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d806cea0 0x7f95d80af7c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f95d806d870 tx=0x7f95c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c994ebbb79b075f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dc9c4640 1 -- 172.21.15.83:0/83350690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab5f0 con 0x7f95d806cea0 2024-05-17T22:08:04.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dc9c4640 1 -- 172.21.15.83:0/83350690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c80b0070 con 0x7f95d806cea0 2024-05-17T22:08:04.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dc9c4640 1 -- 172.21.15.83:0/83350690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab9b0 con 0x7f95d806cea0 2024-05-17T22:08:04.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 -- 172.21.15.83:0/83350690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d806cea0 msgr2=0x7f95d80af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/83350690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d806cea0 0x7f95d80af7c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f95d806d870 tx=0x7f95c801c520 comp rx=0 tx=0).stop 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 -- 172.21.15.83:0/83350690 shutdown_connections 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/83350690 >> v2:172.21.15.155:3300/0 conn(0x7f95d809bcc0 0x7f95d80afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/83350690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d806cea0 0x7f95d80af7c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 -- 172.21.15.83:0/83350690 >> 172.21.15.83:0/83350690 conn(0x7f95d809b8b0 msgr2=0x7f95d80b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 -- 172.21.15.83:0/83350690 shutdown_connections 2024-05-17T22:08:04.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.694+0000 7f95dfc2a640 1 -- 172.21.15.83:0/83350690 wait complete. 2024-05-17T22:08:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 Processor -- start 2024-05-17T22:08:04.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- start start 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95d809bcc0 0x7f95d814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d81602e0 0x7f95d8158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95d816cdd0 con 0x7f95d814ce40 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f95d816cc50 con 0x7f95d81602e0 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f95d816cf50 con 0x7f95d809bcc0 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42652/0 (socket says 172.21.15.83:42652) 2024-05-17T22:08:04.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 -- 172.21.15.83:0/846438900 learned_addr learned my addr 172.21.15.83:0/846438900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dd1c5640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d81602e0 0x7f95d8158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dd9c6640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95d809bcc0 0x7f95d814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 -- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95d809bcc0 msgr2=0x7f95d814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95d809bcc0 0x7f95d814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 -- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d81602e0 msgr2=0x7f95d8158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d81602e0 0x7f95d8158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 -- 172.21.15.83:0/846438900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d81544a0 con 0x7f95d814ce40 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95de1c7640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f95d0007d70 tx=0x7f95d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0014420 con 0x7f95d814ce40 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d0019070 con 0x7f95d814ce40 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d8150470 con 0x7f95d814ce40 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0014820 con 0x7f95d814ce40 2024-05-17T22:08:04.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.698+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81584b0 con 0x7f95d814ce40 2024-05-17T22:08:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d8154670 con 0x7f95d814ce40 2024-05-17T22:08:04.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95d00149c0 con 0x7f95d814ce40 2024-05-17T22:08:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95ce7fc640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 0x7f95bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95dd9c6640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 0x7f95bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f95d008fb30 con 0x7f95d814ce40 2024-05-17T22:08:04.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.702+0000 7f95dd9c6640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 0x7f95bc071910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b810 tx=0x7f95d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:04.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.706+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f95d005ca00 con 0x7f95d814ce40 2024-05-17T22:08:04.884 INFO:tasks.ceph.mds.c.smithi083.stderr:2024-05-17T22:08:04.894+0000 7fd4a8336640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:08:04.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f95d80af7c0 con 0x7f95d814ce40 2024-05-17T22:08:04.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95ce7fc640 1 -- 172.21.15.83:0/846438900 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) ==== 53+0+73 (secure 0 0 0) 0x7f95d005c300 con 0x7f95d814ce40 2024-05-17T22:08:04.888 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 msgr2=0x7f95bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 0x7f95bc071910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b810 tx=0x7f95d4008040 comp rx=0 tx=0).stop 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 msgr2=0x7f95d81591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f95d0007d70 tx=0x7f95d000e4b0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 shutdown_connections 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95d814ce40 0x7f95d81591f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f95bc06f400 0x7f95bc071910 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95d81602e0 0x7f95d8158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 --2- 172.21.15.83:0/846438900 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95d809bcc0 0x7f95d814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.898+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 >> 172.21.15.83:0/846438900 conn(0x7f95d809b8b0 msgr2=0x7f95d80b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.902+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 shutdown_connections 2024-05-17T22:08:04.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:04.902+0000 7f95dfc2a640 1 -- 172.21.15.83:0/846438900 wait complete. 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-17T22:08:04.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:08:04.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:04.910 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-05-17T22:08:04.915 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.916 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41859sec preferred_lft 41859sec 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 02:42:e6:18:ed:9c brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-17T22:08:04.917 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-05-17T22:08:04.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:04.918 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-17T22:08:04.970 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:08:04.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:04.972 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-05-17T22:08:05.022 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-05-17T22:08:05.022 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-17T22:08:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:08:05.024 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:08:05.024 INFO:teuthology.orchestra.run.smithi083.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-05-17T22:08:05.024 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-17T22:08:05.024 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-17T22:08:05.025 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-05-17T22:08:05.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-17T22:08:05.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-17T22:08:05.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-17T22:08:05.025 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-05-17T22:08:05.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-17T22:08:05.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:05.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:05.143 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:08:05.191 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:08:05.192 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:08:05.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:08:05.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:05.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-05-17T22:08:05.247 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-17T22:08:05.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:08:05.247 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:05.295 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-17T22:08:05.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-17T22:08:05.295 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:05.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:08:05.403 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:08:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:08:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:05.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:05.455 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:08:05.503 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-17T22:08:05.503 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-17T22:08:05.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:08:05.562 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:08:05.615 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:08:05.616 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:08:05.617 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:05.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:05.618 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:08:05.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-05-17T22:08:05.666+0000 7fe2e1dd5580 -1 init, newargv = 0x556fb89d1cc0 newargc=15 2024-05-17T22:08:05.657 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49874]: starting ceph client 2024-05-17T22:08:05.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49874]: starting fuse 2024-05-17T22:08:06.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-05-17T22:08:06.781 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-05-17T22:08:06.832 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-05-17T22:08:06.833 INFO:teuthology.orchestra.run.smithi083.stdout:/run/lock 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-05-17T22:08:06.834 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/fs/nfsd 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysusers.service 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/rpc_pipefs 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-05-17T22:08:06.835 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-0 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-1 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-2 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph/osd/ceph-3 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:06.836 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:06.837 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:06.837 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:06.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:06.838 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-05-17T22:08:06.887 INFO:teuthology.orchestra.run.smithi083.stdout:45 2024-05-17T22:08:06.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-05-17T22:08:06.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-05-17T22:08:06.887 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-05-17T22:08:06.888 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-05-17T22:08:06.889 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.012 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.49874.asok 2024-05-17T22:08:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49874.asok 2024-05-17T22:08:07.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:07.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49874.asok status 2024-05-17T22:08:07.201 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "931809a9e5d295bd61984693cac44fe131ded4bc", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3794-g931809a9 (931809a9e5d295bd61984693cac44fe131ded4bc) squid (dev)", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "49874", 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-05-17T22:08:07.202 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 8469, 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 8469 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3303455889 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-05-17T22:08:07.203 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3303455889 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.8469 192.168.0.1:0/3303455889", 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/3303455889", 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 109, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 111, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-05-17T22:08:07.204 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:07.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:07.218 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-05-17T22:08:07.218 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:07.224 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-05-17T22:08:07.224 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:07.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:07.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:07.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:07.279 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-17T22:08:07.352 INFO:teuthology.orchestra.run.smithi083.stdout:./ 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/charlie 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/alpha 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/flushed/bravo 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/unflushed 2024-05-17T22:08:07.353 INFO:teuthology.orchestra.run.smithi083.stdout:./parent/unflushed/jfile 2024-05-17T22:08:07.354 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.354 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.354 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.354 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.354 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-17T22:08:07.355 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.469 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-05-17T22:08:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.479 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.479 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.480 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.480 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.480 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-17T22:08:07.480 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.582 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627776 2024-05-17T22:08:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-17T22:08:07.592 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.698 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627777 2024-05-17T22:08:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.708 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.708 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.709 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.709 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.709 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-17T22:08:07.709 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.813 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628280 2024-05-17T22:08:07.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-17T22:08:07.823 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:07.932 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627778 2024-05-17T22:08:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:07.942 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:07.942 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:07.942 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:07.943 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:07.943 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-17T22:08:07.943 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:08.051 INFO:teuthology.orchestra.run.smithi083.stdout:1099511628279 2024-05-17T22:08:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:08.060 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:08.060 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:08.060 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:08.060 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:08.061 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-17T22:08:08.061 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:08.169 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627779 2024-05-17T22:08:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:08.179 DEBUG:teuthology.orchestra.run.smithi083:> stdin-killer -- python3 -c ' 2024-05-17T22:08:08.179 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-05-17T22:08:08.179 DEBUG:teuthology.orchestra.run.smithi083:> import stat 2024-05-17T22:08:08.179 DEBUG:teuthology.orchestra.run.smithi083:> 2024-05-17T22:08:08.179 DEBUG:teuthology.orchestra.run.smithi083:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-17T22:08:08.180 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-05-17T22:08:08.281 INFO:teuthology.orchestra.run.smithi083.stdout:1099511627780 2024-05-17T22:08:08.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-17T22:08:08.290 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-17T22:08:08.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:08:08.400 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-17T22:08:08.455 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-17T22:08:08.455 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:08:08.455 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:08:08.455 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:08.462 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:08:08.462 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:08.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-17T22:08:08.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 Processor -- start 2024-05-17T22:08:08.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 -- start start 2024-05-17T22:08:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 0x7ffa2c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 0x7ffa2c1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2c0b5640 con 0x7ffa2c0cdf40 2024-05-17T22:08:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2c070870 con 0x7ffa2c0cf830 2024-05-17T22:08:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa32a0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c0b58e0 con 0x7ffa2c15d700 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa2bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 0x7ffa2c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa30fa9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 0x7ffa2c1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa2b7fe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7ffa2c0cdf40 0x7ffa2c15cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:41778/0 (socket says 172.21.15.83:41778) 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.722+0000 7ffa2b7fe640 1 -- 172.21.15.83:0/1284615868 learned_addr learned my addr 172.21.15.83:0/1284615868 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa30fa9640 1 -- 172.21.15.83:0/1284615868 >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 msgr2=0x7ffa2c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa30fa9640 1 --2- 172.21.15.83:0/1284615868 >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 0x7ffa2c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa30fa9640 1 -- 172.21.15.83:0/1284615868 >> v1:172.21.15.155:6789/0 conn(0x7ffa2c0cdf40 legacy=0x7ffa2c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa30fa9640 1 -- 172.21.15.83:0/1284615868 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c06f060 con 0x7ffa2c15d700 2024-05-17T22:08:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa2bfff640 1 --2- 172.21.15.83:0/1284615868 >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 0x7ffa2c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:08:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa30fa9640 1 --2- 172.21.15.83:0/1284615868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 0x7ffa2c1663d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009d90 tx=0x7ffa2401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8011ac10105ecd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa2affd640 1 -- 172.21.15.83:0/1284615868 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240b2070 con 0x7ffa2c15d700 2024-05-17T22:08:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa2affd640 1 -- 172.21.15.83:0/1284615868 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa240a5880 con 0x7ffa2c15d700 2024-05-17T22:08:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.726+0000 7ffa2affd640 1 -- 172.21.15.83:0/1284615868 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240a5c50 con 0x7ffa2c15d700 2024-05-17T22:08:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- 172.21.15.83:0/1284615868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 msgr2=0x7ffa2c1663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/1284615868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 0x7ffa2c1663d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009d90 tx=0x7ffa2401c970 comp rx=0 tx=0).stop 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- 172.21.15.83:0/1284615868 shutdown_connections 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/1284615868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c15d700 0x7ffa2c1663d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/1284615868 >> v2:172.21.15.155:3300/0 conn(0x7ffa2c0cf830 0x7ffa2c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- 172.21.15.83:0/1284615868 >> 172.21.15.83:0/1284615868 conn(0x7ffa2c06cd90 msgr2=0x7ffa2c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- 172.21.15.83:0/1284615868 shutdown_connections 2024-05-17T22:08:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- 172.21.15.83:0/1284615868 wait complete. 2024-05-17T22:08:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 Processor -- start 2024-05-17T22:08:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- start start 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 0x7ffa2c151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c0cf830 0x7ffa2c15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1692b0 con 0x7ffa2c15d700 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c169130 con 0x7ffa2c0cdf40 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa32a0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c169430 con 0x7ffa2c0cf830 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38196/0 (socket says 172.21.15.83:38196) 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 -- 172.21.15.83:0/3227479902 learned_addr learned my addr 172.21.15.83:0/3227479902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa2bfff640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 0x7ffa2c151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa2b7fe640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c0cf830 0x7ffa2c15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 -- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c0cf830 msgr2=0x7ffa2c15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c0cf830 0x7ffa2c15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 -- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 msgr2=0x7ffa2c151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 0x7ffa2c151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa30fa9640 1 -- 172.21.15.83:0/3227479902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c15e350 con 0x7ffa2c15d700 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.730+0000 7ffa2bfff640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 0x7ffa2c151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa30fa9640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffa240b0040 tx=0x7ffa24019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240b2050 con 0x7ffa2c15d700 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa240be070 con 0x7ffa2c15d700 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c15e630 con 0x7ffa2c15d700 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240ad850 con 0x7ffa2c15d700 2024-05-17T22:08:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c15ebf0 con 0x7ffa2c15d700 2024-05-17T22:08:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2c0b3aa0 con 0x7ffa2c15d700 2024-05-17T22:08:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffa240ad9f0 con 0x7ffa2c15d700 2024-05-17T22:08:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 0x7ffa18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa2bfff640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 0x7ffa18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.734+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7ffa2412ae50 con 0x7ffa2c15d700 2024-05-17T22:08:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.738+0000 7ffa2bfff640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 0x7ffa18071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009b10 tx=0x7ffa1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.738+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ffa24016560 con 0x7ffa2c15d700 2024-05-17T22:08:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.914+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa2c073d40 con 0x7ffa2c15d700 2024-05-17T22:08:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.914+0000 7ffa28ff9640 1 -- 172.21.15.83:0/3227479902 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v111) ==== 80+0+242 (secure 0 0 0) 0x7ffa240f7d70 con 0x7ffa2c15d700 2024-05-17T22:08:08.906 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6837/78757339 2024-05-18T22:07:57.881130+0000 2024-05-17T22:08:08.906 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6835/78757339 2024-05-18T22:07:57.881130+0000 2024-05-17T22:08:08.906 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6833/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:08:08.906 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.155:6832/1193425174 2024-05-18T22:07:40.363116+0000 2024-05-17T22:08:08.906 INFO:teuthology.orchestra.run.smithi083.stderr:listed 4 entries 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 msgr2=0x7ffa18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 0x7ffa18071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009b10 tx=0x7ffa1c008040 comp rx=0 tx=0).stop 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 msgr2=0x7ffa2c0d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ffa240b0040 tx=0x7ffa24019710 comp rx=0 tx=0).stop 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 shutdown_connections 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa2c15d700 0x7ffa2c0d1490 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa2c0cf830 0x7ffa2c15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ffa1806f430 0x7ffa18071940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 --2- 172.21.15.83:0/3227479902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa2c0cdf40 0x7ffa2c151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 >> 172.21.15.83:0/3227479902 conn(0x7ffa2c06cd90 msgr2=0x7ffa2c158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 shutdown_connections 2024-05-17T22:08:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:08.918+0000 7ffa32a0c640 1 -- 172.21.15.83:0/3227479902 wait complete. 2024-05-17T22:08:08.926 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-05-17T22:08:08.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:08.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-17T22:08:08.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[49874]: fuse finished with error 0 and tester_r 0 2024-05-17T22:08:08.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:08.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:08.939 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-17T22:08:09.018 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:08:09.019 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:08:09.019 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:09.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:08:09.025 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:08:09.025 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:08:09.025 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:09.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:08:09.067 INFO:tasks.cephfs.mount:Terminating background process 2024-05-17T22:08:09.067 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:08:09.067 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:09.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:08:09.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 Processor -- start 2024-05-17T22:08:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 -- start start 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f145c06f3c0 0x7f145c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f145c1608c0 0x7f145c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145c071a20 con 0x7f145c071610 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f145c06e690 con 0x7f145c06f3c0 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f1462a8e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f145c160cd0 con 0x7f145c1608c0 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146082a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f145c06f3c0 0x7f145c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f145bfff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f145c071610 0x7f145c1601b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34940/0 (socket says 172.21.15.83:34940) 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f145bfff640 1 -- 172.21.15.83:0/2289883351 learned_addr learned my addr 172.21.15.83:0/2289883351 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146102b640 1 --2- 172.21.15.83:0/2289883351 >> v2:172.21.15.155:3300/0 conn(0x7f145c1608c0 0x7f145c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146082a640 1 -- 172.21.15.83:0/2289883351 >> v2:172.21.15.155:3300/0 conn(0x7f145c1608c0 msgr2=0x7f145c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f145b7fe640 1 -- 172.21.15.83:0/2289883351 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 165784762 0 0) 0x7f144c001120 con 0x7f145c071610 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146082a640 1 --2- 172.21.15.83:0/2289883351 >> v2:172.21.15.155:3300/0 conn(0x7f145c1608c0 0x7f145c0b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146082a640 1 -- 172.21.15.83:0/2289883351 >> v1:172.21.15.83:6789/0 conn(0x7f145c071610 legacy=0x7f145c1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.282+0000 7f146082a640 1 -- 172.21.15.83:0/2289883351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c160f50 con 0x7f145c06f3c0 2024-05-17T22:08:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f146082a640 1 --2- 172.21.15.83:0/2289883351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c0710d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f145c0b85b0 tx=0x7f145001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d9368d79b67f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f145b7fe640 1 -- 172.21.15.83:0/2289883351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b0070 con 0x7f145c06f3c0 2024-05-17T22:08:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f145b7fe640 1 -- 172.21.15.83:0/2289883351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500ab600 con 0x7f145c06f3c0 2024-05-17T22:08:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f145b7fe640 1 -- 172.21.15.83:0/2289883351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500ab9d0 con 0x7f145c06f3c0 2024-05-17T22:08:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- 172.21.15.83:0/2289883351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 msgr2=0x7f145c0710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 --2- 172.21.15.83:0/2289883351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c0710d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f145c0b85b0 tx=0x7f145001c3c0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- 172.21.15.83:0/2289883351 shutdown_connections 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 --2- 172.21.15.83:0/2289883351 >> v2:172.21.15.155:3300/0 conn(0x7f145c1608c0 0x7f145c0b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 --2- 172.21.15.83:0/2289883351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c0710d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- 172.21.15.83:0/2289883351 >> 172.21.15.83:0/2289883351 conn(0x7f145c068730 msgr2=0x7f145c068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- 172.21.15.83:0/2289883351 shutdown_connections 2024-05-17T22:08:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- 172.21.15.83:0/2289883351 wait complete. 2024-05-17T22:08:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 Processor -- start 2024-05-17T22:08:09.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.286+0000 7f1462a8e640 1 -- start start 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f145c071610 0x7f145c13ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c1608c0 0x7f145c13f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f145c156fd0 con 0x7f145c06f3c0 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f145c156e50 con 0x7f145c1608c0 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f145c16c930 con 0x7f145c071610 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38228/0 (socket says 172.21.15.83:38228) 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 -- 172.21.15.83:0/3672598024 learned_addr learned my addr 172.21.15.83:0/3672598024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:09.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 -- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f145c071610 msgr2=0x7f145c13ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f145c071610 0x7f145c13ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 -- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c1608c0 msgr2=0x7f145c13f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c1608c0 0x7f145c13f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 -- 172.21.15.83:0/3672598024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c158820 con 0x7f145c06f3c0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f146082a640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1450018fd0 tx=0x7f145001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b0050 con 0x7f145c06f3c0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500bc070 con 0x7f145c06f3c0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500b5a30 con 0x7f145c06f3c0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c158b00 con 0x7f145c06f3c0 2024-05-17T22:08:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c1590c0 con 0x7f145c06f3c0 2024-05-17T22:08:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.290+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c0710d0 con 0x7f145c06f3c0 2024-05-17T22:08:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.294+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14500ab4c0 con 0x7f145c06f3c0 2024-05-17T22:08:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.294+0000 7f14597fa640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 0x7f1448071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.294+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f1450129c70 con 0x7f145c06f3c0 2024-05-17T22:08:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.294+0000 7f145bfff640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 0x7f1448071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.294+0000 7f145bfff640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 0x7f1448071910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f145c15a890 tx=0x7f144c007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.298+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f1450016070 con 0x7f145c06f3c0 2024-05-17T22:08:09.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.494+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145c0d33a0 con 0x7f145c06f3c0 2024-05-17T22:08:09.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.494+0000 7f14597fa640 1 -- 172.21.15.83:0/3672598024 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5191 (secure 0 0 0) 0x7f14500f6a10 con 0x7f145c06f3c0 2024-05-17T22:08:09.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:09.486 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:02.924755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8358,"qdb_cluster":[8358]},"id":8}]} 2024-05-17T22:08:09.486 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 109 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 msgr2=0x7f1448071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 0x7f1448071910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f145c15a890 tx=0x7f144c007970 comp rx=0 tx=0).stop 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 msgr2=0x7f145c1598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f1450018fd0 tx=0x7f145001ac00 comp rx=0 tx=0).stop 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 shutdown_connections 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f144806f400 0x7f1448071910 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f145c1608c0 0x7f145c13f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f145c071610 0x7f145c13ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 --2- 172.21.15.83:0/3672598024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f145c06f3c0 0x7f145c1598c0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 >> 172.21.15.83:0/3672598024 conn(0x7f145c068730 msgr2=0x7f145c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 shutdown_connections 2024-05-17T22:08:09.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.498+0000 7f1462a8e640 1 -- 172.21.15.83:0/3672598024 wait complete. 2024-05-17T22:08:09.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:09.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.666+0000 7fc12b4b4640 1 Processor -- start 2024-05-17T22:08:09.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.666+0000 7fc12b4b4640 1 -- start start 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc12b4b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc12415ffa0 0x7fc1240d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc12b4b4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1240d1150 0x7fc12406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc12b4b4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12406e610 con 0x7fc12406e200 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc12b4b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc1240d34e0 con 0x7fc12415ffa0 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc12b4b4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc1240d3780 con 0x7fc1240d1150 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129250640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc12415ffa0 0x7fc1240d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129a51640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc12406e200 0x7fc124163a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34946/0 (socket says 172.21.15.83:34946) 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129a51640 1 -- 172.21.15.83:0/822144731 learned_addr learned my addr 172.21.15.83:0/822144731 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc128a4f640 1 --2- 172.21.15.83:0/822144731 >> v2:172.21.15.155:3300/0 conn(0x7fc1240d1150 0x7fc12406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129250640 1 -- 172.21.15.83:0/822144731 >> v2:172.21.15.155:3300/0 conn(0x7fc1240d1150 msgr2=0x7fc12406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129250640 1 --2- 172.21.15.83:0/822144731 >> v2:172.21.15.155:3300/0 conn(0x7fc1240d1150 0x7fc12406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129250640 1 -- 172.21.15.83:0/822144731 >> v1:172.21.15.83:6789/0 conn(0x7fc12406e200 legacy=0x7fc124163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc129250640 1 -- 172.21.15.83:0/822144731 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1240d3a60 con 0x7fc12415ffa0 2024-05-17T22:08:09.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.670+0000 7fc11bfff640 1 -- 172.21.15.83:0/822144731 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210199772 0 0) 0x7fc11c0014a0 con 0x7fc12406e200 2024-05-17T22:08:09.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc129250640 1 --2- 172.21.15.83:0/822144731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12415ffa0 0x7fc1240d0c10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b5260 tx=0x7fc12001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec55db944052077 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc11bfff640 1 -- 172.21.15.83:0/822144731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1200b0070 con 0x7fc12415ffa0 2024-05-17T22:08:09.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc11bfff640 1 -- 172.21.15.83:0/822144731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1200ab600 con 0x7fc12415ffa0 2024-05-17T22:08:09.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc11bfff640 1 -- 172.21.15.83:0/822144731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1200ab9d0 con 0x7fc12415ffa0 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- 172.21.15.83:0/822144731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12415ffa0 msgr2=0x7fc1240d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/822144731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12415ffa0 0x7fc1240d0c10 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b5260 tx=0x7fc12001c480 comp rx=0 tx=0).stop 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- 172.21.15.83:0/822144731 shutdown_connections 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/822144731 >> v2:172.21.15.155:3300/0 conn(0x7fc1240d1150 0x7fc12406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/822144731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12415ffa0 0x7fc1240d0c10 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- 172.21.15.83:0/822144731 >> 172.21.15.83:0/822144731 conn(0x7fc1240d28f0 msgr2=0x7fc1240d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- 172.21.15.83:0/822144731 shutdown_connections 2024-05-17T22:08:09.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- 172.21.15.83:0/822144731 wait complete. 2024-05-17T22:08:09.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 Processor -- start 2024-05-17T22:08:09.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.674+0000 7fc12b4b4640 1 -- start start 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc12406e200 0x7fc12414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc12415ffa0 0x7fc1241626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc12416cd00 con 0x7fc1240d1150 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc12416cb80 con 0x7fc12406e200 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc12416ce80 con 0x7fc12415ffa0 2024-05-17T22:08:09.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38248/0 (socket says 172.21.15.83:38248) 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 -- 172.21.15.83:0/3966387868 learned_addr learned my addr 172.21.15.83:0/3966387868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 -- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc12415ffa0 msgr2=0x7fc1241626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc129a51640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc12415ffa0 0x7fc1241626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc129250640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc12406e200 0x7fc12414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc12415ffa0 0x7fc1241626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 -- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc12406e200 msgr2=0x7fc12414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc12406e200 0x7fc12414dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 -- 172.21.15.83:0/3966387868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1240cf2d0 con 0x7fc1240d1150 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc128a4f640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc11400d5f0 tx=0x7fc11400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140125f0 con 0x7fc1240d1150 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc114019070 con 0x7fc1240d1150 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc124164de0 con 0x7fc1240d1150 2024-05-17T22:08:09.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140129b0 con 0x7fc1240d1150 2024-05-17T22:08:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1240cf890 con 0x7fc1240d1150 2024-05-17T22:08:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12406dcc0 con 0x7fc1240d1150 2024-05-17T22:08:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.678+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc114012b50 con 0x7fc1240d1150 2024-05-17T22:08:09.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.682+0000 7fc119ffb640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 0x7fc108071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.682+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7fc11408f7a0 con 0x7fc1240d1150 2024-05-17T22:08:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.682+0000 7fc129250640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 0x7fc108071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:09.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.682+0000 7fc129250640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 0x7fc108071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ad000 tx=0x7fc1200a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:09.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.686+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc11405c5e0 con 0x7fc1240d1150 2024-05-17T22:08:09.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.878+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1240cf020 con 0x7fc1240d1150 2024-05-17T22:08:09.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc119ffb640 1 -- 172.21.15.83:0/3966387868 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5191 (secure 0 0 0) 0x7fc11405bee0 con 0x7fc1240d1150 2024-05-17T22:08:09.871 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:09.871 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:02.924755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8358,"qdb_cluster":[8358]},"id":8}]} 2024-05-17T22:08:09.872 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 109 2024-05-17T22:08:09.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 msgr2=0x7fc108071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 0x7fc108071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ad000 tx=0x7fc1200a6000 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 msgr2=0x7fc1240ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc11400d5f0 tx=0x7fc11400dab0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 shutdown_connections 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc12415ffa0 0x7fc1241626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1240d1150 0x7fc1240ce660 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc10806f400 0x7fc108071910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 --2- 172.21.15.83:0/3966387868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc12406e200 0x7fc12414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 >> 172.21.15.83:0/3966387868 conn(0x7fc1240d28f0 msgr2=0x7fc12415ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 shutdown_connections 2024-05-17T22:08:09.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:09.882+0000 7fc12b4b4640 1 -- 172.21.15.83:0/3966387868 wait complete. 2024-05-17T22:08:09.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:08:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 Processor -- start 2024-05-17T22:08:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 -- start start 2024-05-17T22:08:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff80b5d80 0x7f5ff806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff806cc90 con 0x7f5ff806c790 2024-05-17T22:08:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff806cf70 con 0x7f5ff815fcc0 2024-05-17T22:08:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ffdd55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff80711a0 con 0x7f5ff80b5d80 2024-05-17T22:08:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ff77fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5ff806c790 0x7f5ff8070a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34950/0 (socket says 172.21.15.83:34950) 2024-05-17T22:08:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ff77fe640 1 -- 172.21.15.83:0/4083226574 learned_addr learned my addr 172.21.15.83:0/4083226574 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008825255 0 0) 0x7f5ff806cc90 con 0x7f5ff806c790 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fe0003530 con 0x7f5ff806c790 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.042+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff80b5d80 0x7f5ff806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2575542342 0 0) 0x7f5fe0003530 con 0x7f5ff806c790 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff80b5d80 msgr2=0x7f5ff806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff80b5d80 0x7f5ff806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 >> v1:172.21.15.155:6789/0 conn(0x7f5ff815fcc0 legacy=0x7f5ff80b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff806d250 con 0x7f5ff806c790 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f5ff0001a80 con 0x7f5ff806c790 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 >> v1:172.21.15.83:6789/0 conn(0x7f5ff806c790 legacy=0x7f5ff8070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5fe0003a40 0x7f5fe0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fe0004810 0x7f5fe0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fe0005160 0x7f5fe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80711a0 con 0x7f5fe0005160 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe000c020 con 0x7f5fe0004810 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe000c220 con 0x7f5fe0003a40 2024-05-17T22:08:10.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fe0005160 0x7f5fe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 -- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5fe0003a40 msgr2=0x7f5fe0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5fe0003a40 0x7f5fe0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 -- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fe0004810 msgr2=0x7f5fe0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff6ffd640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fe0004810 0x7f5fe0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fe0004810 0x7f5fe0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 -- 172.21.15.83:0/4083226574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe0003530 con 0x7f5fe0005160 2024-05-17T22:08:10.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff6ffd640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fe0004810 0x7f5fe0004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fe0005160 0x7f5fe000b9d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009ab0 tx=0x7f5fe801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85882dd1490bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80b2070 con 0x7f5fe0005160 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe80a5680 con 0x7f5fe0005160 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff67fc640 1 -- 172.21.15.83:0/4083226574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80096b0 con 0x7f5fe0005160 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 -- 172.21.15.83:0/4083226574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fe0005160 msgr2=0x7f5fe000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/4083226574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fe0005160 0x7f5fe000b9d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009ab0 tx=0x7f5fe801ca90 comp rx=0 tx=0).stop 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ff7fff640 1 -- 172.21.15.83:0/4083226574 reap_dead start 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 -- 172.21.15.83:0/4083226574 shutdown_connections 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 -- 172.21.15.83:0/4083226574 >> 172.21.15.83:0/4083226574 conn(0x7f5ff80d20b0 msgr2=0x7f5ff80d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 -- 172.21.15.83:0/4083226574 shutdown_connections 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.046+0000 7f5ffdd55640 1 -- 172.21.15.83:0/4083226574 wait complete. 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 Processor -- start 2024-05-17T22:08:10.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- start start 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ff806fe40 0x7f5ff8070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff81503c0 0x7f5ff81507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80b5d80 con 0x7f5ff8147b40 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff806d810 con 0x7f5ff806fe40 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff80b5f00 con 0x7f5ff81503c0 2024-05-17T22:08:10.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38262/0 (socket says 172.21.15.83:38262) 2024-05-17T22:08:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 -- 172.21.15.83:0/3926814934 learned_addr learned my addr 172.21.15.83:0/3926814934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff7fff640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff81503c0 0x7f5ff81507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 -- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff81503c0 msgr2=0x7f5ff81507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff81503c0 0x7f5ff81507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 -- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ff806fe40 msgr2=0x7f5ff8070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ff806fe40 0x7f5ff8070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 -- 172.21.15.83:0/3926814934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff8070790 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ff77fe640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0002020 tx=0x7f5ff000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0018070 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff00070d0 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff815cd10 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00074a0 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.050+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff815d2d0 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff806dd30 con 0x7f5ff8147b40 2024-05-17T22:08:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ff001b070 con 0x7f5ff8147b40 2024-05-17T22:08:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5fdffff640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 0x7f5fe4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f5ff008f200 con 0x7f5ff8147b40 2024-05-17T22:08:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5ff6ffd640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 0x7f5fe4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.054+0000 7f5ff6ffd640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 0x7f5fe4071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81621b0 tx=0x7f5fec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.058+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f5ff8070790 con 0x7f5ff8147b40 2024-05-17T22:08:10.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.218+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ff8070b10 con 0x7f5ff8147b40 2024-05-17T22:08:10.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.218+0000 7f5fdffff640 1 -- 172.21.15.83:0/3926814934 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16860 (secure 0 0 0) 0x7f5ff005c220 con 0x7f5ff8147b40 2024-05-17T22:08:10.209 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:10.209 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":111,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:57.881192+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:07:09.949308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-17T22:07:11.233706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.83:6835/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.155:6833/1193425174":"2024-05-18T22:07:40.363116+0000","172.21.15.155:6832/1193425174":"2024-05-18T22:07:40.363116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 msgr2=0x7f5fe4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 0x7f5fe4071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ff81621b0 tx=0x7f5fec009040 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 msgr2=0x7f5ff8160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0002020 tx=0x7f5ff000a430 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 shutdown_connections 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5ff81503c0 0x7f5ff81507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f5fe406f400 0x7f5fe4071910 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ff806fe40 0x7f5ff8070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 --2- 172.21.15.83:0/3926814934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff8147b40 0x7f5ff8160120 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 >> 172.21.15.83:0/3926814934 conn(0x7f5ff80d20b0 msgr2=0x7f5ff81633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 shutdown_connections 2024-05-17T22:08:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.222+0000 7f5ffdd55640 1 -- 172.21.15.83:0/3926814934 wait complete. 2024-05-17T22:08:10.229 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:08:10.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 Processor -- start 2024-05-17T22:08:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 -- start start 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8bd806cea0 0x7f8bd80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd809bcc0 0x7f8bd8167380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bd809c0d0 con 0x7f8bd80b4910 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd80afc40 con 0x7f8bd809bcc0 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bde7fd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8bd815e110 con 0x7f8bd806cea0 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd809bcc0 0x7f8bd8167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bd809bcc0 0x7f8bd8167380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38274/0 (socket says 172.21.15.83:38274) 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 -- 172.21.15.83:0/734561884 learned_addr learned my addr 172.21.15.83:0/734561884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bd77fe640 1 --2- 172.21.15.83:0/734561884 >> v2:172.21.15.155:3300/0 conn(0x7f8bd806cea0 0x7f8bd80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 -- 172.21.15.83:0/734561884 >> v2:172.21.15.155:3300/0 conn(0x7f8bd806cea0 msgr2=0x7f8bd80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 --2- 172.21.15.83:0/734561884 >> v2:172.21.15.155:3300/0 conn(0x7f8bd806cea0 0x7f8bd80b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 -- 172.21.15.83:0/734561884 >> v1:172.21.15.83:6789/0 conn(0x7f8bd80b4910 legacy=0x7f8bd806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.382+0000 7f8bdcd9a640 1 -- 172.21.15.83:0/734561884 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd80b3a20 con 0x7f8bd809bcc0 2024-05-17T22:08:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bdcd9a640 1 --2- 172.21.15.83:0/734561884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd809bcc0 0x7f8bd8167380 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8073a60 tx=0x7f8bd001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938897af8b1d0a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bd6ffd640 1 -- 172.21.15.83:0/734561884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a5730 con 0x7f8bd809bcc0 2024-05-17T22:08:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bd6ffd640 1 -- 172.21.15.83:0/734561884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd00a58d0 con 0x7f8bd809bcc0 2024-05-17T22:08:10.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bd6ffd640 1 -- 172.21.15.83:0/734561884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a5ca0 con 0x7f8bd809bcc0 2024-05-17T22:08:10.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- 172.21.15.83:0/734561884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd809bcc0 msgr2=0x7f8bd8167380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/734561884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd809bcc0 0x7f8bd8167380 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8073a60 tx=0x7f8bd001c960 comp rx=0 tx=0).stop 2024-05-17T22:08:10.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- 172.21.15.83:0/734561884 shutdown_connections 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/734561884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd809bcc0 0x7f8bd8167380 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/734561884 >> v2:172.21.15.155:3300/0 conn(0x7f8bd806cea0 0x7f8bd80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- 172.21.15.83:0/734561884 >> 172.21.15.83:0/734561884 conn(0x7f8bd809b8b0 msgr2=0x7f8bd80b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- 172.21.15.83:0/734561884 shutdown_connections 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- 172.21.15.83:0/734561884 wait complete. 2024-05-17T22:08:10.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 Processor -- start 2024-05-17T22:08:10.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.386+0000 7f8bde7fd640 1 -- start start 2024-05-17T22:08:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd809bcc0 0x7f8bd81f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd80b64c0 0x7f8bd80b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd816a260 con 0x7f8bd80b4910 2024-05-17T22:08:10.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd816a0e0 con 0x7f8bd80b64c0 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd816a3e0 con 0x7f8bd809bcc0 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38284/0 (socket says 172.21.15.83:38284) 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 -- 172.21.15.83:0/2741059525 learned_addr learned my addr 172.21.15.83:0/2741059525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bdcd9a640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd80b64c0 0x7f8bd80b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 -- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd809bcc0 msgr2=0x7f8bd81f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd809bcc0 0x7f8bd81f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 -- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd80b64c0 msgr2=0x7f8bd80b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd80b64c0 0x7f8bd80b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 -- 172.21.15.83:0/2741059525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd80b7000 con 0x7f8bd80b4910 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd77fe640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c850 tx=0x7f8bcc00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc015070 con 0x7f8bd80b4910 2024-05-17T22:08:10.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0079d0 con 0x7f8bd80b4910 2024-05-17T22:08:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd8157780 con 0x7f8bd80b4910 2024-05-17T22:08:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc007dc0 con 0x7f8bd80b4910 2024-05-17T22:08:10.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd8157d00 con 0x7f8bd80b4910 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd806c790 con 0x7f8bd80b4910 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.390+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bcc018070 con 0x7f8bd80b4910 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.394+0000 7f8bd4ff9640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 0x7f8bc4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.394+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f8bcc08f1a0 con 0x7f8bd80b4910 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.394+0000 7f8bd7fff640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 0x7f8bc4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.394+0000 7f8bd7fff640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 0x7f8bc4071940 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000f70 tx=0x7f8bc80079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.398+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8bd8157780 con 0x7f8bd80b4910 2024-05-17T22:08:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.586+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd8070db0 con 0x7f8bd80b4910 2024-05-17T22:08:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.586+0000 7f8bd4ff9640 1 -- 172.21.15.83:0/2741059525 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5191 (secure 0 0 0) 0x7f8bcc05bfe0 con 0x7f8bd80b4910 2024-05-17T22:08:10.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:10.578 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:02.924755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8358,"qdb_cluster":[8358]},"id":8}]} 2024-05-17T22:08:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 109 2024-05-17T22:08:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 msgr2=0x7f8bc4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 0x7f8bc4071940 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000f70 tx=0x7f8bc80079c0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 msgr2=0x7f8bd81fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c850 tx=0x7f8bcc00cd10 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 shutdown_connections 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8bc406f430 0x7f8bc4071940 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bd80b64c0 0x7f8bd80b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd80b4910 0x7f8bd81fb9e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 --2- 172.21.15.83:0/2741059525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bd809bcc0 0x7f8bd81f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 >> 172.21.15.83:0/2741059525 conn(0x7f8bd809b8b0 msgr2=0x7f8bd80b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 shutdown_connections 2024-05-17T22:08:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.590+0000 7f8bde7fd640 1 -- 172.21.15.83:0/2741059525 wait complete. 2024-05-17T22:08:10.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:10.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 Processor -- start 2024-05-17T22:08:10.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 -- start start 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6a400b5a50 0x7f6a400b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a40072e30 0x7f6a40073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a40073780 con 0x7f6a4015fef0 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6a4006ff60 con 0x7f6a40072e30 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a4699a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f6a400b49b0 con 0x7f6a400b5a50 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a40072e30 0x7f6a40073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a40072e30 0x7f6a40073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38300/0 (socket says 172.21.15.83:38300) 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 -- 172.21.15.83:0/3109656985 learned_addr learned my addr 172.21.15.83:0/3109656985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:10.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a3f7fe640 1 --2- 172.21.15.83:0/3109656985 >> v2:172.21.15.155:3300/0 conn(0x7f6a400b5a50 0x7f6a400b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 -- 172.21.15.83:0/3109656985 >> v2:172.21.15.155:3300/0 conn(0x7f6a400b5a50 msgr2=0x7f6a400b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 --2- 172.21.15.83:0/3109656985 >> v2:172.21.15.155:3300/0 conn(0x7f6a400b5a50 0x7f6a400b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 -- 172.21.15.83:0/3109656985 >> v1:172.21.15.155:6789/0 conn(0x7f6a4015fef0 legacy=0x7f6a4006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:10.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.750+0000 7f6a44f37640 1 -- 172.21.15.83:0/3109656985 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400b4c90 con 0x7f6a40072e30 2024-05-17T22:08:10.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a44f37640 1 --2- 172.21.15.83:0/3109656985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a40072e30 0x7f6a40073240 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6a400d1530 tx=0x7f6a3801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bab521c8aa81cf server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a3effd640 1 -- 172.21.15.83:0/3109656985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a6500 con 0x7f6a40072e30 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a3effd640 1 -- 172.21.15.83:0/3109656985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a380a66a0 con 0x7f6a40072e30 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a3effd640 1 -- 172.21.15.83:0/3109656985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a380a6a70 con 0x7f6a40072e30 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 -- 172.21.15.83:0/3109656985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a40072e30 msgr2=0x7f6a40073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 --2- 172.21.15.83:0/3109656985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a40072e30 0x7f6a40073240 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6a400d1530 tx=0x7f6a3801c780 comp rx=0 tx=0).stop 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 -- 172.21.15.83:0/3109656985 shutdown_connections 2024-05-17T22:08:10.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 --2- 172.21.15.83:0/3109656985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a40072e30 0x7f6a40073240 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 --2- 172.21.15.83:0/3109656985 >> v2:172.21.15.155:3300/0 conn(0x7f6a400b5a50 0x7f6a400b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 -- 172.21.15.83:0/3109656985 >> 172.21.15.83:0/3109656985 conn(0x7f6a4006cd90 msgr2=0x7f6a400b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 -- 172.21.15.83:0/3109656985 shutdown_connections 2024-05-17T22:08:10.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.754+0000 7f6a4699a640 1 -- 172.21.15.83:0/3109656985 wait complete. 2024-05-17T22:08:10.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 Processor -- start 2024-05-17T22:08:10.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- start start 2024-05-17T22:08:10.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 0x7f6a401517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4015fef0 0x7f6a400afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400b4750 con 0x7f6a400b5a50 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400b45d0 con 0x7f6a40072e30 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400af9b0 con 0x7f6a4015fef0 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38304/0 (socket says 172.21.15.83:38304) 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 -- 172.21.15.83:0/1193463972 learned_addr learned my addr 172.21.15.83:0/1193463972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3ffff640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 0x7f6a401517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a44f37640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4015fef0 0x7f6a400afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 -- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4015fef0 msgr2=0x7f6a400afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4015fef0 0x7f6a400afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 -- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 msgr2=0x7f6a401517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 0x7f6a401517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 -- 172.21.15.83:0/1193463972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4016a830 con 0x7f6a400b5a50 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3ffff640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 0x7f6a401517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3f7fe640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c7f0 tx=0x7f6a3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34015070 con 0x7f6a400b5a50 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34007820 con 0x7f6a400b5a50 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a400b0c10 con 0x7f6a400b5a50 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34007c10 con 0x7f6a400b5a50 2024-05-17T22:08:10.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.758+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a40157cc0 con 0x7f6a400b5a50 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a40073240 con 0x7f6a400b5a50 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6a34018070 con 0x7f6a400b5a50 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a3cff9640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 0x7f6a2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a3ffff640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 0x7f6a2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f6a3408f1c0 con 0x7f6a400b5a50 2024-05-17T22:08:10.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.762+0000 7f6a3ffff640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 0x7f6a2c071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000a830 tx=0x7f6a30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:10.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.766+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f6a3405c000 con 0x7f6a400b5a50 2024-05-17T22:08:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a400d1a20 con 0x7f6a400b5a50 2024-05-17T22:08:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a3cff9640 1 -- 172.21.15.83:0/1193463972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5191 (secure 0 0 0) 0x7f6a3405b900 con 0x7f6a400b5a50 2024-05-17T22:08:10.940 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:10.940 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-17T22:07:12.387539+0000","modified":"2024-05-17T22:08:02.924755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8358":{"gid":8358,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":4,"addr":"172.21.15.83:6837/1541446976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1541446976},{"type":"v1","addr":"172.21.15.83:6837","nonce":1541446976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8358,"qdb_cluster":[8358]},"id":8}]} 2024-05-17T22:08:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 109 2024-05-17T22:08:10.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 msgr2=0x7f6a2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 0x7f6a2c071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000a830 tx=0x7f6a30008040 comp rx=0 tx=0).stop 2024-05-17T22:08:10.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 msgr2=0x7f6a400d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c7f0 tx=0x7f6a3400ccb0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 shutdown_connections 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a4015fef0 0x7f6a400afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a400b5a50 0x7f6a400d0fc0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f6a2c06f400 0x7f6a2c071910 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 --2- 172.21.15.83:0/1193463972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a40072e30 0x7f6a401517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.950+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 >> 172.21.15.83:0/1193463972 conn(0x7f6a4006cd90 msgr2=0x7f6a4015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.954+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 shutdown_connections 2024-05-17T22:08:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:10.954+0000 7f6a4699a640 1 -- 172.21.15.83:0/1193463972 wait complete. 2024-05-17T22:08:10.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:08:11.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.110+0000 7fc38431b640 1 Processor -- start 2024-05-17T22:08:11.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc38431b640 1 -- start start 2024-05-17T22:08:11.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc38431b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc37c0b2d20 0x7fc37c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc38431b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37c161620 con 0x7fc37c0c9c70 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc38431b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37c161890 con 0x7fc37c0728a0 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc38431b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c0736d0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3818b6640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc37c0c9c70 0x7fc37c0b2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34978/0 (socket says 172.21.15.83:34978) 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3818b6640 1 -- 172.21.15.83:0/4122311461 learned_addr learned my addr 172.21.15.83:0/4122311461 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3828b8640 1 --2- 172.21.15.83:0/4122311461 >> v2:172.21.15.155:3300/0 conn(0x7fc37c0b2d20 0x7fc37c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271530723 0 0) 0x7fc37c161620 con 0x7fc37c0c9c70 2024-05-17T22:08:11.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc368003530 con 0x7fc37c0c9c70 2024-05-17T22:08:11.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 537622969 0 0) 0x7fc368003530 con 0x7fc37c0c9c70 2024-05-17T22:08:11.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3828b8640 1 -- 172.21.15.83:0/4122311461 >> v1:172.21.15.155:6789/0 conn(0x7fc37c0728a0 legacy=0x7fc37c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3828b8640 1 -- 172.21.15.83:0/4122311461 >> v1:172.21.15.83:6789/0 conn(0x7fc37c0c9c70 legacy=0x7fc37c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.114+0000 7fc3828b8640 1 -- 172.21.15.83:0/4122311461 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c161b70 con 0x7fc37c0b2d20 2024-05-17T22:08:11.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc3828b8640 1 --2- 172.21.15.83:0/4122311461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15e9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc374009d90 tx=0x7fc37401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3c29be12c20230 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740b2070 con 0x7fc37c0b2d20 2024-05-17T22:08:11.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3740a55c0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc3810b5640 1 -- 172.21.15.83:0/4122311461 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c001700 con 0x7fc37c0b2d20 2024-05-17T22:08:11.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 -- 172.21.15.83:0/4122311461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0b2d20 msgr2=0x7fc37c15e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 --2- 172.21.15.83:0/4122311461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15e9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc374009d90 tx=0x7fc37401c890 comp rx=0 tx=0).stop 2024-05-17T22:08:11.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 -- 172.21.15.83:0/4122311461 shutdown_connections 2024-05-17T22:08:11.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 --2- 172.21.15.83:0/4122311461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15e9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 -- 172.21.15.83:0/4122311461 >> 172.21.15.83:0/4122311461 conn(0x7fc37c0b5250 msgr2=0x7fc37c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:11.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 -- 172.21.15.83:0/4122311461 shutdown_connections 2024-05-17T22:08:11.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 -- 172.21.15.83:0/4122311461 wait complete. 2024-05-17T22:08:11.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.118+0000 7fc38431b640 1 Processor -- start 2024-05-17T22:08:11.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- start start 2024-05-17T22:08:11.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 0x7fc37c14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37c0c9c70 0x7fc37c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c16b8d0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c16b750 con 0x7fc37c0728a0 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c16ba50 con 0x7fc37c0c9c70 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38310/0 (socket says 172.21.15.83:38310) 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 -- 172.21.15.83:0/3947041874 learned_addr learned my addr 172.21.15.83:0/3947041874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:11.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3820b7640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 0x7fc37c14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3828b8640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37c0c9c70 0x7fc37c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 -- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37c0c9c70 msgr2=0x7fc37c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37c0c9c70 0x7fc37c15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 -- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 msgr2=0x7fc37c14d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 0x7fc37c14d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 -- 172.21.15.83:0/3947041874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c14d830 con 0x7fc37c0b2d20 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3820b7640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 0x7fc37c14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc3818b6640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00c700 tx=0x7fc36c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c005ae0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c005ca0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0072b0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c159490 con 0x7fc37c0b2d20 2024-05-17T22:08:11.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.122+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c159a50 con 0x7fc37c0b2d20 2024-05-17T22:08:11.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c0b2610 con 0x7fc37c0b2d20 2024-05-17T22:08:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc36c007450 con 0x7fc37c0b2d20 2024-05-17T22:08:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc372ffd640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 0x7fc360071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc3820b7640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 0x7fc360071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7fc36c08f0c0 con 0x7fc37c0b2d20 2024-05-17T22:08:11.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.126+0000 7fc3820b7640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 0x7fc360071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f70 tx=0x7fc378009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.130+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc36c05bf70 con 0x7fc37c0b2d20 2024-05-17T22:08:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.282+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc37c0c8a40 con 0x7fc37c0b2d20 2024-05-17T22:08:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc372ffd640 1 -- 172.21.15.83:0/3947041874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16860 (secure 0 0 0) 0x7fc36c05b870 con 0x7fc37c0b2d20 2024-05-17T22:08:11.275 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:11.275 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":111,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:07:57.881192+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-17T22:07:09.949308+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-17T22:07:11.233706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.83:6835/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.155:6833/1193425174":"2024-05-18T22:07:40.363116+0000","172.21.15.155:6832/1193425174":"2024-05-18T22:07:40.363116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 msgr2=0x7fc360071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 0x7fc360071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f70 tx=0x7fc378009040 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 msgr2=0x7fc37c15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00c700 tx=0x7fc36c00cbc0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 shutdown_connections 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37c0c9c70 0x7fc37c15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37c0b2d20 0x7fc37c15c090 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc36006f400 0x7fc360071910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 --2- 172.21.15.83:0/3947041874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc37c0728a0 0x7fc37c14d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 >> 172.21.15.83:0/3947041874 conn(0x7fc37c0b5250 msgr2=0x7fc37c0d35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 shutdown_connections 2024-05-17T22:08:11.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.286+0000 7fc38431b640 1 -- 172.21.15.83:0/3947041874 wait complete. 2024-05-17T22:08:11.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-17T22:08:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.446+0000 7f21be9b9640 1 Processor -- start 2024-05-17T22:08:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 -- start start 2024-05-17T22:08:11.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b80b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 0x7f21b81664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b80b6400 con 0x7f21b80b0230 2024-05-17T22:08:11.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f21b80b66e0 con 0x7f21b80b3180 2024-05-17T22:08:11.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21be9b9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21b80b69c0 con 0x7f21b80b5e50 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21bcf56640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 0x7f21b81664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b80b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b77fe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f21b80b0230 0x7f21b80b0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.83:41802/0 (socket says 172.21.15.83:41802) 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b77fe640 1 -- 172.21.15.83:0/812140257 learned_addr learned my addr 172.21.15.83:0/812140257 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b7fff640 1 -- 172.21.15.83:0/812140257 >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 msgr2=0x7f21b81664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b7fff640 1 --2- 172.21.15.83:0/812140257 >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 0x7f21b81664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b7fff640 1 -- 172.21.15.83:0/812140257 >> v1:172.21.15.155:6789/0 conn(0x7f21b80b0230 legacy=0x7f21b80b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21b7fff640 1 -- 172.21.15.83:0/812140257 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b80b6ca0 con 0x7f21b80b5e50 2024-05-17T22:08:11.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.450+0000 7f21bcf56640 1 --2- 172.21.15.83:0/812140257 >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 0x7f21b81664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-17T22:08:11.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21b7fff640 1 --2- 172.21.15.83:0/812140257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b80b4910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097f0 tx=0x7f21a801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a8b119b566fb94 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21b6ffd640 1 -- 172.21.15.83:0/812140257 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80b0070 con 0x7f21b80b5e50 2024-05-17T22:08:11.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21b6ffd640 1 -- 172.21.15.83:0/812140257 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a80ab600 con 0x7f21b80b5e50 2024-05-17T22:08:11.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21b6ffd640 1 -- 172.21.15.83:0/812140257 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80ab9d0 con 0x7f21b80b5e50 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 -- 172.21.15.83:0/812140257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 msgr2=0x7f21b80b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 --2- 172.21.15.83:0/812140257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b80b4910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097f0 tx=0x7f21a801c3b0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 -- 172.21.15.83:0/812140257 shutdown_connections 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 --2- 172.21.15.83:0/812140257 >> v2:172.21.15.155:3300/0 conn(0x7f21b80b3180 0x7f21b81664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 --2- 172.21.15.83:0/812140257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b80b4910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 -- 172.21.15.83:0/812140257 >> 172.21.15.83:0/812140257 conn(0x7f21b809b8b0 msgr2=0x7f21b80b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:11.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 -- 172.21.15.83:0/812140257 shutdown_connections 2024-05-17T22:08:11.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.454+0000 7f21be9b9640 1 -- 172.21.15.83:0/812140257 wait complete. 2024-05-17T22:08:11.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 Processor -- start 2024-05-17T22:08:11.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 -- start start 2024-05-17T22:08:11.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21b80b0230 0x7f21b8150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b8157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21b81693d0 con 0x7f21b80b3180 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8169250 con 0x7f21b80b0230 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21be9b9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8169550 con 0x7f21b80b5e50 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38324/0 (socket says 172.21.15.83:38324) 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 -- 172.21.15.83:0/2545645055 learned_addr learned my addr 172.21.15.83:0/2545645055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21b80b0230 0x7f21b8150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21bcf56640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b8157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 -- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 msgr2=0x7f21b8157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b8157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 -- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21b80b0230 msgr2=0x7f21b8150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21b80b0230 0x7f21b8150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 -- 172.21.15.83:0/2545645055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b815ef70 con 0x7f21b80b3180 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b77fe640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c3c0 tx=0x7f21ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac017070 con 0x7f21b80b3180 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.458+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ac005d10 con 0x7f21b80b3180 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b815f250 con 0x7f21b80b3180 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac007310 con 0x7f21b80b3180 2024-05-17T22:08:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b815f810 con 0x7f21b80b3180 2024-05-17T22:08:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b80b4910 con 0x7f21b80b3180 2024-05-17T22:08:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f21ac01a070 con 0x7f21b80b3180 2024-05-17T22:08:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b4ff9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 0x7f21a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f21ac08e650 con 0x7f21b80b3180 2024-05-17T22:08:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 0x7f21a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.462+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 0x7f21a4071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f21b8069710 tx=0x7f21a80a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.466+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f21ac05b520 con 0x7f21b80b3180 2024-05-17T22:08:11.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.650+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f21b80b3d20 con 0x7f21b80b3180 2024-05-17T22:08:11.641 INFO:tasks.ceph.mon.a.smithi083.stderr:2024-05-17T22:08:11.650+0000 7fb427004640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-17T22:08:11.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.654+0000 7f21b4ff9640 1 -- 172.21.15.83:0/2545645055 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v110) ==== 161+0+0 (secure 0 0 0) 0x7f21ac05ae20 con 0x7f21b80b3180 2024-05-17T22:08:11.645 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 msgr2=0x7f21a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 0x7f21a4071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f21b8069710 tx=0x7f21a80a6000 comp rx=0 tx=0).stop 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 msgr2=0x7f21b813be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00c3c0 tx=0x7f21ac00c880 comp rx=0 tx=0).stop 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 shutdown_connections 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21b80b5e50 0x7f21b8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b80b3180 0x7f21b813be20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f21a406f400 0x7f21a4071910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 --2- 172.21.15.83:0/2545645055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21b80b0230 0x7f21b8150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 >> 172.21.15.83:0/2545645055 conn(0x7f21b809b8b0 msgr2=0x7f21b815c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 shutdown_connections 2024-05-17T22:08:11.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.658+0000 7f21be9b9640 1 -- 172.21.15.83:0/2545645055 wait complete. 2024-05-17T22:08:11.652 INFO:tasks.ceph.mds.c.smithi083.stderr: -1354> 2024-05-17T22:08:04.894+0000 7fd4a8336640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-17T22:08:11.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-17T22:08:11.709 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2024-05-17T22:08:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 Processor -- start 2024-05-17T22:08:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 -- start start 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac156dc0 0x7f9fac06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fac06c830 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fac06dd90 con 0x7f9fac0742b0 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fb281f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fac06db40 con 0x7f9fac156dc0 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fab7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9fac0ae7b0 0x7f9fac0b6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35004/0 (socket says 172.21.15.83:35004) 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fab7fe640 1 -- 172.21.15.83:0/1327045596 learned_addr learned my addr 172.21.15.83:0/1327045596 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac156dc0 0x7f9fac06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782741723 0 0) 0x7f9fac06c830 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f94003530 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2610762496 0 0) 0x7f9f94003530 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac156dc0 msgr2=0x7f9fac06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac156dc0 0x7f9fac06d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 >> v1:172.21.15.155:6789/0 conn(0x7f9fac0742b0 legacy=0x7f9fac0746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac06cad0 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9fa0001720 con 0x7f9fac0ae7b0 2024-05-17T22:08:11.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 >> v1:172.21.15.83:6789/0 conn(0x7f9fac0ae7b0 legacy=0x7f9fac0b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f94003a40 0x7f9f94003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94004810 0x7f9f94004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94005160 0x7f9f9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac06db40 con 0x7f9f94003a40 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9400bfc0 con 0x7f9f94005160 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.830+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9400c1c0 con 0x7f9f94004810 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f94003a40 0x7f9f94003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fab7fe640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94004810 0x7f9f94004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb0dbc640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94005160 0x7f9f9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 -- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94004810 msgr2=0x7f9f94004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f94004810 0x7f9f94004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 -- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94005160 msgr2=0x7f9f9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94005160 0x7f9f9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 -- 172.21.15.83:0/1327045596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f94003530 con 0x7f9f94003a40 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f94003a40 0x7f9f94003eb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ae0 tx=0x7f9f9c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a00a780da13a48 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0b2070 con 0x7f9f94003a40 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0a5570 con 0x7f9f94003a40 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9faaffd640 1 -- 172.21.15.83:0/1327045596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0096b0 con 0x7f9f94003a40 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 -- 172.21.15.83:0/1327045596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f94003a40 msgr2=0x7f9f94003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1327045596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f94003a40 0x7f9f94003eb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ae0 tx=0x7f9f9c01ca90 comp rx=0 tx=0).stop 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb0dbc640 1 -- 172.21.15.83:0/1327045596 reap_dead start 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 -- 172.21.15.83:0/1327045596 shutdown_connections 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 -- 172.21.15.83:0/1327045596 >> 172.21.15.83:0/1327045596 conn(0x7f9fac073790 msgr2=0x7f9fac073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 -- 172.21.15.83:0/1327045596 shutdown_connections 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.834+0000 7f9fb281f640 1 -- 172.21.15.83:0/1327045596 wait complete. 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 Processor -- start 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- start start 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94003a40 0x7f9fac151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac152540 0x7f9fac153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac0b0a20 con 0x7f9fac154140 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac0b08a0 con 0x7f9f94003a40 2024-05-17T22:08:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9fac0b0ba0 con 0x7f9fac152540 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38354/0 (socket says 172.21.15.83:38354) 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 -- 172.21.15.83:0/1763988925 learned_addr learned my addr 172.21.15.83:0/1763988925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94003a40 0x7f9fac151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fab7fe640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac152540 0x7f9fac153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 -- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac152540 msgr2=0x7f9fac153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac152540 0x7f9fac153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 -- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94003a40 msgr2=0x7f9fac151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94003a40 0x7f9fac151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 -- 172.21.15.83:0/1763988925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac154550 con 0x7f9fac154140 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb0dbc640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000f90 tx=0x7f9fa400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4003980 con 0x7f9fac154140 2024-05-17T22:08:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4018070 con 0x7f9fac154140 2024-05-17T22:08:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1591f0 con 0x7f9fac154140 2024-05-17T22:08:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4003da0 con 0x7f9fac154140 2024-05-17T22:08:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.838+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac15c3c0 con 0x7f9fac154140 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac0b6e30 con 0x7f9fac154140 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fa40050d0 con 0x7f9fac154140 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fa8ff9640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5967+0+0 (secure 0 0 0) 0x7f9fa408f280 con 0x7f9fac154140 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.842+0000 7f9fabfff640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0af000 tx=0x7f9f9c0046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:11.846+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9fac154550 con 0x7f9fac154140 2024-05-17T22:08:12.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.030+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9fac0b6060 con 0x7f9fac154140 2024-05-17T22:08:12.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.658+0000 7f9fa8ff9640 1 -- 172.21.15.83:0/1763988925 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v111) ==== 104+0+0 (secure 0 0 0) 0x7f9fa405c070 con 0x7f9fac154140 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 msgr2=0x7f9f98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0af000 tx=0x7f9f9c0046d0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 msgr2=0x7f9fac15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000f90 tx=0x7f9fa400ec10 comp rx=0 tx=0).stop 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 shutdown_connections 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fac154140 0x7f9fac15a760 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9fac152540 0x7f9fac153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 --2- 172.21.15.83:0/1763988925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f94003a40 0x7f9fac151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 >> 172.21.15.83:0/1763988925 conn(0x7f9fac073790 msgr2=0x7f9fac161260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 shutdown_connections 2024-05-17T22:08:12.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.662+0000 7f9fb281f640 1 -- 172.21.15.83:0/1763988925 wait complete. 2024-05-17T22:08:12.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-17T22:08:12.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.882+0000 7fc410c54640 1 Processor -- start 2024-05-17T22:08:12.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.882+0000 7fc410c54640 1 -- start start 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc410c54640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc40c15fd80 0x7fc40c161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc410c54640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c0d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc410c54640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0d3c10 con 0x7fc40c0d2280 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc410c54640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc40c0d11e0 con 0x7fc40c15fd80 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc410c54640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d3ec0 con 0x7fc40c0d49a0 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc40c15fd80 0x7fc40c161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc40c15fd80 0x7fc40c161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38368/0 (socket says 172.21.15.83:38368) 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 -- 172.21.15.83:0/2614114921 learned_addr learned my addr 172.21.15.83:0/2614114921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40ad9d640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c0d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 -- 172.21.15.83:0/2614114921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 msgr2=0x7fc40c0d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c0d4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 -- 172.21.15.83:0/2614114921 >> v1:172.21.15.155:6789/0 conn(0x7fc40c0d2280 legacy=0x7fc40c0d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:12.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 -- 172.21.15.83:0/2614114921 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c0d41a0 con 0x7fc40c15fd80 2024-05-17T22:08:12.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40a59c640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c161330 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009870 tx=0x7fc3fc01cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd7bacd04f65a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:12.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40959a640 1 -- 172.21.15.83:0/2614114921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0ab5f0 con 0x7fc40c15fd80 2024-05-17T22:08:12.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40959a640 1 -- 172.21.15.83:0/2614114921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc0b0070 con 0x7fc40c15fd80 2024-05-17T22:08:12.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.886+0000 7fc40959a640 1 -- 172.21.15.83:0/2614114921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0ab9b0 con 0x7fc40c15fd80 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- 172.21.15.83:0/2614114921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 msgr2=0x7fc40c161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c161330 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009870 tx=0x7fc3fc01cfc0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- 172.21.15.83:0/2614114921 shutdown_connections 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c0d4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- 172.21.15.83:0/2614114921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c161330 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- 172.21.15.83:0/2614114921 >> 172.21.15.83:0/2614114921 conn(0x7fc40c074ac0 msgr2=0x7fc40c06de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- 172.21.15.83:0/2614114921 shutdown_connections 2024-05-17T22:08:12.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- 172.21.15.83:0/2614114921 wait complete. 2024-05-17T22:08:12.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 Processor -- start 2024-05-17T22:08:12.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- start start 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc40c0d2280 0x7fc40c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0d0e40 con 0x7fc40c15fd80 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0d0cc0 con 0x7fc40c0d2280 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc410c54640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d0fc0 con 0x7fc40c0d49a0 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc40ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc40ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38378/0 (socket says 172.21.15.83:38378) 2024-05-17T22:08:12.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc40ad9d640 1 -- 172.21.15.83:0/3491460935 learned_addr learned my addr 172.21.15.83:0/3491460935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.890+0000 7fc409d9b640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40a59c640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc40c0d2280 0x7fc40c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 -- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 msgr2=0x7fc40c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c15eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 -- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc40c0d2280 msgr2=0x7fc40c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc40c0d2280 0x7fc40c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 -- 172.21.15.83:0/3491460935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c154ad0 con 0x7fc40c15fd80 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40ad9d640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc40400ccb0 tx=0x7fc40400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040039a0 con 0x7fc40c15fd80 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404003b60 con 0x7fc40c15fd80 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c0cba40 con 0x7fc40c15fd80 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404004020 con 0x7fc40c15fd80 2024-05-17T22:08:12.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c0cf7d0 con 0x7fc40c15fd80 2024-05-17T22:08:12.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c0d4db0 con 0x7fc40c15fd80 2024-05-17T22:08:12.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4040041c0 con 0x7fc40c15fd80 2024-05-17T22:08:12.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 0x7fc3ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:12.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40a59c640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 0x7fc3ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:12.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6025+0+0 (secure 0 0 0) 0x7fc40408f7f0 con 0x7fc40c15fd80 2024-05-17T22:08:12.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.894+0000 7fc40a59c640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 0x7fc3ec071910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0af000 tx=0x7fc3fc004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:12.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:12.898+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fc404014030 con 0x7fc40c15fd80 2024-05-17T22:08:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.062+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc40c0b9a60 con 0x7fc40c15fd80 2024-05-17T22:08:13.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.666+0000 7fc3faffd640 1 -- 172.21.15.83:0/3491460935 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) ==== 181+0+0 (secure 0 0 0) 0x7fc40405c630 con 0x7fc40c15fd80 2024-05-17T22:08:13.658 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-05-17T22:08:13.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 msgr2=0x7fc3ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:13.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 0x7fc3ec071910 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc0af000 tx=0x7fc3fc004740 comp rx=0 tx=0).stop 2024-05-17T22:08:13.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 msgr2=0x7fc40c15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:13.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc40400ccb0 tx=0x7fc40400a5c0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 shutdown_connections 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc40c15fd80 0x7fc40c15cb80 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc40c0d49a0 0x7fc40c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fc3ec06f400 0x7fc3ec071910 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 --2- 172.21.15.83:0/3491460935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc40c0d2280 0x7fc40c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 >> 172.21.15.83:0/3491460935 conn(0x7fc40c074ac0 msgr2=0x7fc40c15b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:13.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 shutdown_connections 2024-05-17T22:08:13.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.670+0000 7fc410c54640 1 -- 172.21.15.83:0/3491460935 wait complete. 2024-05-17T22:08:13.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-17T22:08:13.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.854+0000 7f8d9ec18640 1 Processor -- start 2024-05-17T22:08:13.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.854+0000 7f8d9ec18640 1 -- start start 2024-05-17T22:08:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9ec18640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d9809bd50 0x7f8d9815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9ec18640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9806cea0 0x7f8d980cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9ec18640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d980b4910 con 0x7f8d980af470 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9ec18640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8d980b4bd0 con 0x7f8d9809bd50 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9ec18640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8d980b4eb0 con 0x7f8d9806cea0 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d9809bd50 0x7f8d9815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d9809bd50 0x7f8d9815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38386/0 (socket says 172.21.15.83:38386) 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/3831410828 learned_addr learned my addr 172.21.15.83:0/3831410828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9d1b5640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9806cea0 0x7f8d980cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/3831410828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9806cea0 msgr2=0x7f8d980cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:13.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9806cea0 0x7f8d980cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/3831410828 >> v1:172.21.15.155:6789/0 conn(0x7f8d980af470 legacy=0x7f8d9806c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:13.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.858+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/3831410828 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d980cef70 con 0x7f8d9809bd50 2024-05-17T22:08:13.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9c9b4640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d9815da40 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009870 tx=0x7f8d8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93da02934c591fb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:13.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d977fe640 1 -- 172.21.15.83:0/3831410828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880b2070 con 0x7f8d9809bd50 2024-05-17T22:08:13.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d977fe640 1 -- 172.21.15.83:0/3831410828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d880a9070 con 0x7f8d9809bd50 2024-05-17T22:08:13.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d977fe640 1 -- 172.21.15.83:0/3831410828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880ad880 con 0x7f8d9809bd50 2024-05-17T22:08:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- 172.21.15.83:0/3831410828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 msgr2=0x7f8d9815da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d9815da40 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009870 tx=0x7f8d8801c4d0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- 172.21.15.83:0/3831410828 shutdown_connections 2024-05-17T22:08:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9806cea0 0x7f8d980cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/3831410828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d9815da40 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- 172.21.15.83:0/3831410828 >> 172.21.15.83:0/3831410828 conn(0x7f8d9809b8b0 msgr2=0x7f8d980b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- 172.21.15.83:0/3831410828 shutdown_connections 2024-05-17T22:08:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- 172.21.15.83:0/3831410828 wait complete. 2024-05-17T22:08:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 Processor -- start 2024-05-17T22:08:13.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.862+0000 7f8d9ec18640 1 -- start start 2024-05-17T22:08:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 0x7f8d9815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9813f920 0x7f8d9815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d980d0570 con 0x7f8d9809bd50 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8d980d03f0 con 0x7f8d980af470 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8d980d06f0 con 0x7f8d9813f920 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38396/0 (socket says 172.21.15.83:38396) 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/2426780536 learned_addr learned my addr 172.21.15.83:0/2426780536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9d1b5640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9813f920 0x7f8d9815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d97fff640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 0x7f8d9815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9813f920 msgr2=0x7f8d9815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9813f920 0x7f8d9815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 msgr2=0x7f8d9815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 0x7f8d9815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 -- 172.21.15.83:0/2426780536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d980d09c0 con 0x7f8d9809bd50 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d97fff640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 0x7f8d9815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:13.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9c9b4640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8d880b0040 tx=0x7f8d88019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880bc070 con 0x7f8d9809bd50 2024-05-17T22:08:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d880b6640 con 0x7f8d9809bd50 2024-05-17T22:08:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d981506b0 con 0x7f8d9809bd50 2024-05-17T22:08:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d880b2040 con 0x7f8d9809bd50 2024-05-17T22:08:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d981f8c90 con 0x7f8d9809bd50 2024-05-17T22:08:13.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d980cf490 con 0x7f8d9809bd50 2024-05-17T22:08:13.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d880bf070 con 0x7f8d9809bd50 2024-05-17T22:08:13.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.866+0000 7f8d957fa640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 0x7f8d84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:13.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.870+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5587+0+0 (secure 0 0 0) 0x7f8d88128790 con 0x7f8d9809bd50 2024-05-17T22:08:13.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.870+0000 7f8d97fff640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 0x7f8d84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.870+0000 7f8d97fff640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 0x7f8d84071910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d9813fe20 tx=0x7f8d8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:13.874+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8d88016150 con 0x7f8d9809bd50 2024-05-17T22:08:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.030+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d98150fa0 con 0x7f8d9809bd50 2024-05-17T22:08:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.686+0000 7f8d957fa640 1 -- 172.21.15.83:0/2426780536 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) ==== 169+0+0 (secure 0 0 0) 0x7f8d880f5780 con 0x7f8d9809bd50 2024-05-17T22:08:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 msgr2=0x7f8d84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 0x7f8d84071910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d9813fe20 tx=0x7f8d8c008040 comp rx=0 tx=0).stop 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 msgr2=0x7f8d98150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f8d880b0040 tx=0x7f8d88019270 comp rx=0 tx=0).stop 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 shutdown_connections 2024-05-17T22:08:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9813f920 0x7f8d9815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8d8406f400 0x7f8d84071910 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d980af470 0x7f8d9815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 --2- 172.21.15.83:0/2426780536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d9809bd50 0x7f8d98150170 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 >> 172.21.15.83:0/2426780536 conn(0x7f8d9809b8b0 msgr2=0x7f8d9815a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 shutdown_connections 2024-05-17T22:08:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.690+0000 7f8d9ec18640 1 -- 172.21.15.83:0/2426780536 wait complete. 2024-05-17T22:08:14.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-17T22:08:14.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 Processor -- start 2024-05-17T22:08:14.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 -- start start 2024-05-17T22:08:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3fc15d700 0x7fe3fc1673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3fc0b5640 con 0x7fe3fc0cf830 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3fc0b58e0 con 0x7fe3fc0cdf40 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe404663640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe3fc070870 con 0x7fe3fc15d700 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe3fc0cf830 0x7fe3fc0b3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35012/0 (socket says 172.21.15.83:35012) 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 -- 172.21.15.83:0/836173721 learned_addr learned my addr 172.21.15.83:0/836173721 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe402c00640 1 --2- 172.21.15.83:0/836173721 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc15d700 0x7fe3fc1673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409314135 0 0) 0x7fe3fc0b5640 con 0x7fe3fc0cf830 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e8003530 con 0x7fe3fc0cf830 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3631671041 0 0) 0x7fe3e8003530 con 0x7fe3fc0cf830 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc15d700 msgr2=0x7fe3fc1673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 --2- 172.21.15.83:0/836173721 >> v2:172.21.15.155:3300/0 conn(0x7fe3fc15d700 0x7fe3fc1673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 >> v1:172.21.15.155:6789/0 conn(0x7fe3fc0cdf40 legacy=0x7fe3fc15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc06f060 con 0x7fe3fc0cf830 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fe3f8001720 con 0x7fe3fc0cf830 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 >> v1:172.21.15.83:6789/0 conn(0x7fe3fc0cf830 legacy=0x7fe3fc0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3e8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e8004810 0x7fe3e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e8005310 0x7fe3e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e800c280 con 0x7fe3e8003a60 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e800c430 con 0x7fe3e8005310 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e800c630 con 0x7fe3e8004810 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3e8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe401bfe640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e8004810 0x7fe3e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe402c00640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e8005310 0x7fe3e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 -- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e8004810 msgr2=0x7fe3e8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3e8004810 0x7fe3e8004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 -- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e8005310 msgr2=0x7fe3e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e8005310 0x7fe3e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe4023ff640 1 -- 172.21.15.83:0/836173721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e8003530 con 0x7fe3e8003a60 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.878+0000 7fe402c00640 1 -- 172.21.15.83:0/836173721 reap_dead start 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe4023ff640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3e8003ed0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0b5810 tx=0x7fe3f801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc90c791b7fc86e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007d20 con 0x7fe3e8003a60 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f80a9020 con 0x7fe3e8003a60 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe4013fd640 1 -- 172.21.15.83:0/836173721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8005b20 con 0x7fe3e8003a60 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 -- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 msgr2=0x7fe3e8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3e8003ed0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0b5810 tx=0x7fe3f801cb80 comp rx=0 tx=0).stop 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 -- 172.21.15.83:0/836173721 shutdown_connections 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 --2- 172.21.15.83:0/836173721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3e8003ed0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 -- 172.21.15.83:0/836173721 >> 172.21.15.83:0/836173721 conn(0x7fe3fc06cd90 msgr2=0x7fe3fc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 -- 172.21.15.83:0/836173721 shutdown_connections 2024-05-17T22:08:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.882+0000 7fe404663640 1 -- 172.21.15.83:0/836173721 wait complete. 2024-05-17T22:08:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 Processor -- start 2024-05-17T22:08:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- start start 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3fc0aad90 0x7fe3fc0ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc159240 0x7fe3fc159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc06f9c0 con 0x7fe3e8003a60 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc06f840 con 0x7fe3fc159240 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc06fb40 con 0x7fe3fc0aad90 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38412/0 (socket says 172.21.15.83:38412) 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 -- 172.21.15.83:0/2290886008 learned_addr learned my addr 172.21.15.83:0/2290886008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe402c00640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc159240 0x7fe3fc159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe401bfe640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3fc0aad90 0x7fe3fc0ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 -- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3fc0aad90 msgr2=0x7fe3fc0ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3fc0aad90 0x7fe3fc0ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 -- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc159240 msgr2=0x7fe3fc159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc159240 0x7fe3fc159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 -- 172.21.15.83:0/2290886008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc159b90 con 0x7fe3e8003a60 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe4023ff640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801c640 tx=0x7fe3e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80049f0 con 0x7fe3e8003a60 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f8004b90 con 0x7fe3e8003a60 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc156d60 con 0x7fe3e8003a60 2024-05-17T22:08:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80070d0 con 0x7fe3e8003a60 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc0ac0c0 con 0x7fe3e8003a60 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.886+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc06fed0 con 0x7fe3e8003a60 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.890+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe3f80bd060 con 0x7fe3e8003a60 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.890+0000 7fe3f2ffd640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 0x7fe3e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.890+0000 7fe401bfe640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 0x7fe3e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.890+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fe3f8128110 con 0x7fe3e8003a60 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.890+0000 7fe401bfe640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 0x7fe3e0071910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007fd0 tx=0x7fe3ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:14.894+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe3fc159b90 con 0x7fe3e8003a60 2024-05-17T22:08:15.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.054+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe3fc0adb80 con 0x7fe3e8003a60 2024-05-17T22:08:15.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe3f2ffd640 1 -- 172.21.15.83:0/2290886008 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fe3f8016930 con 0x7fe3e8003a60 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 msgr2=0x7fe3e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 0x7fe3e0071910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007fd0 tx=0x7fe3ec009040 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 msgr2=0x7fe3fc14c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801c640 tx=0x7fe3e8005840 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 shutdown_connections 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe3e006f400 0x7fe3e0071910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3fc159240 0x7fe3fc159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3fc0aad90 0x7fe3fc0ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 --2- 172.21.15.83:0/2290886008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8003a60 0x7fe3fc14c1c0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 >> 172.21.15.83:0/2290886008 conn(0x7fe3fc06cd90 msgr2=0x7fe3fc0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 shutdown_connections 2024-05-17T22:08:15.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.062+0000 7fe404663640 1 -- 172.21.15.83:0/2290886008 wait complete. 2024-05-17T22:08:15.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-17T22:08:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 Processor -- start 2024-05-17T22:08:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 -- start start 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa99806e480 0x7fa99815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9980d44a0 0x7fa9981698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9980d55e0 con 0x7fa9980d0370 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa9980b8d00 con 0x7fa99806e480 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa99cc2f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fa9980d4a60 con 0x7fa9980d44a0 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa99806e480 0x7fa99815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa99806e480 0x7fa99815fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38416/0 (socket says 172.21.15.83:38416) 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 -- 172.21.15.83:0/242958190 learned_addr learned my addr 172.21.15.83:0/242958190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa996d9d640 1 --2- 172.21.15.83:0/242958190 >> v2:172.21.15.155:3300/0 conn(0x7fa9980d44a0 0x7fa9981698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 -- 172.21.15.83:0/242958190 >> v2:172.21.15.155:3300/0 conn(0x7fa9980d44a0 msgr2=0x7fa9981698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 --2- 172.21.15.83:0/242958190 >> v2:172.21.15.155:3300/0 conn(0x7fa9980d44a0 0x7fa9981698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 -- 172.21.15.83:0/242958190 >> v1:172.21.15.155:6789/0 conn(0x7fa9980d0370 legacy=0x7fa99806dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.238+0000 7fa995d9b640 1 -- 172.21.15.83:0/242958190 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9980d4d40 con 0x7fa99806e480 2024-05-17T22:08:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa995d9b640 1 --2- 172.21.15.83:0/242958190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa99806e480 0x7fa99815fd00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa9980d43f0 tx=0x7fa98001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e5c3f087b065c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99559a640 1 -- 172.21.15.83:0/242958190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800b0070 con 0x7fa99806e480 2024-05-17T22:08:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99559a640 1 -- 172.21.15.83:0/242958190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9800ab600 con 0x7fa99806e480 2024-05-17T22:08:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99559a640 1 -- 172.21.15.83:0/242958190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800ab9d0 con 0x7fa99806e480 2024-05-17T22:08:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 -- 172.21.15.83:0/242958190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa99806e480 msgr2=0x7fa99815fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/242958190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa99806e480 0x7fa99815fd00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa9980d43f0 tx=0x7fa98001c520 comp rx=0 tx=0).stop 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 -- 172.21.15.83:0/242958190 shutdown_connections 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/242958190 >> v2:172.21.15.155:3300/0 conn(0x7fa9980d44a0 0x7fa9981698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/242958190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa99806e480 0x7fa99815fd00 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 -- 172.21.15.83:0/242958190 >> 172.21.15.83:0/242958190 conn(0x7fa998074a40 msgr2=0x7fa9980d5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 -- 172.21.15.83:0/242958190 shutdown_connections 2024-05-17T22:08:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.242+0000 7fa99cc2f640 1 -- 172.21.15.83:0/242958190 wait complete. 2024-05-17T22:08:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 Processor -- start 2024-05-17T22:08:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- start start 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9980d44a0 0x7fa998150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa998162900 0x7fa998162d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980bb300 con 0x7fa9980d0370 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980bb180 con 0x7fa9980d44a0 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fa99816c650 con 0x7fa998162900 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38424/0 (socket says 172.21.15.83:38424) 2024-05-17T22:08:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 -- 172.21.15.83:0/1983547882 learned_addr learned my addr 172.21.15.83:0/1983547882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa995d9b640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9980d44a0 0x7fa998150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa996d9d640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa998162900 0x7fa998162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 -- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa998162900 msgr2=0x7fa998162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa998162900 0x7fa998162d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 -- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9980d44a0 msgr2=0x7fa998150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9980d44a0 0x7fa998150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 -- 172.21.15.83:0/1983547882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa998163250 con 0x7fa9980d0370 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99659c640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c730 tx=0x7fa98800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988003b80 con 0x7fa9980d0370 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa988003d40 con 0x7fa9980d0370 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99815a910 con 0x7fa9980d0370 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98800aa90 con 0x7fa9980d0370 2024-05-17T22:08:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9981618e0 con 0x7fa9980d0370 2024-05-17T22:08:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.246+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99815fd00 con 0x7fa9980d0370 2024-05-17T22:08:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.250+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa98800ac30 con 0x7fa9980d0370 2024-05-17T22:08:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.250+0000 7fa98effd640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 0x7fa978071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.250+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fa98808f8e0 con 0x7fa9980d0370 2024-05-17T22:08:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.250+0000 7fa995d9b640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 0x7fa978071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.250+0000 7fa995d9b640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 0x7fa978071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9800ad000 tx=0x7fa9800a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:15.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.254+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fa998163250 con 0x7fa9980d0370 2024-05-17T22:08:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.466+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fa9980b60a0 con 0x7fa9980d0370 2024-05-17T22:08:15.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.690+0000 7fa98effd640 1 -- 172.21.15.83:0/1983547882 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v482) ==== 144+0+0 (secure 0 0 0) 0x7fa98805cb90 con 0x7fa9980d0370 2024-05-17T22:08:15.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 msgr2=0x7fa978071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 0x7fa978071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa9800ad000 tx=0x7fa9800a6000 comp rx=0 tx=0).stop 2024-05-17T22:08:15.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 msgr2=0x7fa99815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c730 tx=0x7fa98800cbf0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 shutdown_connections 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa998162900 0x7fa998162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fa97806f430 0x7fa978071940 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9980d44a0 0x7fa998150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 --2- 172.21.15.83:0/1983547882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9980d0370 0x7fa99815a3d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 >> 172.21.15.83:0/1983547882 conn(0x7fa998074a40 msgr2=0x7fa9980baa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 shutdown_connections 2024-05-17T22:08:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:15.694+0000 7fa99cc2f640 1 -- 172.21.15.83:0/1983547882 wait complete. 2024-05-17T22:08:15.709 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-17T22:08:15.709 INFO:tasks.cephfs_test_runner: 2024-05-17T22:08:15.709 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-17T22:08:15.709 INFO:tasks.cephfs_test_runner:Ran 7 tests in 618.271s 2024-05-17T22:08:15.710 INFO:tasks.cephfs_test_runner: 2024-05-17T22:08:15.711 INFO:tasks.cephfs_test_runner:OK 2024-05-17T22:08:15.711 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-17T22:08:15.720 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-17T22:08:15.761 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-17T22:08:15.761 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:08:15.761 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:15.768 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:08:15.768 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-05-17T22:08:15.768 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:15.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336488k,nr_inodes=4084122,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20938 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-17T22:08:15.815 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:08:15.815 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:15.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:08:15.822 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-17T22:08:15.822 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-17T22:08:15.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336496k,nr_inodes=4084124,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16626 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-17T22:08:15.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:15.871 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-05-17T22:08:15.877 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-17T22:08:15.877 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-17T22:08:15.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:15.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-17T22:08:15.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:15.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-17T22:08:16.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:16.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-05-17T22:08:16.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:16.156 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2024-05-17T22:08:16.163 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-17T22:08:16.163 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-17T22:08:16.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:16.164 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-17T22:08:16.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:16.219 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-17T22:08:16.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-17T22:08:16.272 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2024-05-17T22:08:16.439 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-17T22:08:16.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-17T22:08:16.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:08:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.618+0000 7f18557ec640 1 Processor -- start 2024-05-17T22:08:16.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.618+0000 7f18557ec640 1 -- start start 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f18557ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1850072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f18557ec640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1850073780 con 0x7f1850073370 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f18557ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185015e790 con 0x7f18500d1040 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f18557ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f185015ea40 con 0x7f1850144080 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1850073370 0x7f185006f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:35016/0 (socket says 172.21.15.83:35016) 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 -- 172.21.15.83:0/3480351045 learned_addr learned my addr 172.21.15.83:0/3480351045 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:16.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184e7fc640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1850072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465628071 0 0) 0x7f18480014a0 con 0x7f1850073370 2024-05-17T22:08:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1838003530 con 0x7f1850073370 2024-05-17T22:08:16.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 19455877 0 0) 0x7f1838003530 con 0x7f1850073370 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 msgr2=0x7f1850072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1850072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 >> v1:172.21.15.155:6789/0 conn(0x7f18500d1040 legacy=0x7f185015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185015ed20 con 0x7f1850073370 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f1848001c50 con 0x7f1850073370 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 >> v1:172.21.15.83:6789/0 conn(0x7f1850073370 legacy=0x7f185006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1838003a40 0x7f1838003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1838004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1838004d50 0x7f1838005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f185015ea40 con 0x7f1838004d50 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f183800bb90 con 0x7f1838003a40 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f183800bd90 con 0x7f1850144080 2024-05-17T22:08:16.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1838004d50 0x7f1838005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184effd640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1838003a40 0x7f1838003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184e7fc640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1838004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 -- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 msgr2=0x7f1838004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1850144080 0x7f1838004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 -- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1838003a40 msgr2=0x7f1838003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1838003a40 0x7f1838003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.622+0000 7f184f7fe640 1 -- 172.21.15.83:0/3480351045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1838003530 con 0x7f1838004d50 2024-05-17T22:08:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1838004d50 0x7f1838005200 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f184800aa00 tx=0x7f184801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba3bc79c6c74745 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800a620 con 0x7f1838004d50 2024-05-17T22:08:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18480ad650 con 0x7f1838004d50 2024-05-17T22:08:16.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f184dffb640 1 -- 172.21.15.83:0/3480351045 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480ada40 con 0x7f1838004d50 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 -- 172.21.15.83:0/3480351045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1838004d50 msgr2=0x7f1838005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 --2- 172.21.15.83:0/3480351045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1838004d50 0x7f1838005200 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f184800aa00 tx=0x7f184801d2a0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f184f7fe640 1 -- 172.21.15.83:0/3480351045 reap_dead start 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 -- 172.21.15.83:0/3480351045 shutdown_connections 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 -- 172.21.15.83:0/3480351045 >> 172.21.15.83:0/3480351045 conn(0x7f185006cd90 msgr2=0x7f18500b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 -- 172.21.15.83:0/3480351045 shutdown_connections 2024-05-17T22:08:16.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.626+0000 7f18557ec640 1 -- 172.21.15.83:0/3480351045 wait complete. 2024-05-17T22:08:16.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 Processor -- start 2024-05-17T22:08:16.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- start start 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 0x7f185015d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f185015d6d0 0x7f185015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f185006f210 con 0x7f185015f780 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f185006ef30 con 0x7f185015d6d0 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f1850144490 con 0x7f185013fa20 2024-05-17T22:08:16.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38438/0 (socket says 172.21.15.83:38438) 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 -- 172.21.15.83:0/3736268462 learned_addr learned my addr 172.21.15.83:0/3736268462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184effd640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 0x7f185015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184e7fc640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f185015d6d0 0x7f185015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 -- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 msgr2=0x7f185015d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 0x7f185015d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 -- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f185015d6d0 msgr2=0x7f185015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f185015d6d0 0x7f185015f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 -- 172.21.15.83:0/3736268462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850140c70 con 0x7f185015f780 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184effd640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 0x7f185015d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f184f7fe640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f184801db40 tx=0x7f1838005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480b2040 con 0x7f185015f780 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18480014a0 con 0x7f185015f780 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800a620 con 0x7f185015f780 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1850140f10 con 0x7f185015f780 2024-05-17T22:08:16.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501414d0 con 0x7f185015f780 2024-05-17T22:08:16.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.630+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850072e30 con 0x7f185015f780 2024-05-17T22:08:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.634+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18480ad650 con 0x7f185015f780 2024-05-17T22:08:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.634+0000 7f18377fe640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 0x7f183c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.634+0000 7f184effd640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 0x7f183c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.634+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f1848128c00 con 0x7f185015f780 2024-05-17T22:08:16.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.634+0000 7f184effd640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 0x7f183c071910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f185013f250 tx=0x7f1840009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.638+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f18480be030 con 0x7f185015f780 2024-05-17T22:08:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.798+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f185015dae0 con 0x7f183c06f400 2024-05-17T22:08:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.798+0000 7f18377fe640 1 -- 172.21.15.83:0/3736268462 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f185015dae0 con 0x7f183c06f400 2024-05-17T22:08:16.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:08:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 msgr2=0x7f183c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 0x7f183c071910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f185013f250 tx=0x7f1840009040 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 msgr2=0x7f185015fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f184801db40 tx=0x7f1838005840 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 shutdown_connections 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f185015f780 0x7f185015fb90 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f183c06f400 0x7f183c071910 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f185015d6d0 0x7f185015f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 --2- 172.21.15.83:0/3736268462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185013fa20 0x7f185015d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 >> 172.21.15.83:0/3736268462 conn(0x7f185006cd90 msgr2=0x7f18500cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 shutdown_connections 2024-05-17T22:08:16.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.802+0000 7f18557ec640 1 -- 172.21.15.83:0/3736268462 wait complete. 2024-05-17T22:08:16.812 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-05-17T22:08:15.205519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278096,"kb_used_data":28632,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696624,"statfs":{"total":773094113280,"available":772809342976,"internally_reserved":0,"allocated":29319168,"data_stored":22871610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001175"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":283,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-17T22:08:13.799762+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:08:13.799762+0000","last_peered":"2024-05-17T22:08:13.799762+0000","last_clean":"2024-05-17T22:08:13.799762+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:08:13.799762+0000","last_undegraded":"2024-05-17T22:08:13.799762+0000","last_fullsized":"2024-05-17T22:08:13.799762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1208391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":612398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":618023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5619712,"data_stored":4809114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":611787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":6217728,"data_stored":5399524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:08:16.812 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:08:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.970+0000 7f93bb820640 1 Processor -- start 2024-05-17T22:08:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 -- start start 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b415fe20 0x7f93b4160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f93b4072e30 0x7f93b4073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b4073780 con 0x7f93b415d2f0 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f93b415c830 con 0x7f93b4072e30 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93bb820640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93b41608a0 con 0x7f93b415fe20 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f93b4072e30 0x7f93b4073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b95bc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f93b415d2f0 0x7f93b4159c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46170/0 (socket says 172.21.15.83:46170) 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b95bc640 1 -- 172.21.15.83:0/2970691248 learned_addr learned my addr 172.21.15.83:0/2970691248 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f93b4072e30 0x7f93b4073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53494/0 (socket says 172.21.15.83:53494) 2024-05-17T22:08:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b8dbb640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b415fe20 0x7f93b4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 -- 172.21.15.83:0/2970691248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b415fe20 msgr2=0x7f93b4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:16.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b415fe20 0x7f93b4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 -- 172.21.15.83:0/2970691248 >> v1:172.21.15.83:6789/0 conn(0x7f93b415d2f0 legacy=0x7f93b4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.974+0000 7f93b9dbd640 1 -- 172.21.15.83:0/2970691248 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4160b80 con 0x7f93b4072e30 2024-05-17T22:08:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93b9dbd640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b4072e30 0x7f93b4073240 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f93b000ba60 tx=0x7f93b001eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d641a5bd86cd4d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93abfff640 1 -- 172.21.15.83:0/2970691248 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b4070 con 0x7f93b4072e30 2024-05-17T22:08:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93abfff640 1 -- 172.21.15.83:0/2970691248 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b00a7a40 con 0x7f93b4072e30 2024-05-17T22:08:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93abfff640 1 -- 172.21.15.83:0/2970691248 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00a7e10 con 0x7f93b4072e30 2024-05-17T22:08:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- 172.21.15.83:0/2970691248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b4072e30 msgr2=0x7f93b4073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b4072e30 0x7f93b4073240 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f93b000ba60 tx=0x7f93b001eae0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- 172.21.15.83:0/2970691248 shutdown_connections 2024-05-17T22:08:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b4072e30 0x7f93b4073240 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 --2- 172.21.15.83:0/2970691248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b415fe20 0x7f93b4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- 172.21.15.83:0/2970691248 >> 172.21.15.83:0/2970691248 conn(0x7f93b406cd90 msgr2=0x7f93b40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- 172.21.15.83:0/2970691248 shutdown_connections 2024-05-17T22:08:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- 172.21.15.83:0/2970691248 wait complete. 2024-05-17T22:08:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 Processor -- start 2024-05-17T22:08:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.978+0000 7f93bb820640 1 -- start start 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b4072e30 0x7f93b40b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 0x7f93b40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40b04f0 con 0x7f93b415fe20 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40b0370 con 0x7f93b415d2f0 2024-05-17T22:08:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f93b40b0670 con 0x7f93b4072e30 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53500/0 (socket says 172.21.15.83:53500) 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 -- 172.21.15.83:0/591302792 learned_addr learned my addr 172.21.15.83:0/591302792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b95bc640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b4072e30 0x7f93b40b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b8dbb640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 0x7f93b40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 -- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b4072e30 msgr2=0x7f93b40b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b4072e30 0x7f93b40b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 -- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 msgr2=0x7f93b40ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 0x7f93b40ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 -- 172.21.15.83:0/591302792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b40b1a00 con 0x7f93b415fe20 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b8dbb640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 0x7f93b40ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93b9dbd640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f93b000ba60 tx=0x7f93b001b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b4040 con 0x7f93b415fe20 2024-05-17T22:08:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b00b8420 con 0x7f93b415fe20 2024-05-17T22:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b87f0 con 0x7f93b415fe20 2024-05-17T22:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b4152bf0 con 0x7f93b415fe20 2024-05-17T22:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b40b1ce0 con 0x7f93b415fe20 2024-05-17T22:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.982+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b40b1770 con 0x7f93b415fe20 2024-05-17T22:08:16.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.986+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f93b00b8990 con 0x7f93b415fe20 2024-05-17T22:08:16.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.986+0000 7f93a9ffb640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 0x7f9398071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:16.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.986+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f93b0129ba0 con 0x7f93b415fe20 2024-05-17T22:08:16.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.986+0000 7f93b95bc640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 0x7f9398071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.986+0000 7f93b95bc640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 0x7f9398071910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0012f0 tx=0x7f93ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:16.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:16.990+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f93b0018760 con 0x7f93b415fe20 2024-05-17T22:08:17.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.150+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93b4073240 con 0x7f939806f400 2024-05-17T22:08:17.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.150+0000 7f93a9ffb640 1 -- 172.21.15.83:0/591302792 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f93b4073240 con 0x7f939806f400 2024-05-17T22:08:17.141 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:17.142 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:08:17.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 msgr2=0x7f9398071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 0x7f9398071910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0012f0 tx=0x7f93ac009040 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 msgr2=0x7f93b41536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f93b000ba60 tx=0x7f93b001b880 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 shutdown_connections 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93b415fe20 0x7f93b41536d0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f939806f400 0x7f9398071910 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b415d2f0 0x7f93b40ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 --2- 172.21.15.83:0/591302792 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93b4072e30 0x7f93b40b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 >> 172.21.15.83:0/591302792 conn(0x7f93b406cd90 msgr2=0x7f93b40d1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 shutdown_connections 2024-05-17T22:08:17.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.154+0000 7f93bb820640 1 -- 172.21.15.83:0/591302792 wait complete. 2024-05-17T22:08:17.164 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-05-17T22:08:15.205519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278096,"kb_used_data":28632,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696624,"statfs":{"total":773094113280,"available":772809342976,"internally_reserved":0,"allocated":29319168,"data_stored":22871610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001175"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":283,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-17T22:08:13.799762+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:08:13.799762+0000","last_peered":"2024-05-17T22:08:13.799762+0000","last_clean":"2024-05-17T22:08:13.799762+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:08:13.799762+0000","last_undegraded":"2024-05-17T22:08:13.799762+0000","last_fullsized":"2024-05-17T22:08:13.799762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1208391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":612398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":618023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5619712,"data_stored":4809114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":611787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":6217728,"data_stored":5399524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:08:17.165 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-17T22:08:17.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:08:17.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.318+0000 7ff41be3a640 1 Processor -- start 2024-05-17T22:08:17.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 -- start start 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4141461c0 0x7ff41415e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4140706e0 0x7ff4141605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4141465d0 con 0x7ff41415e6b0 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff41406ef30 con 0x7ff4141461c0 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41be3a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff41406f210 con 0x7ff4140706e0 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4141461c0 0x7ff41415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff4141461c0 0x7ff41415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53516/0 (socket says 172.21.15.83:53516) 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 -- 172.21.15.83:0/3094714108 learned_addr learned my addr 172.21.15.83:0/3094714108 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff41a3d7640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4140706e0 0x7ff4141605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 -- 172.21.15.83:0/3094714108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4140706e0 msgr2=0x7ff4141605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4140706e0 0x7ff4141605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 -- 172.21.15.83:0/3094714108 >> v1:172.21.15.155:6789/0 conn(0x7ff41415e6b0 legacy=0x7ff41406ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.322+0000 7ff419bd6640 1 -- 172.21.15.83:0/3094714108 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41406f4f0 con 0x7ff4141461c0 2024-05-17T22:08:17.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff419bd6640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4141461c0 0x7ff41415e170 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7ff410009870 tx=0x7ff41001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef4bb9a4ebefda0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff418bd4640 1 -- 172.21.15.83:0/3094714108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4100ab640 con 0x7ff4141461c0 2024-05-17T22:08:17.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff418bd4640 1 -- 172.21.15.83:0/3094714108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4100b0070 con 0x7ff4141461c0 2024-05-17T22:08:17.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff418bd4640 1 -- 172.21.15.83:0/3094714108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4100aba00 con 0x7ff4141461c0 2024-05-17T22:08:17.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- 172.21.15.83:0/3094714108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4141461c0 msgr2=0x7ff41415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4141461c0 0x7ff41415e170 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7ff410009870 tx=0x7ff41001c530 comp rx=0 tx=0).stop 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- 172.21.15.83:0/3094714108 shutdown_connections 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4140706e0 0x7ff4141605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 --2- 172.21.15.83:0/3094714108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4141461c0 0x7ff41415e170 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- 172.21.15.83:0/3094714108 >> 172.21.15.83:0/3094714108 conn(0x7ff41406cd90 msgr2=0x7ff4140b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- 172.21.15.83:0/3094714108 shutdown_connections 2024-05-17T22:08:17.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- 172.21.15.83:0/3094714108 wait complete. 2024-05-17T22:08:17.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 Processor -- start 2024-05-17T22:08:17.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.326+0000 7ff41be3a640 1 -- start start 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4141461c0 0x7ff41413ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 0x7ff41413d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff414167930 con 0x7ff4140706e0 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff4140d12b0 con 0x7ff41415e6b0 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7ff414167ab0 con 0x7ff4141461c0 2024-05-17T22:08:17.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53532/0 (socket says 172.21.15.83:53532) 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 -- 172.21.15.83:0/1405385385 learned_addr learned my addr 172.21.15.83:0/1405385385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41a3d7640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 0x7ff41413d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff4193d5640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4141461c0 0x7ff41413ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 -- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4141461c0 msgr2=0x7ff41413ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4141461c0 0x7ff41413ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 -- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 msgr2=0x7ff41413d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 0x7ff41413d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 -- 172.21.15.83:0/1405385385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4140b4940 con 0x7ff4140706e0 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41a3d7640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 0x7ff41413d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff419bd6640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff4100ad040 tx=0x7ff41001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4100be3f0 con 0x7ff4140706e0 2024-05-17T22:08:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4100be590 con 0x7ff4140706e0 2024-05-17T22:08:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4140b3050 con 0x7ff4140706e0 2024-05-17T22:08:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4100be980 con 0x7ff4140706e0 2024-05-17T22:08:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41415a530 con 0x7ff4140706e0 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.330+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4141605a0 con 0x7ff4140706e0 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.334+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4100b43f0 con 0x7ff4140706e0 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.334+0000 7ff40a7fc640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 0x7ff3f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.334+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7ff4101294c0 con 0x7ff4140706e0 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.334+0000 7ff4193d5640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 0x7ff3f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.334+0000 7ff4193d5640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 0x7ff3f8071910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff404000f70 tx=0x7ff404009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.338+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7ff4100c4060 con 0x7ff4140706e0 2024-05-17T22:08:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.494+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4140b41a0 con 0x7ff3f806f400 2024-05-17T22:08:17.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff40a7fc640 1 -- 172.21.15.83:0/1405385385 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7ff4140b41a0 con 0x7ff3f806f400 2024-05-17T22:08:17.487 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:17.487 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 msgr2=0x7ff3f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 0x7ff3f8071910 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff404000f70 tx=0x7ff404009040 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 msgr2=0x7ff4140b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff4100ad040 tx=0x7ff41001ad70 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 shutdown_connections 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7ff3f806f400 0x7ff3f8071910 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff41415e6b0 0x7ff41413d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4141461c0 0x7ff41413ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 --2- 172.21.15.83:0/1405385385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4140706e0 0x7ff4140b0f70 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 >> 172.21.15.83:0/1405385385 conn(0x7ff41406cd90 msgr2=0x7ff41415fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:17.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 shutdown_connections 2024-05-17T22:08:17.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.498+0000 7ff41be3a640 1 -- 172.21.15.83:0/1405385385 wait complete. 2024-05-17T22:08:17.510 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-05-17T22:08:17.206016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278096,"kb_used_data":28632,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696624,"statfs":{"total":773094113280,"available":772809342976,"internally_reserved":0,"allocated":29319168,"data_stored":22871610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.547505"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":283,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-17T22:08:13.799762+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:08:13.799762+0000","last_peered":"2024-05-17T22:08:13.799762+0000","last_clean":"2024-05-17T22:08:13.799762+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:08:13.799762+0000","last_undegraded":"2024-05-17T22:08:13.799762+0000","last_fullsized":"2024-05-17T22:08:13.799762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1208391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":612398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":618023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":5619712,"data_stored":4809114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":611787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":5476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":5607424,"data_stored":4806210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":6217728,"data_stored":5399524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:08:17.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-17T22:08:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.669+0000 7f0746351640 1 Processor -- start 2024-05-17T22:08:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.669+0000 7f0746351640 1 -- start start 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f0746351640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f0740144080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f0746351640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f074015e080 0x7f0740072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f0746351640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f074015ea40 con 0x7f074015e490 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f0746351640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f074015ed20 con 0x7f074015e080 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f0746351640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f074015f000 con 0x7f07400d1040 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f074015e080 0x7f0740072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f074015e080 0x7f0740072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53540/0 (socket says 172.21.15.83:53540) 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 -- 172.21.15.83:0/1569192779 learned_addr learned my addr 172.21.15.83:0/1569192779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073ffff640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f0740144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 -- 172.21.15.83:0/1569192779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 msgr2=0x7f0740144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f0740144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 -- 172.21.15.83:0/1569192779 >> v1:172.21.15.155:6789/0 conn(0x7f074015e490 legacy=0x7f07401616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.673+0000 7f073f7fe640 1 -- 172.21.15.83:0/1569192779 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074006ef30 con 0x7f074015e080 2024-05-17T22:08:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f073f7fe640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e080 0x7f0740072e30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0734009870 tx=0x7f073401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bc14a59cf0fc09 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f073effd640 1 -- 172.21.15.83:0/1569192779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07340ab5f0 con 0x7f074015e080 2024-05-17T22:08:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f073effd640 1 -- 172.21.15.83:0/1569192779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07340b0070 con 0x7f074015e080 2024-05-17T22:08:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f073effd640 1 -- 172.21.15.83:0/1569192779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07340ab9b0 con 0x7f074015e080 2024-05-17T22:08:17.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- 172.21.15.83:0/1569192779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e080 msgr2=0x7f0740072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e080 0x7f0740072e30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0734009870 tx=0x7f073401c560 comp rx=0 tx=0).stop 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- 172.21.15.83:0/1569192779 shutdown_connections 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e080 0x7f0740072e30 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 --2- 172.21.15.83:0/1569192779 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f0740144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- 172.21.15.83:0/1569192779 >> 172.21.15.83:0/1569192779 conn(0x7f074006cd90 msgr2=0x7f07400b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- 172.21.15.83:0/1569192779 shutdown_connections 2024-05-17T22:08:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- 172.21.15.83:0/1569192779 wait complete. 2024-05-17T22:08:17.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 Processor -- start 2024-05-17T22:08:17.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.677+0000 7f0746351640 1 -- start start 2024-05-17T22:08:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f07401417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f074015e080 0x7f0740157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f074016abe0 con 0x7f074015e490 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f074016aa60 con 0x7f074015e080 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f074016ad60 con 0x7f07400d1040 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53544/0 (socket says 172.21.15.83:53544) 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 -- 172.21.15.83:0/60848398 learned_addr learned my addr 172.21.15.83:0/60848398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073f7fe640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f074015e080 0x7f0740157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073ffff640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f07401417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 -- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 msgr2=0x7f07401417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f07401417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 -- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f074015e080 msgr2=0x7f0740157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f074015e080 0x7f0740157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 -- 172.21.15.83:0/60848398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07401446d0 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f07448ee640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f073800b410 tx=0x7f073800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738015020 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0738011830 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401449b0 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738011c60 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0740141ce0 con 0x7f074015e490 2024-05-17T22:08:17.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0740141f10 con 0x7f074015e490 2024-05-17T22:08:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.681+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0738018020 con 0x7f074015e490 2024-05-17T22:08:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.685+0000 7f073cff9640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 0x7f072c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.685+0000 7f073ffff640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 0x7f072c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.685+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f073808f3c0 con 0x7f074015e490 2024-05-17T22:08:17.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.685+0000 7f073ffff640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 0x7f072c071910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0730009e90 tx=0x7f073000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.685+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0740141f10 con 0x7f074015e490 2024-05-17T22:08:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.849+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0740144080 con 0x7f074015e490 2024-05-17T22:08:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.853+0000 7f073cff9640 1 -- 172.21.15.83:0/60848398 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+13264 (secure 0 0 0) 0x7f073801f020 con 0x7f074015e490 2024-05-17T22:08:17.844 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:17.844 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":115,"fsid":"5ad90e8e-75cb-414c-a7e5-c0c2d376e9ce","created":"2024-05-17T21:57:06.317411+0000","modified":"2024-05-17T22:08:14.685775+0000","last_up_change":"2024-05-17T21:57:18.169014+0000","last_in_change":"2024-05-17T21:57:08.940892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-17T21:57:19.014572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d56ff5-e600-4fb8-b3ec-af99c7b1feed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6817","nonce":622135753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6819","nonce":622135753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6823","nonce":622135753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":622135753},{"type":"v1","addr":"172.21.15.83:6821","nonce":622135753}]},"public_addr":"172.21.15.83:6817/622135753","cluster_addr":"172.21.15.83:6819/622135753","heartbeat_back_addr":"172.21.15.83:6823/622135753","heartbeat_front_addr":"172.21.15.83:6821/622135753","state":["exists","up"]},{"osd":1,"uuid":"57dc8e47-0c7f-4b36-a07a-b0f33948ed67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6801","nonce":3404427378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6803","nonce":3404427378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6807","nonce":3404427378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3404427378},{"type":"v1","addr":"172.21.15.83:6805","nonce":3404427378}]},"public_addr":"172.21.15.83:6801/3404427378","cluster_addr":"172.21.15.83:6803/3404427378","heartbeat_back_addr":"172.21.15.83:6807/3404427378","heartbeat_front_addr":"172.21.15.83:6805/3404427378","state":["exists","up"]},{"osd":2,"uuid":"cf16c6e7-af1a-4df6-83c0-7d11c4b4e1d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6809","nonce":3743214759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6811","nonce":3743214759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6815","nonce":3743214759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3743214759},{"type":"v1","addr":"172.21.15.83:6813","nonce":3743214759}]},"public_addr":"172.21.15.83:6809/3743214759","cluster_addr":"172.21.15.83:6811/3743214759","heartbeat_back_addr":"172.21.15.83:6815/3743214759","heartbeat_front_addr":"172.21.15.83:6813/3743214759","state":["exists","up"]},{"osd":3,"uuid":"39aa7753-240c-48cb-8eca-ba1b7106cffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6825","nonce":2257217219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6827","nonce":2257217219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6831","nonce":2257217219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2257217219},{"type":"v1","addr":"172.21.15.83:6829","nonce":2257217219}]},"public_addr":"172.21.15.83:6825/2257217219","cluster_addr":"172.21.15.83:6827/2257217219","heartbeat_back_addr":"172.21.15.83:6831/2257217219","heartbeat_front_addr":"172.21.15.83:6829/2257217219","state":["exists","up"]},{"osd":4,"uuid":"a24fee4b-1bc7-4d13-92f9-db435b71cd60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6825","nonce":505763805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6827","nonce":505763805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6831","nonce":505763805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":505763805},{"type":"v1","addr":"172.21.15.155:6829","nonce":505763805}]},"public_addr":"172.21.15.155:6825/505763805","cluster_addr":"172.21.15.155:6827/505763805","heartbeat_back_addr":"172.21.15.155:6831/505763805","heartbeat_front_addr":"172.21.15.155:6829/505763805","state":["exists","up"]},{"osd":5,"uuid":"cbbb1c7a-e4a3-4569-9900-ec42752e7266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6809","nonce":758721566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6811","nonce":758721566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6815","nonce":758721566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":758721566},{"type":"v1","addr":"172.21.15.155:6813","nonce":758721566}]},"public_addr":"172.21.15.155:6809/758721566","cluster_addr":"172.21.15.155:6811/758721566","heartbeat_back_addr":"172.21.15.155:6815/758721566","heartbeat_front_addr":"172.21.15.155:6813/758721566","state":["exists","up"]},{"osd":6,"uuid":"10f68bf4-5160-4d6f-a486-d651cc4e79bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6801","nonce":1025803281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6803","nonce":1025803281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6807","nonce":1025803281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":1025803281},{"type":"v1","addr":"172.21.15.155:6805","nonce":1025803281}]},"public_addr":"172.21.15.155:6801/1025803281","cluster_addr":"172.21.15.155:6803/1025803281","heartbeat_back_addr":"172.21.15.155:6807/1025803281","heartbeat_front_addr":"172.21.15.155:6805/1025803281","state":["exists","up"]},{"osd":7,"uuid":"3d54ba58-d840-4d28-87b4-0a564e3c58e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6817","nonce":3020759009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6819","nonce":3020759009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6823","nonce":3020759009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":3020759009},{"type":"v1","addr":"172.21.15.155:6821","nonce":3020759009}]},"public_addr":"172.21.15.155:6817/3020759009","cluster_addr":"172.21.15.155:6819/3020759009","heartbeat_back_addr":"172.21.15.155:6823/3020759009","heartbeat_front_addr":"172.21.15.155:6821/3020759009","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.631405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.340682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.493171+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.470757+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.456114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.455735+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.225912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-17T21:57:14.594450+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.83:6837/1541446976":"2024-05-18T22:08:11.652583+0000","172.21.15.83:6835/1541446976":"2024-05-18T22:08:11.652583+0000","172.21.15.83:6835/78757339":"2024-05-18T22:07:57.881130+0000","172.21.15.155:6833/1193425174":"2024-05-18T22:07:40.363116+0000","172.21.15.155:6832/1193425174":"2024-05-18T22:07:40.363116+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.853+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 msgr2=0x7f072c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.853+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 0x7f072c071910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0730009e90 tx=0x7f073000b040 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.853+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 msgr2=0x7f0740146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.853+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f073800b410 tx=0x7f073800b8d0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 shutdown_connections 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f074015e490 0x7f0740146d90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f072c06f400 0x7f072c071910 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f074015e080 0x7f0740157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 --2- 172.21.15.83:0/60848398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07400d1040 0x7f07401417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 >> 172.21.15.83:0/60848398 conn(0x7f074006cd90 msgr2=0x7f07400b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 shutdown_connections 2024-05-17T22:08:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:17.857+0000 7f0746351640 1 -- 172.21.15.83:0/60848398 wait complete. 2024-05-17T22:08:18.869 INFO:tasks.ceph:Scrubbing osd.0 2024-05-17T22:08:18.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:19.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.045+0000 7fdb6dcf4640 1 Processor -- start 2024-05-17T22:08:19.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.045+0000 7fdb6dcf4640 1 -- start start 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb68145fd0 0x7fdb680cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb680d1210 con 0x7fdb680d03b0 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb680d0970 con 0x7fdb68160b00 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdb680ce360 con 0x7fdb68145fd0 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb68145fd0 0x7fdb680cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb68145fd0 0x7fdb680cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53560/0 (socket says 172.21.15.83:53560) 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 -- 172.21.15.83:0/189812809 learned_addr learned my addr 172.21.15.83:0/189812809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb677fe640 1 --1- 172.21.15.83:0/189812809 >> v1:172.21.15.83:6789/0 conn(0x7fdb680d03b0 0x7fdb681603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46172/0 (socket says 172.21.15.83:46172) 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 -- 172.21.15.83:0/189812809 >> v1:172.21.15.155:6789/0 conn(0x7fdb68160b00 legacy=0x7fdb6806f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 -- 172.21.15.83:0/189812809 >> v1:172.21.15.83:6789/0 conn(0x7fdb680d03b0 legacy=0x7fdb681603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 -- 172.21.15.83:0/189812809 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb680d0c50 con 0x7fdb68145fd0 2024-05-17T22:08:19.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb667fc640 1 -- 172.21.15.83:0/189812809 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425420775 0 0) 0x7fdb680d1210 con 0x7fdb680d03b0 2024-05-17T22:08:19.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb67fff640 1 --2- 172.21.15.83:0/189812809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb68145fd0 0x7fdb680cde20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e50 tx=0x7fdb6001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9807d037f1d38e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb667fc640 1 -- 172.21.15.83:0/189812809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600a57f0 con 0x7fdb68145fd0 2024-05-17T22:08:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb667fc640 1 -- 172.21.15.83:0/189812809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb600a5990 con 0x7fdb68145fd0 2024-05-17T22:08:19.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb667fc640 1 -- 172.21.15.83:0/189812809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600a5d60 con 0x7fdb68145fd0 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/189812809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb68145fd0 msgr2=0x7fdb680cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 --2- 172.21.15.83:0/189812809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb68145fd0 0x7fdb680cde20 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e50 tx=0x7fdb6001ca30 comp rx=0 tx=0).stop 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/189812809 shutdown_connections 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 --2- 172.21.15.83:0/189812809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb68145fd0 0x7fdb680cde20 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/189812809 >> 172.21.15.83:0/189812809 conn(0x7fdb6806cd90 msgr2=0x7fdb680b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/189812809 shutdown_connections 2024-05-17T22:08:19.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.049+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/189812809 wait complete. 2024-05-17T22:08:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 Processor -- start 2024-05-17T22:08:19.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 -- start start 2024-05-17T22:08:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 0x7fdb68142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb68145fd0 0x7fdb68135b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb6813b930 0x7fdb6815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68070540 con 0x7fdb680d03b0 2024-05-17T22:08:19.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdb680703c0 con 0x7fdb68145fd0 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb6dcf4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fdb680706c0 con 0x7fdb6813b930 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 0x7fdb68142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 0x7fdb68142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53574/0 (socket says 172.21.15.83:53574) 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 -- 172.21.15.83:0/174952483 learned_addr learned my addr 172.21.15.83:0/174952483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb66ffd640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb68145fd0 0x7fdb68135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 -- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb6813b930 msgr2=0x7fdb6815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb67fff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb6813b930 0x7fdb6815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb6813b930 0x7fdb6815aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 -- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb68145fd0 msgr2=0x7fdb68135b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb68145fd0 0x7fdb68135b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb68143110 con 0x7fdb680d03b0 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb66ffd640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb68145fd0 0x7fdb68135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb67fff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb6813b930 0x7fdb6815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:19.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.053+0000 7fdb677fe640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 0x7fdb68142bd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c460 tx=0x7fdb5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58018070 con 0x7fdb680d03b0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb58007500 con 0x7fdb680d03b0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb68140e00 con 0x7fdb680d03b0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58004a30 con 0x7fdb680d03b0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681413c0 con 0x7fdb680d03b0 2024-05-17T22:08:19.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb680cde20 con 0x7fdb680d03b0 2024-05-17T22:08:19.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdb5801b070 con 0x7fdb680d03b0 2024-05-17T22:08:19.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb5406f400 0x7fdb54071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fdb5808e930 con 0x7fdb680d03b0 2024-05-17T22:08:19.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb66ffd640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb5406f400 0x7fdb54071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7fdb54075450 0x7fdb540778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb54077f70 con 0x7fdb54075450 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fdb5808edc0 con 0x7fdb680d03b0 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb67fff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7fdb54075450 0x7fdb540778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb66ffd640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb5406f400 0x7fdb54071910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb68133710 tx=0x7fdb5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb67fff640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7fdb54075450 0x7fdb540778a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.057+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== osd.0 v2:172.21.15.83:6816/622135753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb54077f70 con 0x7fdb54075450 2024-05-17T22:08:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 --> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdb680b3f40 con 0x7fdb54075450 2024-05-17T22:08:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb4ffff640 1 -- 172.21.15.83:0/174952483 <== osd.0 v2:172.21.15.83:6816/622135753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdb680b3f40 con 0x7fdb54075450 2024-05-17T22:08:19.064 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:19.064 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:19.064 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7fdb54075450 msgr2=0x7fdb540778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6816/622135753,v1:172.21.15.83:6817/622135753] conn(0x7fdb54075450 0x7fdb540778a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb5406f400 msgr2=0x7fdb54071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fdb5406f400 0x7fdb54071910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb68133710 tx=0x7fdb5c008040 comp rx=0 tx=0).stop 2024-05-17T22:08:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 msgr2=0x7fdb68142bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 --2- 172.21.15.83:0/174952483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb680d03b0 0x7fdb68142bd0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c460 tx=0x7fdb5800c920 comp rx=0 tx=0).stop 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb67fff640 1 -- 172.21.15.83:0/174952483 reap_dead start 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 shutdown_connections 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 >> 172.21.15.83:0/174952483 conn(0x7fdb6806cd90 msgr2=0x7fdb68160340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 shutdown_connections 2024-05-17T22:08:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.073+0000 7fdb6dcf4640 1 -- 172.21.15.83:0/174952483 wait complete. 2024-05-17T22:08:19.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-17T22:08:19.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 Processor -- start 2024-05-17T22:08:19.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 -- start start 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b68c0 0x7fe5a40cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5a41438f0 0x7fe5a40b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a40b4db0 con 0x7fe5a415eae0 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a40b71b0 con 0x7fe5a41438f0 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a9474640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40ce530 con 0x7fe5a40b68c0 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5a41438f0 0x7fe5a40b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5a41438f0 0x7fe5a40b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53588/0 (socket says 172.21.15.83:53588) 2024-05-17T22:08:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a27fc640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe5a415eae0 0x7fe5a41431e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46174/0 (socket says 172.21.15.83:46174) 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 -- 172.21.15.83:0/1916218908 learned_addr learned my addr 172.21.15.83:0/1916218908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a2ffd640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b68c0 0x7fe5a40cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 -- 172.21.15.83:0/1916218908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b68c0 msgr2=0x7fe5a40cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b68c0 0x7fe5a40cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 -- 172.21.15.83:0/1916218908 >> v1:172.21.15.83:6789/0 conn(0x7fe5a415eae0 legacy=0x7fe5a41431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a37fe640 1 -- 172.21.15.83:0/1916218908 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40b6ed0 con 0x7fe5a41438f0 2024-05-17T22:08:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.233+0000 7fe5a1ffb640 1 -- 172.21.15.83:0/1916218908 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 845692627 0 0) 0x7fe5a40b4db0 con 0x7fe5a415eae0 2024-05-17T22:08:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a37fe640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a41438f0 0x7fe5a40b0c10 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40b4f80 tx=0x7fe59401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b35bff2fec698b server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a1ffb640 1 -- 172.21.15.83:0/1916218908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940ab5f0 con 0x7fe5a41438f0 2024-05-17T22:08:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a1ffb640 1 -- 172.21.15.83:0/1916218908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5940b0070 con 0x7fe5a41438f0 2024-05-17T22:08:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a1ffb640 1 -- 172.21.15.83:0/1916218908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940ab9b0 con 0x7fe5a41438f0 2024-05-17T22:08:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 -- 172.21.15.83:0/1916218908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a41438f0 msgr2=0x7fe5a40b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a41438f0 0x7fe5a40b0c10 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40b4f80 tx=0x7fe59401c4e0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 -- 172.21.15.83:0/1916218908 shutdown_connections 2024-05-17T22:08:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a41438f0 0x7fe5a40b0c10 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 --2- 172.21.15.83:0/1916218908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b68c0 0x7fe5a40cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 -- 172.21.15.83:0/1916218908 >> 172.21.15.83:0/1916218908 conn(0x7fe5a40ce6e0 msgr2=0x7fe5a40ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 -- 172.21.15.83:0/1916218908 shutdown_connections 2024-05-17T22:08:19.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.237+0000 7fe5a9474640 1 -- 172.21.15.83:0/1916218908 wait complete. 2024-05-17T22:08:19.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 Processor -- start 2024-05-17T22:08:19.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- start start 2024-05-17T22:08:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 0x7fe5a40b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 0x7fe5a4146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40b16e0 con 0x7fe5a40b68c0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40b1560 con 0x7fe5a415eae0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a406d280 con 0x7fe5a40b2df0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53592/0 (socket says 172.21.15.83:53592) 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 -- 172.21.15.83:0/4003648452 learned_addr learned my addr 172.21.15.83:0/4003648452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a37fe640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 0x7fe5a4146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a27fc640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 0x7fe5a40b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 -- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 msgr2=0x7fe5a4146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 0x7fe5a4146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 -- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 msgr2=0x7fe5a40b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 0x7fe5a40b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 -- 172.21.15.83:0/4003648452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40b3200 con 0x7fe5a40b68c0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a27fc640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 0x7fe5a40b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a37fe640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 0x7fe5a4146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a2ffd640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00ccc0 tx=0x7fe59c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0039a0 con 0x7fe5a40b68c0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59c003b40 con 0x7fe5a40b68c0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a4144590 con 0x7fe5a40b68c0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004030 con 0x7fe5a40b68c0 2024-05-17T22:08:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a4144870 con 0x7fe5a40b68c0 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.241+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a40cdff0 con 0x7fe5a40b68c0 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.245+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe59c0041d0 con 0x7fe5a40b68c0 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.245+0000 7fe59b7fe640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 0x7fe584071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.245+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fe59c08f560 con 0x7fe5a40b68c0 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.245+0000 7fe5a27fc640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 0x7fe584071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.245+0000 7fe5a27fc640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 0x7fe584071940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000f70 tx=0x7fe58c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.249+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fe5a40b3200 con 0x7fe5a40b68c0 2024-05-17T22:08:19.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fe5a406e700 con 0x7fe58406f430 2024-05-17T22:08:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe59b7fe640 1 -- 172.21.15.83:0/4003648452 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe5a406e700 con 0x7fe58406f430 2024-05-17T22:08:19.403 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 msgr2=0x7fe584071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 0x7fe584071940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000f70 tx=0x7fe58c009040 comp rx=0 tx=0).stop 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 msgr2=0x7fe5a41422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00ccc0 tx=0x7fe59c00a5c0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 shutdown_connections 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5a40b2df0 0x7fe5a4146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe58406f430 0x7fe584071940 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5a415eae0 0x7fe5a40b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 --2- 172.21.15.83:0/4003648452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5a40b68c0 0x7fe5a41422f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 >> 172.21.15.83:0/4003648452 conn(0x7fe5a40ce6e0 msgr2=0x7fe5a40cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 shutdown_connections 2024-05-17T22:08:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.413+0000 7fe5a9474640 1 -- 172.21.15.83:0/4003648452 wait complete. 2024-05-17T22:08:19.426 INFO:tasks.ceph:Scrubbing osd.1 2024-05-17T22:08:19.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 Processor -- start 2024-05-17T22:08:19.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 -- start start 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d741675a0 0x7f0d740c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d740d18e0 0x7f0d740b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d740c5f90 con 0x7f0d740c5b80 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0d740b2470 con 0x7f0d741675a0 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7916e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f0d740d1cf0 con 0x7f0d740d18e0 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d72d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d741675a0 0x7f0d740c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7259c640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0d740c5b80 0x7f0d740d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46176/0 (socket says 172.21.15.83:46176) 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7259c640 1 -- 172.21.15.83:0/2331022067 learned_addr learned my addr 172.21.15.83:0/2331022067 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d7359e640 1 --2- 172.21.15.83:0/2331022067 >> v2:172.21.15.155:3300/0 conn(0x7f0d740d18e0 0x7f0d740b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d72d9d640 1 -- 172.21.15.83:0/2331022067 >> v2:172.21.15.155:3300/0 conn(0x7f0d740d18e0 msgr2=0x7f0d740b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d72d9d640 1 --2- 172.21.15.83:0/2331022067 >> v2:172.21.15.155:3300/0 conn(0x7f0d740d18e0 0x7f0d740b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d72d9d640 1 -- 172.21.15.83:0/2331022067 >> v1:172.21.15.83:6789/0 conn(0x7f0d740c5b80 legacy=0x7f0d740d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d71d9b640 1 -- 172.21.15.83:0/2331022067 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917429520 0 0) 0x7f0d68001120 con 0x7f0d740c5b80 2024-05-17T22:08:19.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.577+0000 7f0d72d9d640 1 -- 172.21.15.83:0/2331022067 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d740d1f40 con 0x7f0d741675a0 2024-05-17T22:08:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d72d9d640 1 --2- 172.21.15.83:0/2331022067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d741675a0 0x7f0d740c5640 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f0d740740b0 tx=0x7f0d6c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fd5134a3ca01d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d71d9b640 1 -- 172.21.15.83:0/2331022067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0b4070 con 0x7f0d741675a0 2024-05-17T22:08:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d71d9b640 1 -- 172.21.15.83:0/2331022067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c0af600 con 0x7f0d741675a0 2024-05-17T22:08:19.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d71d9b640 1 -- 172.21.15.83:0/2331022067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0af9d0 con 0x7f0d741675a0 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 -- 172.21.15.83:0/2331022067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d741675a0 msgr2=0x7f0d740c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 --2- 172.21.15.83:0/2331022067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d741675a0 0x7f0d740c5640 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f0d740740b0 tx=0x7f0d6c01f110 comp rx=0 tx=0).stop 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 -- 172.21.15.83:0/2331022067 shutdown_connections 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 --2- 172.21.15.83:0/2331022067 >> v2:172.21.15.155:3300/0 conn(0x7f0d740d18e0 0x7f0d740b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 --2- 172.21.15.83:0/2331022067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d741675a0 0x7f0d740c5640 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 -- 172.21.15.83:0/2331022067 >> 172.21.15.83:0/2331022067 conn(0x7f0d740b4200 msgr2=0x7f0d740b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 -- 172.21.15.83:0/2331022067 shutdown_connections 2024-05-17T22:08:19.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.581+0000 7f0d7916e640 1 -- 172.21.15.83:0/2331022067 wait complete. 2024-05-17T22:08:19.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 Processor -- start 2024-05-17T22:08:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- start start 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d740c5b80 0x7f0d74153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 0x7f0d740ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d741675a0 0x7f0d740ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741679b0 con 0x7f0d740d18e0 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0d740b2100 con 0x7f0d741675a0 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0d74167b30 con 0x7f0d740c5b80 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 0x7f0d740ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 0x7f0d740ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53612/0 (socket says 172.21.15.83:53612) 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 -- 172.21.15.83:0/860696032 learned_addr learned my addr 172.21.15.83:0/860696032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 -- 172.21.15.83:0/860696032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d740c5b80 msgr2=0x7f0d74153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d740c5b80 0x7f0d74153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 -- 172.21.15.83:0/860696032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d741675a0 msgr2=0x7f0d740ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d741675a0 0x7f0d740ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7415e7b0 con 0x7f0d740d18e0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7259c640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 0x7f0d740ce7f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c430 tx=0x7f0d6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680074e0 con 0x7f0d740d18e0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d680047f0 con 0x7f0d740d18e0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d74153f50 con 0x7f0d740d18e0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68004be0 con 0x7f0d740d18e0 2024-05-17T22:08:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7415ed00 con 0x7f0d740d18e0 2024-05-17T22:08:19.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d740b00b0 con 0x7f0d740d18e0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.585+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0d68011070 con 0x7f0d740d18e0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d6006f400 0x7f0d60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d72d9d640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d6006f400 0x7f0d60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f0d6808f530 con 0x7f0d740d18e0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f0d60075450 0x7f0d600778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d68005e50 con 0x7f0d60075450 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f0d6808f9c0 con 0x7f0d740d18e0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d7359e640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f0d60075450 0x7f0d600778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d72d9d640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d6006f400 0x7f0d60071910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0b1000 tx=0x7f0d6c004ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d7359e640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f0d60075450 0x7f0d600778a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.589+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== osd.1 v2:172.21.15.83:6800/3404427378 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d68005e50 con 0x7f0d60075450 2024-05-17T22:08:19.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 --> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0d7415f420 con 0x7f0d60075450 2024-05-17T22:08:19.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d5b7fe640 1 -- 172.21.15.83:0/860696032 <== osd.1 v2:172.21.15.83:6800/3404427378 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0d7415f420 con 0x7f0d60075450 2024-05-17T22:08:19.593 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:19.593 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:19.593 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f0d60075450 msgr2=0x7f0d600778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6800/3404427378,v1:172.21.15.83:6801/3404427378] conn(0x7f0d60075450 0x7f0d600778a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d6006f400 msgr2=0x7f0d60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0d6006f400 0x7f0d60071910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0b1000 tx=0x7f0d6c004ac0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 msgr2=0x7f0d740ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 --2- 172.21.15.83:0/860696032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d740d18e0 0x7f0d740ce7f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c430 tx=0x7f0d6800c8f0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7359e640 1 -- 172.21.15.83:0/860696032 reap_dead start 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 shutdown_connections 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 >> 172.21.15.83:0/860696032 conn(0x7f0d740b4200 msgr2=0x7f0d74075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 shutdown_connections 2024-05-17T22:08:19.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.601+0000 7f0d7916e640 1 -- 172.21.15.83:0/860696032 wait complete. 2024-05-17T22:08:19.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-17T22:08:19.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 Processor -- start 2024-05-17T22:08:19.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 -- start start 2024-05-17T22:08:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd5fc0d5210 0x7fd5fc0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5fc0d4170 con 0x7fd5fc0d2000 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5fc0d4450 con 0x7fd5fc0b4810 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.749+0000 7fd6043c5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fd5fc0d5620 con 0x7fd5fc0d5210 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602962640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd5fc0d2000 0x7fd5fc0d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46178/0 (socket says 172.21.15.83:46178) 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602962640 1 -- 172.21.15.83:0/3728721611 learned_addr learned my addr 172.21.15.83:0/3728721611 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756201426 0 0) 0x7fd5fc0d4170 con 0x7fd5fc0d2000 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd601960640 1 --2- 172.21.15.83:0/3728721611 >> v2:172.21.15.155:3300/0 conn(0x7fd5fc0d5210 0x7fd5fc0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e8003530 con 0x7fd5fc0d2000 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3616250767 0 0) 0x7fd5e8003530 con 0x7fd5fc0d2000 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 >> v2:172.21.15.155:3300/0 conn(0x7fd5fc0d5210 msgr2=0x7fd5fc0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 --2- 172.21.15.83:0/3728721611 >> v2:172.21.15.155:3300/0 conn(0x7fd5fc0d5210 0x7fd5fc0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 >> v1:172.21.15.155:6789/0 conn(0x7fd5fc0b4810 legacy=0x7fd5fc15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc0d4730 con 0x7fd5fc0d2000 2024-05-17T22:08:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fd5f4001a80 con 0x7fd5fc0d2000 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 >> v1:172.21.15.83:6789/0 conn(0x7fd5fc0d2000 legacy=0x7fd5fc0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 0x7fd5e8003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5e8004810 0x7fd5e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8005260 0x7fd5e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc0d5620 con 0x7fd5e8003a60 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e800c1f0 con 0x7fd5e8005260 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e800c3f0 con 0x7fd5e8004810 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 0x7fd5e8003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602962640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8005260 0x7fd5e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 -- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5e8004810 msgr2=0x7fd5e8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd601960640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5e8004810 0x7fd5e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5e8004810 0x7fd5e8004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 -- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8005260 msgr2=0x7fd5e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8005260 0x7fd5e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 -- 172.21.15.83:0/3728721611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8003530 con 0x7fd5e8003a60 2024-05-17T22:08:19.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602962640 1 -- 172.21.15.83:0/3728721611 reap_dead start 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd602161640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 0x7fd5e8003ed0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8001790 tx=0x7fd5f801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d34d19718858c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80b0070 con 0x7fd5e8003a60 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f80052d0 con 0x7fd5e8003a60 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.753+0000 7fd60115f640 1 -- 172.21.15.83:0/3728721611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80013e0 con 0x7fd5e8003a60 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 msgr2=0x7fd5e8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 0x7fd5e8003ed0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8001790 tx=0x7fd5f801c3d0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- 172.21.15.83:0/3728721611 shutdown_connections 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 --2- 172.21.15.83:0/3728721611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5e8003a60 0x7fd5e8003ed0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- 172.21.15.83:0/3728721611 >> 172.21.15.83:0/3728721611 conn(0x7fd5fc06e010 msgr2=0x7fd5fc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- 172.21.15.83:0/3728721611 shutdown_connections 2024-05-17T22:08:19.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- 172.21.15.83:0/3728721611 wait complete. 2024-05-17T22:08:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 Processor -- start 2024-05-17T22:08:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.757+0000 7fd6043c5640 1 -- start start 2024-05-17T22:08:19.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8003a60 0x7fd5fc1ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5fc0b75f0 0x7fd5fc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc0d2410 con 0x7fd5fc1f8590 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc0d06a0 con 0x7fd5e8003a60 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd5fc0d2590 con 0x7fd5fc0b75f0 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53634/0 (socket says 172.21.15.83:53634) 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 -- 172.21.15.83:0/4173805864 learned_addr learned my addr 172.21.15.83:0/4173805864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd602161640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8003a60 0x7fd5fc1ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd602962640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5fc0b75f0 0x7fd5fc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 -- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5fc0b75f0 msgr2=0x7fd5fc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5fc0b75f0 0x7fd5fc14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 -- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8003a60 msgr2=0x7fd5fc1ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8003a60 0x7fd5fc1ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 -- 172.21.15.83:0/4173805864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc0b7a00 con 0x7fd5fc1f8590 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd601960640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0010830 tx=0x7fd5f0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000f040 con 0x7fd5fc1f8590 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f000eb00 con 0x7fd5fc1f8590 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f001c400 con 0x7fd5fc1f8590 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1ecb00 con 0x7fd5fc1f8590 2024-05-17T22:08:19.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1ed010 con 0x7fd5fc1f8590 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc1ecd80 con 0x7fd5fc1f8590 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5f001c5a0 con 0x7fd5fc1f8590 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 0x7fd5e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fd5f008f590 con 0x7fd5fc1f8590 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd602161640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 0x7fd5e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:19.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.761+0000 7fd602161640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 0x7fd5e0071940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80ad000 tx=0x7fd5f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:19.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.765+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd5fc0b7a00 con 0x7fd5fc1f8590 2024-05-17T22:08:19.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd5fc0cf900 con 0x7fd5e006f430 2024-05-17T22:08:19.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd5eeffd640 1 -- 172.21.15.83:0/4173805864 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5fc0cf900 con 0x7fd5e006f430 2024-05-17T22:08:19.908 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2024-05-17T22:08:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 msgr2=0x7fd5e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 0x7fd5e0071940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80ad000 tx=0x7fd5f8009040 comp rx=0 tx=0).stop 2024-05-17T22:08:19.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 msgr2=0x7fd5fc0b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0010830 tx=0x7fd5f0010cf0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 shutdown_connections 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5fc0b75f0 0x7fd5fc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5fc1f8590 0x7fd5fc0b70b0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd5e006f430 0x7fd5e0071940 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 --2- 172.21.15.83:0/4173805864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e8003a60 0x7fd5fc1ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 >> 172.21.15.83:0/4173805864 conn(0x7fd5fc06e010 msgr2=0x7fd5fc0b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 shutdown_connections 2024-05-17T22:08:19.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:19.917+0000 7fd6043c5640 1 -- 172.21.15.83:0/4173805864 wait complete. 2024-05-17T22:08:19.928 INFO:tasks.ceph:Scrubbing osd.2 2024-05-17T22:08:19.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:20.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 Processor -- start 2024-05-17T22:08:20.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 -- start start 2024-05-17T22:08:20.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f638c1641e0 0x7f638c0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c06c790 0x7f638c06cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638c06d0e0 con 0x7f638c0af900 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f638c0af470 con 0x7f638c1641e0 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f639290c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f638c0b5a20 con 0x7f638c06c790 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f638c1641e0 0x7f638c0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f6390ea9640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f638c0af900 0x7f638c0afd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46182/0 (socket says 172.21.15.83:46182) 2024-05-17T22:08:20.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f6390ea9640 1 -- 172.21.15.83:0/4277372730 learned_addr learned my addr 172.21.15.83:0/4277372730 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638affd640 1 -- 172.21.15.83:0/4277372730 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448460896 0 0) 0x7f63840014a0 con 0x7f638c0af900 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638b7fe640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c06c790 0x7f638c06cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638affd640 1 -- 172.21.15.83:0/4277372730 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6374003530 con 0x7f638c0af900 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638bfff640 1 -- 172.21.15.83:0/4277372730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c06c790 msgr2=0x7f638c06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638bfff640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c06c790 0x7f638c06cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638bfff640 1 -- 172.21.15.83:0/4277372730 >> v1:172.21.15.83:6789/0 conn(0x7f638c0af900 legacy=0x7f638c0afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.085+0000 7f638bfff640 1 -- 172.21.15.83:0/4277372730 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c0b5d00 con 0x7f638c1641e0 2024-05-17T22:08:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f638bfff640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b7650 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f637c0097c0 tx=0x7f637c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3275e0a2e83b5f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f638affd640 1 -- 172.21.15.83:0/4277372730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0b0070 con 0x7f638c1641e0 2024-05-17T22:08:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f638affd640 1 -- 172.21.15.83:0/4277372730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c0ab600 con 0x7f638c1641e0 2024-05-17T22:08:20.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f638affd640 1 -- 172.21.15.83:0/4277372730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0b0c30 con 0x7f638c1641e0 2024-05-17T22:08:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 -- 172.21.15.83:0/4277372730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 msgr2=0x7f638c0b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b7650 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f637c0097c0 tx=0x7f637c01c380 comp rx=0 tx=0).stop 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 -- 172.21.15.83:0/4277372730 shutdown_connections 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c06c790 0x7f638c06cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 --2- 172.21.15.83:0/4277372730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b7650 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 -- 172.21.15.83:0/4277372730 >> 172.21.15.83:0/4277372730 conn(0x7f638c09b8b0 msgr2=0x7f638c0b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 -- 172.21.15.83:0/4277372730 shutdown_connections 2024-05-17T22:08:20.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.089+0000 7f639290c640 1 -- 172.21.15.83:0/4277372730 wait complete. 2024-05-17T22:08:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 Processor -- start 2024-05-17T22:08:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- start start 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638c06c790 0x7f638c1fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c0af900 0x7f638c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f638c16cd80 con 0x7f638c1641e0 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1610b0 con 0x7f638c06c790 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f638c16cf00 con 0x7f638c0af900 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53654/0 (socket says 172.21.15.83:53654) 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 -- 172.21.15.83:0/1669620392 learned_addr learned my addr 172.21.15.83:0/1669620392 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:20.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f638b7fe640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c0af900 0x7f638c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 -- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c0af900 msgr2=0x7f638c14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f638c0af900 0x7f638c14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 -- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638c06c790 msgr2=0x7f638c1fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f638bfff640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638c06c790 0x7f638c1fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638c06c790 0x7f638c1fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c15eb00 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6390ea9640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b1d70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f638400ca70 tx=0x7f638400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638400abd0 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638400ad70 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384012620 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c1588b0 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c15f050 con 0x7f638c1641e0 2024-05-17T22:08:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.093+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f638c06cba0 con 0x7f638c1641e0 2024-05-17T22:08:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63840127c0 con 0x7f638c1641e0 2024-05-17T22:08:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f638bfff640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f63840900b0 con 0x7f638c1641e0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f6378075450 0x7f63780778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6384006e50 con 0x7f6378075450 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f6384090540 con 0x7f638c1641e0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f638b7fe640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f6378075450 0x7f63780778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f638bfff640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f637c0ad000 tx=0x7f637c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f638b7fe640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f6378075450 0x7f63780778a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.097+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== osd.2 v2:172.21.15.83:6808/3743214759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6384006e50 con 0x7f6378075450 2024-05-17T22:08:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.109+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 --> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f638c158660 con 0x7f6378075450 2024-05-17T22:08:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f6388ff9640 1 -- 172.21.15.83:0/1669620392 <== osd.2 v2:172.21.15.83:6808/3743214759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f638c158660 con 0x7f6378075450 2024-05-17T22:08:20.102 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:20.103 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:20.103 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f6378075450 msgr2=0x7f63780778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6808/3743214759,v1:172.21.15.83:6809/3743214759] conn(0x7f6378075450 0x7f63780778a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 msgr2=0x7f6378071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f637806f400 0x7f6378071910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f637c0ad000 tx=0x7f637c0a6000 comp rx=0 tx=0).stop 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 msgr2=0x7f638c0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 --2- 172.21.15.83:0/1669620392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f638c1641e0 0x7f638c0b1d70 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f638400ca70 tx=0x7f638400cf30 comp rx=0 tx=0).stop 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f6390ea9640 1 -- 172.21.15.83:0/1669620392 reap_dead start 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 shutdown_connections 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 >> 172.21.15.83:0/1669620392 conn(0x7f638c09b8b0 msgr2=0x7f638c0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 shutdown_connections 2024-05-17T22:08:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.113+0000 7f639290c640 1 -- 172.21.15.83:0/1669620392 wait complete. 2024-05-17T22:08:20.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-17T22:08:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.269+0000 7f901fab0640 1 Processor -- start 2024-05-17T22:08:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.269+0000 7f901fab0640 1 -- start start 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901fab0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f90180b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901fab0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f901815d700 0x7f90181663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901fab0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90180b5640 con 0x7f90180cdf40 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901fab0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9018070870 con 0x7f901815d700 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901fab0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f90180b58e0 con 0x7f90180cf830 2024-05-17T22:08:20.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f901815d700 0x7f90181663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901d04b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f90180cdf40 0x7f901815cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46198/0 (socket says 172.21.15.83:46198) 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901d04b640 1 -- 172.21.15.83:0/3449719446 learned_addr learned my addr 172.21.15.83:0/3449719446 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901d84c640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f90180b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 -- 172.21.15.83:0/3449719446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 msgr2=0x7f90180b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f90180b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 -- 172.21.15.83:0/3449719446 >> v1:172.21.15.83:6789/0 conn(0x7f90180cdf40 legacy=0x7f901815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 -- 172.21.15.83:0/3449719446 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901806f060 con 0x7f901815d700 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901c84a640 1 -- 172.21.15.83:0/3449719446 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793868184 0 0) 0x7f90180b5640 con 0x7f90180cdf40 2024-05-17T22:08:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901d84c640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f90180b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.273+0000 7f901e04d640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901815d700 0x7f90181663d0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59caf284cec60f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901c84a640 1 -- 172.21.15.83:0/3449719446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100b2070 con 0x7f901815d700 2024-05-17T22:08:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901c84a640 1 -- 172.21.15.83:0/3449719446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90100a58a0 con 0x7f901815d700 2024-05-17T22:08:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901c84a640 1 -- 172.21.15.83:0/3449719446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100a5c70 con 0x7f901815d700 2024-05-17T22:08:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- 172.21.15.83:0/3449719446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901815d700 msgr2=0x7f90181663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901815d700 0x7f90181663d0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9010009d90 tx=0x7f901001c990 comp rx=0 tx=0).stop 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- 172.21.15.83:0/3449719446 shutdown_connections 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f901815d700 0x7f90181663d0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 --2- 172.21.15.83:0/3449719446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f90180b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- 172.21.15.83:0/3449719446 >> 172.21.15.83:0/3449719446 conn(0x7f901806cd90 msgr2=0x7f90180b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- 172.21.15.83:0/3449719446 shutdown_connections 2024-05-17T22:08:20.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- 172.21.15.83:0/3449719446 wait complete. 2024-05-17T22:08:20.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 Processor -- start 2024-05-17T22:08:20.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.277+0000 7f901fab0640 1 -- start start 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f9018152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 0x7f9018152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90181692b0 con 0x7f90180cdf40 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9018169130 con 0x7f901815d700 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9018169430 con 0x7f90180cf830 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53686/0 (socket says 172.21.15.83:53686) 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 -- 172.21.15.83:0/2074664382 learned_addr learned my addr 172.21.15.83:0/2074664382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901e04d640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 0x7f9018152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d04b640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f9018152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 -- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 msgr2=0x7f9018152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f9018152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 -- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 msgr2=0x7f9018152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 0x7f9018152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 -- 172.21.15.83:0/2074664382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90180b0410 con 0x7f90180cdf40 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901e04d640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 0x7f9018152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d84c640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f901400eb00 tx=0x7f9014007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014007c80 con 0x7f90180cdf40 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9014007e20 con 0x7f90180cdf40 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014010660 con 0x7f90180cdf40 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90180d1b80 con 0x7f90180cdf40 2024-05-17T22:08:20.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90180b0130 con 0x7f90180cdf40 2024-05-17T22:08:20.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90180b3aa0 con 0x7f90180cdf40 2024-05-17T22:08:20.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f901400b030 con 0x7f90180cdf40 2024-05-17T22:08:20.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 0x7f8ffc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f9014091780 con 0x7f90180cdf40 2024-05-17T22:08:20.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d04b640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 0x7f8ffc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.281+0000 7f901d04b640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 0x7f8ffc071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f90180b69e0 tx=0x7f9008007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.285+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f901405ea30 con 0x7f90180cdf40 2024-05-17T22:08:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.445+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f901815a780 con 0x7f8ffc06f400 2024-05-17T22:08:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f900e7fc640 1 -- 172.21.15.83:0/2074664382 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f901815a780 con 0x7f8ffc06f400 2024-05-17T22:08:20.439 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 msgr2=0x7f8ffc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 0x7f8ffc071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f90180b69e0 tx=0x7f9008007c10 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 msgr2=0x7f901814de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f901400eb00 tx=0x7f9014007040 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 shutdown_connections 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8ffc06f400 0x7f8ffc071910 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f901815d700 0x7f9018152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90180cf830 0x7f9018152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 --2- 172.21.15.83:0/2074664382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90180cdf40 0x7f901814de30 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 >> 172.21.15.83:0/2074664382 conn(0x7f901806cd90 msgr2=0x7f9018154500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 shutdown_connections 2024-05-17T22:08:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.449+0000 7f901fab0640 1 -- 172.21.15.83:0/2074664382 wait complete. 2024-05-17T22:08:20.459 INFO:tasks.ceph:Scrubbing osd.3 2024-05-17T22:08:20.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 Processor -- start 2024-05-17T22:08:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 -- start start 2024-05-17T22:08:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d415fd80 0x7f72d4161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d40d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72d40d49a0 0x7f72d40d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f72d40d2bd0 con 0x7f72d415fd80 2024-05-17T22:08:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f72d40d3c10 con 0x7f72d40d49a0 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d884a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72d40d11e0 con 0x7f72d40d2280 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d415fd80 0x7f72d4161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d415fd80 0x7f72d4161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53690/0 (socket says 172.21.15.83:53690) 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 -- 172.21.15.83:0/380445492 learned_addr learned my addr 172.21.15.83:0/380445492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d2d9d640 1 --2- 172.21.15.83:0/380445492 >> v2:172.21.15.155:3300/0 conn(0x7f72d40d49a0 0x7f72d40d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d1d9b640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d40d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 -- 172.21.15.83:0/380445492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 msgr2=0x7f72d40d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d40d2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 -- 172.21.15.83:0/380445492 >> v2:172.21.15.155:3300/0 conn(0x7f72d40d49a0 msgr2=0x7f72d40d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 --2- 172.21.15.83:0/380445492 >> v2:172.21.15.155:3300/0 conn(0x7f72d40d49a0 0x7f72d40d4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.609+0000 7f72d259c640 1 -- 172.21.15.83:0/380445492 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d40d3ec0 con 0x7f72d415fd80 2024-05-17T22:08:20.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d259c640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d4161330 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f72d408e3e0 tx=0x7f72cc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0ad0a7a3e7ad93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d159a640 1 -- 172.21.15.83:0/380445492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0a5760 con 0x7f72d415fd80 2024-05-17T22:08:20.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d159a640 1 -- 172.21.15.83:0/380445492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc0a5900 con 0x7f72d415fd80 2024-05-17T22:08:20.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d159a640 1 -- 172.21.15.83:0/380445492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0a5cd0 con 0x7f72d415fd80 2024-05-17T22:08:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 -- 172.21.15.83:0/380445492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 msgr2=0x7f72d4161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d4161330 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f72d408e3e0 tx=0x7f72cc01c9f0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 -- 172.21.15.83:0/380445492 shutdown_connections 2024-05-17T22:08:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 --2- 172.21.15.83:0/380445492 >> v2:172.21.15.155:3300/0 conn(0x7f72d40d49a0 0x7f72d40d4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d40d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 --2- 172.21.15.83:0/380445492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d4161330 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 -- 172.21.15.83:0/380445492 >> 172.21.15.83:0/380445492 conn(0x7f72d4074ac0 msgr2=0x7f72d406de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 -- 172.21.15.83:0/380445492 shutdown_connections 2024-05-17T22:08:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.613+0000 7f72d884a640 1 -- 172.21.15.83:0/380445492 wait complete. 2024-05-17T22:08:20.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 Processor -- start 2024-05-17T22:08:20.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 -- start start 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d40d49a0 0x7f72d40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41619f0 con 0x7f72d415fd80 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72d4161870 con 0x7f72d40d49a0 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d884a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f72d40d0cc0 con 0x7f72d40d2280 2024-05-17T22:08:20.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53696/0 (socket says 172.21.15.83:53696) 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 -- 172.21.15.83:0/3731121313 learned_addr learned my addr 172.21.15.83:0/3731121313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d259c640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d1d9b640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d40d49a0 0x7f72d40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 -- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 msgr2=0x7f72d414e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72d40d2280 0x7f72d414e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 -- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d40d49a0 msgr2=0x7f72d40cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d40d49a0 0x7f72d40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.617+0000 7f72d2d9d640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d40cec70 con 0x7f72d415fd80 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d2d9d640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d40cce60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c5c0 tx=0x7f72c400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4015070 con 0x7f72d415fd80 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c4004240 con 0x7f72d415fd80 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d40cef50 con 0x7f72d415fd80 2024-05-17T22:08:20.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003620 con 0x7f72d415fd80 2024-05-17T22:08:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d40cf510 con 0x7f72d415fd80 2024-05-17T22:08:20.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f72d40d4db0 con 0x7f72d415fd80 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f72c40037c0 con 0x7f72d415fd80 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72b406f430 0x7f72b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f72c408f2e0 con 0x7f72d415fd80 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d259c640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72b406f430 0x7f72b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7f72b4075480 0x7f72b40778d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72c4009f20 con 0x7f72b4075480 2024-05-17T22:08:20.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f72c408f770 con 0x7f72d415fd80 2024-05-17T22:08:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d1d9b640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7f72b4075480 0x7f72b40778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d259c640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72b406f430 0x7f72b4071940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0ae000 tx=0x7f72cc0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.621+0000 7f72d1d9b640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7f72b4075480 0x7f72b40778d0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.625+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== osd.3 v2:172.21.15.83:6824/2257217219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72c4009f20 con 0x7f72b4075480 2024-05-17T22:08:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 --> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f72d40cea20 con 0x7f72b4075480 2024-05-17T22:08:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72caffd640 1 -- 172.21.15.83:0/3731121313 <== osd.3 v2:172.21.15.83:6824/2257217219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f72d40cea20 con 0x7f72b4075480 2024-05-17T22:08:20.627 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:20.627 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:20.627 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7f72b4075480 msgr2=0x7f72b40778d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6824/2257217219,v1:172.21.15.83:6825/2257217219] conn(0x7f72b4075480 0x7f72b40778d0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72b406f430 msgr2=0x7f72b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f72b406f430 0x7f72b4071940 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0ae000 tx=0x7f72cc0a7070 comp rx=0 tx=0).stop 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 msgr2=0x7f72d40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 --2- 172.21.15.83:0/3731121313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d415fd80 0x7f72d40cce60 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c5c0 tx=0x7f72c400ca80 comp rx=0 tx=0).stop 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d2d9d640 1 -- 172.21.15.83:0/3731121313 reap_dead start 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 shutdown_connections 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 >> 172.21.15.83:0/3731121313 conn(0x7f72d4074ac0 msgr2=0x7f72d40b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 shutdown_connections 2024-05-17T22:08:20.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.637+0000 7f72d884a640 1 -- 172.21.15.83:0/3731121313 wait complete. 2024-05-17T22:08:20.644 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-17T22:08:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.785+0000 7fd396d6e640 1 Processor -- start 2024-05-17T22:08:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.785+0000 7fd396d6e640 1 -- start start 2024-05-17T22:08:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd396d6e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd3900d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd396d6e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3900d4170 con 0x7fd3900d2000 2024-05-17T22:08:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd396d6e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3900d4450 con 0x7fd3900d0290 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd396d6e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900d4730 con 0x7fd3900b4810 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd3900d2000 0x7fd3900d2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46214/0 (socket says 172.21.15.83:46214) 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 -- 172.21.15.83:0/1939797755 learned_addr learned my addr 172.21.15.83:0/1939797755 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666883090 0 0) 0x7fd3900d4170 con 0x7fd3900d2000 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd378003530 con 0x7fd3900d2000 2024-05-17T22:08:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd394b0a640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd3900d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184018873 0 0) 0x7fd378003530 con 0x7fd3900d2000 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 msgr2=0x7fd3900d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd3900d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 >> v1:172.21.15.155:6789/0 conn(0x7fd3900d0290 legacy=0x7fd39015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3900d4a10 con 0x7fd3900d2000 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7fd388001a80 con 0x7fd3900d2000 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 >> v1:172.21.15.83:6789/0 conn(0x7fd3900d2000 legacy=0x7fd3900d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd378003a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd378004010 0x7fd378004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd378004d50 0x7fd378005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900d4730 con 0x7fd378004d50 2024-05-17T22:08:20.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd37800bc10 con 0x7fd378004010 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd37800be10 con 0x7fd3900b4810 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd378004d50 0x7fd378005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd394b0a640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd378003a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd38ffff640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd378004010 0x7fd378004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 -- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 msgr2=0x7fd378003a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd3900b4810 0x7fd378003a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 -- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd378004010 msgr2=0x7fd378004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd378004010 0x7fd378004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.789+0000 7fd39530b640 1 -- 172.21.15.83:0/1939797755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd378003530 con 0x7fd378004d50 2024-05-17T22:08:20.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd39530b640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd378004d50 0x7fd378005200 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd3880013e0 tx=0x7fd38801d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba68195327360eb server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800a480 con 0x7fd378004d50 2024-05-17T22:08:20.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3880ad630 con 0x7fd378004d50 2024-05-17T22:08:20.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd38f7fe640 1 -- 172.21.15.83:0/1939797755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880ada20 con 0x7fd378004d50 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- 172.21.15.83:0/1939797755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd378004d50 msgr2=0x7fd378005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 --2- 172.21.15.83:0/1939797755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd378004d50 0x7fd378005200 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd3880013e0 tx=0x7fd38801d280 comp rx=0 tx=0).stop 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd39530b640 1 -- 172.21.15.83:0/1939797755 reap_dead start 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- 172.21.15.83:0/1939797755 shutdown_connections 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- 172.21.15.83:0/1939797755 >> 172.21.15.83:0/1939797755 conn(0x7fd39006e010 msgr2=0x7fd39006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- 172.21.15.83:0/1939797755 shutdown_connections 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- 172.21.15.83:0/1939797755 wait complete. 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 Processor -- start 2024-05-17T22:08:20.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.793+0000 7fd396d6e640 1 -- start start 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd390161f20 0x7fd3900cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 0x7fd3901eb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900d5210 con 0x7fd3901eb9d0 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900d06a0 con 0x7fd3901f9dc0 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900d5390 con 0x7fd390161f20 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53718/0 (socket says 172.21.15.83:53718) 2024-05-17T22:08:20.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 -- 172.21.15.83:0/319473589 learned_addr learned my addr 172.21.15.83:0/319473589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd394b0a640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd390161f20 0x7fd3900cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38ffff640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 0x7fd3901eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 -- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd390161f20 msgr2=0x7fd3900cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd390161f20 0x7fd3900cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 -- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 msgr2=0x7fd3901eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 0x7fd3901eb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 -- 172.21.15.83:0/319473589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901624e0 con 0x7fd3901eb9d0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38ffff640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 0x7fd3901eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd39530b640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd38801db20 tx=0x7fd378005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880bb070 con 0x7fd3901eb9d0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38800a460 con 0x7fd3901eb9d0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388001ff0 con 0x7fd3901eb9d0 2024-05-17T22:08:20.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901627c0 con 0x7fd3901eb9d0 2024-05-17T22:08:20.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd390162d80 con 0x7fd3901eb9d0 2024-05-17T22:08:20.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900d3fd0 con 0x7fd3901eb9d0 2024-05-17T22:08:20.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3880be070 con 0x7fd3901eb9d0 2024-05-17T22:08:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 0x7fd37c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fd3881293e0 con 0x7fd3901eb9d0 2024-05-17T22:08:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.797+0000 7fd394b0a640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 0x7fd37c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:20.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.801+0000 7fd394b0a640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 0x7fd37c071940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd3900d2530 tx=0x7fd380008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:20.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.801+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fd3880c5020 con 0x7fd3901eb9d0 2024-05-17T22:08:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.961+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fd39015f4e0 con 0x7fd37c06f430 2024-05-17T22:08:20.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.961+0000 7fd38d7fa640 1 -- 172.21.15.83:0/319473589 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd39015f4e0 con 0x7fd37c06f430 2024-05-17T22:08:20.953 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 msgr2=0x7fd37c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 0x7fd37c071940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd3900d2530 tx=0x7fd380008040 comp rx=0 tx=0).stop 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 msgr2=0x7fd390160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd38801db20 tx=0x7fd378005840 comp rx=0 tx=0).stop 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 shutdown_connections 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3901eb9d0 0x7fd390160b20 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fd37c06f430 0x7fd37c071940 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd3901f9dc0 0x7fd3901eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 --2- 172.21.15.83:0/319473589 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd390161f20 0x7fd3900cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 >> 172.21.15.83:0/319473589 conn(0x7fd39006e010 msgr2=0x7fd3900b8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 shutdown_connections 2024-05-17T22:08:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:20.965+0000 7fd396d6e640 1 -- 172.21.15.83:0/319473589 wait complete. 2024-05-17T22:08:20.974 INFO:tasks.ceph:Scrubbing osd.4 2024-05-17T22:08:20.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:21.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 Processor -- start 2024-05-17T22:08:21.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 -- start start 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f824c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c15ec60 con 0x7f824c15df30 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c0b5be0 con 0x7f824c072e50 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f82508ff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0b6230 con 0x7f824c0b66c0 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824ad9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f824c15df30 0x7f824c15e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46220/0 (socket says 172.21.15.83:46220) 2024-05-17T22:08:21.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824ad9d640 1 -- 172.21.15.83:0/3112217703 learned_addr learned my addr 172.21.15.83:0/3112217703 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824a59c640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f824c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216858039 0 0) 0x7f824c15ec60 con 0x7f824c15df30 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8238003530 con 0x7f824c15df30 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3573655206 0 0) 0x7f8238003530 con 0x7f824c15df30 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 msgr2=0x7f824c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f824c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 >> v1:172.21.15.155:6789/0 conn(0x7f824c072e50 legacy=0x7f824c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0b5ec0 con 0x7f824c15df30 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f823c001a80 con 0x7f824c15df30 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 >> v1:172.21.15.83:6789/0 conn(0x7f824c15df30 legacy=0x7f824c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8238003a40 0x7f8238003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f8238004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f82380052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0b6230 con 0x7f8238004810 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f823800bb90 con 0x7f8238003a40 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.125+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f823800bd90 con 0x7f824c0b66c0 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f8238004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824a59c640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8238003a40 0x7f8238003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824ad9d640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f82380052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 -- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 msgr2=0x7f82380052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c0b66c0 0x7f82380052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 -- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8238003a40 msgr2=0x7f8238003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8238003a40 0x7f8238003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 -- 172.21.15.83:0/3112217703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8238003530 con 0x7f8238004810 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f8249d9b640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f8238004c20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8234001730 tx=0x7f823401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8563d69825ae244e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234001380 con 0x7f8238004810 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82340052d0 con 0x7f8238004810 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824959a640 1 -- 172.21.15.83:0/3112217703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234003840 con 0x7f8238004810 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 -- 172.21.15.83:0/3112217703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 msgr2=0x7f8238004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 --2- 172.21.15.83:0/3112217703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f8238004c20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8234001730 tx=0x7f823401c3e0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f824ad9d640 1 -- 172.21.15.83:0/3112217703 reap_dead start 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 -- 172.21.15.83:0/3112217703 shutdown_connections 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 -- 172.21.15.83:0/3112217703 >> 172.21.15.83:0/3112217703 conn(0x7f824c06cd90 msgr2=0x7f824c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 -- 172.21.15.83:0/3112217703 shutdown_connections 2024-05-17T22:08:21.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.129+0000 7f82508ff640 1 -- 172.21.15.83:0/3112217703 wait complete. 2024-05-17T22:08:21.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 Processor -- start 2024-05-17T22:08:21.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- start start 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f824c13f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c141b30 0x7f824c141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824c1f3d30 0x7f824c1f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0b0f50 con 0x7f8238004810 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0b0dd0 con 0x7f824c1f3d30 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0b44c0 con 0x7f824c141b30 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f824c13f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f824c13f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53728/0 (socket says 172.21.15.83:53728) 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 -- 172.21.15.83:0/2130555783 learned_addr learned my addr 172.21.15.83:0/2130555783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f8249d9b640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c141b30 0x7f824c141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824ad9d640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824c1f3d30 0x7f824c1f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 -- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c141b30 msgr2=0x7f824c141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f824c141b30 0x7f824c141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 -- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824c1f3d30 msgr2=0x7f824c1f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824c1f3d30 0x7f824c1f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c142820 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824ad9d640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f824c1f3d30 0x7f824c1f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f824a59c640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f824c13f990 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8244011610 tx=0x7f8244011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400f8f0 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824400fa90 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c142b00 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400fe40 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.133+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1430c0 con 0x7f8238004810 2024-05-17T22:08:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f824c073260 con 0x7f8238004810 2024-05-17T22:08:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8244015460 con 0x7f8238004810 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f822c06f400 0x7f822c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8249d9b640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f822c06f400 0x7f822c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f824408fda0 con 0x7f8238004810 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f822c075450 0x7f822c0778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8244013e50 con 0x7f822c075450 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f8244090230 con 0x7f8238004810 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f8249d9b640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f822c06f400 0x7f822c071910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f82340ac000 tx=0x7f82340011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f824ad9d640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f822c075450 0x7f822c0778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.137+0000 7f824ad9d640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f822c075450 0x7f822c0778a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.141+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== osd.4 v2:172.21.15.155:6824/505763805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8244013e50 con 0x7f822c075450 2024-05-17T22:08:21.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 --> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f824c0b3dd0 con 0x7f822c075450 2024-05-17T22:08:21.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f8242ffd640 1 -- 172.21.15.83:0/2130555783 <== osd.4 v2:172.21.15.155:6824/505763805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f824c0b3dd0 con 0x7f822c075450 2024-05-17T22:08:21.144 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:21.144 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:21.145 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:21.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f822c075450 msgr2=0x7f822c0778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.155:6824/505763805,v1:172.21.15.155:6825/505763805] conn(0x7f822c075450 0x7f822c0778a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f822c06f400 msgr2=0x7f822c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f822c06f400 0x7f822c071910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f82340ac000 tx=0x7f82340011a0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 msgr2=0x7f824c13f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 --2- 172.21.15.83:0/2130555783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8238004810 0x7f824c13f990 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8244011610 tx=0x7f8244011ad0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f824ad9d640 1 -- 172.21.15.83:0/2130555783 reap_dead start 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 shutdown_connections 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 >> 172.21.15.83:0/2130555783 conn(0x7f824c06cd90 msgr2=0x7f824c146800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 shutdown_connections 2024-05-17T22:08:21.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.153+0000 7f82508ff640 1 -- 172.21.15.83:0/2130555783 wait complete. 2024-05-17T22:08:21.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-17T22:08:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 Processor -- start 2024-05-17T22:08:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 -- start start 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef78072410 0x7fef781686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7809c710 con 0x7fef7809ba00 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7809c9f0 con 0x7fef780b88b0 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7e9a8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fef780d1a40 con 0x7fef78072410 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef78072410 0x7fef781686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef78072410 0x7fef781686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53740/0 (socket says 172.21.15.83:53740) 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef6f7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef7809ba00 0x7fef7809be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46222/0 (socket says 172.21.15.83:46222) 2024-05-17T22:08:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 -- 172.21.15.83:0/3138234335 learned_addr learned my addr 172.21.15.83:0/3138234335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 -- 172.21.15.83:0/3138234335 >> v1:172.21.15.155:6789/0 conn(0x7fef780b88b0 legacy=0x7fef780d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 -- 172.21.15.83:0/3138234335 >> v1:172.21.15.83:6789/0 conn(0x7fef7809ba00 legacy=0x7fef7809be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef7cf45640 1 -- 172.21.15.83:0/3138234335 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7809ccd0 con 0x7fef78072410 2024-05-17T22:08:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.305+0000 7fef777fe640 1 -- 172.21.15.83:0/3138234335 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028559752 0 0) 0x7fef7809c710 con 0x7fef7809ba00 2024-05-17T22:08:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7cf45640 1 --2- 172.21.15.83:0/3138234335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef781686e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fef780d1bc0 tx=0x7fef7001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838e4b0b578aa4aa server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef777fe640 1 -- 172.21.15.83:0/3138234335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700a56c0 con 0x7fef78072410 2024-05-17T22:08:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef777fe640 1 -- 172.21.15.83:0/3138234335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef700a5860 con 0x7fef78072410 2024-05-17T22:08:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef777fe640 1 -- 172.21.15.83:0/3138234335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700a5c30 con 0x7fef78072410 2024-05-17T22:08:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- 172.21.15.83:0/3138234335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 msgr2=0x7fef781686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/3138234335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef781686e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fef780d1bc0 tx=0x7fef7001c8f0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- 172.21.15.83:0/3138234335 shutdown_connections 2024-05-17T22:08:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/3138234335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef781686e0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- 172.21.15.83:0/3138234335 >> 172.21.15.83:0/3138234335 conn(0x7fef780d3620 msgr2=0x7fef780d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- 172.21.15.83:0/3138234335 shutdown_connections 2024-05-17T22:08:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- 172.21.15.83:0/3138234335 wait complete. 2024-05-17T22:08:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 Processor -- start 2024-05-17T22:08:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.309+0000 7fef7e9a8640 1 -- start start 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef7809ba00 0x7fef780b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef780b88b0 0x7fef780b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef7816bc70 con 0x7fef78072410 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef7809d040 con 0x7fef780b88b0 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fef7816bdf0 con 0x7fef7809ba00 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53748/0 (socket says 172.21.15.83:53748) 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 -- 172.21.15.83:0/2716862357 learned_addr learned my addr 172.21.15.83:0/2716862357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef6f7fe640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef7809ba00 0x7fef780b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7cf45640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef780b88b0 0x7fef780b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 -- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef7809ba00 msgr2=0x7fef780b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef7809ba00 0x7fef780b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 -- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef780b88b0 msgr2=0x7fef780b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef780b88b0 0x7fef780b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 -- 172.21.15.83:0/2716862357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef780cc980 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef77fff640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b410 tx=0x7fef6800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68015030 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68004020 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef780ccc60 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68004450 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef780cd220 con 0x7fef78072410 2024-05-17T22:08:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef780d11d0 con 0x7fef78072410 2024-05-17T22:08:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fef68018070 con 0x7fef78072410 2024-05-17T22:08:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 0x7fef58071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fef6808f3d0 con 0x7fef78072410 2024-05-17T22:08:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.313+0000 7fef6f7fe640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 0x7fef58071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.317+0000 7fef6f7fe640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 0x7fef58071940 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fef780ca5c0 tx=0x7fef60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.317+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fef6805c5f0 con 0x7fef78072410 2024-05-17T22:08:21.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.465+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fef7815a2f0 con 0x7fef5806f430 2024-05-17T22:08:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.465+0000 7fef757fa640 1 -- 172.21.15.83:0/2716862357 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fef7815a2f0 con 0x7fef5806f430 2024-05-17T22:08:21.457 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 msgr2=0x7fef58071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 0x7fef58071940 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fef780ca5c0 tx=0x7fef60009040 comp rx=0 tx=0).stop 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 msgr2=0x7fef78158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b410 tx=0x7fef6800b8d0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 shutdown_connections 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fef5806f430 0x7fef58071940 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef780b88b0 0x7fef780b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef7809ba00 0x7fef780b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 --2- 172.21.15.83:0/2716862357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef78072410 0x7fef78158f30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 >> 172.21.15.83:0/2716862357 conn(0x7fef780d3620 msgr2=0x7fef78071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 shutdown_connections 2024-05-17T22:08:21.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.469+0000 7fef7e9a8640 1 -- 172.21.15.83:0/2716862357 wait complete. 2024-05-17T22:08:21.478 INFO:tasks.ceph:Scrubbing osd.5 2024-05-17T22:08:21.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.625+0000 7f77d9927640 1 Processor -- start 2024-05-17T22:08:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.625+0000 7f77d9927640 1 -- start start 2024-05-17T22:08:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 0x7f77d4161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 0x7f77d40d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d40d49a0 0x7f77d40d4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f77d40d2bd0 con 0x7f77d40d49a0 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f77d40d3c10 con 0x7f77d415fd80 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77d40d11e0 con 0x7f77d40d2280 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d40d49a0 0x7f77d40d4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d40d49a0 0x7f77d40d4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53750/0 (socket says 172.21.15.83:53750) 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 -- 172.21.15.83:0/3204707702 learned_addr learned my addr 172.21.15.83:0/3204707702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 -- 172.21.15.83:0/3204707702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 msgr2=0x7f77d40d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3204707702 >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 0x7f77d4161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d27fc640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 0x7f77d40d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 0x7f77d40d2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 -- 172.21.15.83:0/3204707702 >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 msgr2=0x7f77d4161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3204707702 >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 0x7f77d4161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 -- 172.21.15.83:0/3204707702 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d40d3ec0 con 0x7f77d40d49a0 2024-05-17T22:08:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d27fc640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 0x7f77d40d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3204707702 >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 0x7f77d4161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40d4db0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f77d40b8370 tx=0x7f77cc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8143c86a8d28be6c server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d1ffb640 1 -- 172.21.15.83:0/3204707702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0a5760 con 0x7f77d40d49a0 2024-05-17T22:08:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d1ffb640 1 -- 172.21.15.83:0/3204707702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0a5900 con 0x7f77d40d49a0 2024-05-17T22:08:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d1ffb640 1 -- 172.21.15.83:0/3204707702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0a5cd0 con 0x7f77d40d49a0 2024-05-17T22:08:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 -- 172.21.15.83:0/3204707702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 msgr2=0x7f77d40d4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.629+0000 7f77d9927640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40d4db0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f77d40b8370 tx=0x7f77cc01c9f0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- 172.21.15.83:0/3204707702 shutdown_connections 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40d4db0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- 172.21.15.83:0/3204707702 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d40d2280 0x7f77d40d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- 172.21.15.83:0/3204707702 >> v2:172.21.15.155:3300/0 conn(0x7f77d415fd80 0x7f77d4161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- 172.21.15.83:0/3204707702 >> 172.21.15.83:0/3204707702 conn(0x7f77d4074ac0 msgr2=0x7f77d406de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- 172.21.15.83:0/3204707702 shutdown_connections 2024-05-17T22:08:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- 172.21.15.83:0/3204707702 wait complete. 2024-05-17T22:08:21.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 Processor -- start 2024-05-17T22:08:21.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- start start 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40d2280 0x7f77d40cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415fd80 0x7f77d4158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41619f0 con 0x7f77d40d49a0 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4161870 con 0x7f77d40d2280 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f77d40d0cc0 con 0x7f77d415fd80 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53764/0 (socket says 172.21.15.83:53764) 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 -- 172.21.15.83:0/3004235116 learned_addr learned my addr 172.21.15.83:0/3004235116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415fd80 0x7f77d4158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40d2280 0x7f77d40cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 -- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415fd80 msgr2=0x7f77d4158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415fd80 0x7f77d4158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 -- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40d2280 msgr2=0x7f77d40cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40d2280 0x7f77d40cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d40cd330 con 0x7f77d40d49a0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40d2280 0x7f77d40cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d27fc640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40caf90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c7f0 tx=0x7f77c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8015070 con 0x7f77d40d49a0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8007820 con 0x7f77d40d49a0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.633+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d40cd830 con 0x7f77d40d49a0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8007c10 con 0x7f77d40d49a0 2024-05-17T22:08:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d40cddf0 con 0x7f77d40d49a0 2024-05-17T22:08:21.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77d40d4db0 con 0x7f77d40d49a0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77c8018070 con 0x7f77d40d49a0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f77c006f400 0x7f77c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f77c808f7a0 con 0x7f77d40d49a0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f77c006f400 0x7f77c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7f77c0075450 0x7f77c00778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77c800ee50 con 0x7f77c0075450 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f77c808fc30 con 0x7f77d40d49a0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d2ffd640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f77c006f400 0x7f77c0071910 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b840 tx=0x7f77c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7f77c0075450 0x7f77c00778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.637+0000 7f77d37fe640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7f77c0075450 0x7f77c00778a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.641+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== osd.5 v2:172.21.15.155:6808/758721566 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77c800ee50 con 0x7f77c0075450 2024-05-17T22:08:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 --> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f77d40cd5e0 con 0x7f77c0075450 2024-05-17T22:08:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77bbfff640 1 -- 172.21.15.83:0/3004235116 <== osd.5 v2:172.21.15.155:6808/758721566 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f77d40cd5e0 con 0x7f77c0075450 2024-05-17T22:08:21.646 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:21.646 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:21.646 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7f77c0075450 msgr2=0x7f77c00778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.155:6808/758721566,v1:172.21.15.155:6809/758721566] conn(0x7f77c0075450 0x7f77c00778a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f77c006f400 msgr2=0x7f77c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f77c006f400 0x7f77c0071910 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b840 tx=0x7f77c4008040 comp rx=0 tx=0).stop 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 msgr2=0x7f77d40caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 --2- 172.21.15.83:0/3004235116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d40d49a0 0x7f77d40caf90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f77c800c7f0 tx=0x7f77c800ccb0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d37fe640 1 -- 172.21.15.83:0/3004235116 reap_dead start 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 shutdown_connections 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 >> 172.21.15.83:0/3004235116 conn(0x7f77d4074ac0 msgr2=0x7f77d40b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 shutdown_connections 2024-05-17T22:08:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.653+0000 7f77d9927640 1 -- 172.21.15.83:0/3004235116 wait complete. 2024-05-17T22:08:21.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-17T22:08:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.809+0000 7f8c02dd9640 1 Processor -- start 2024-05-17T22:08:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.809+0000 7f8c02dd9640 1 -- start start 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c02dd9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8bfc0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c02dd9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8bfc143f20 0x7f8bfc072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c02dd9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bfc0737d0 con 0x7f8bfc0733c0 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c02dd9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8bfc0b4110 con 0x7f8bfc143f20 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c02dd9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc070770 con 0x7f8bfc0b3d00 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8bfc0733c0 0x7f8bfc06eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46234/0 (socket says 172.21.15.83:46234) 2024-05-17T22:08:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 -- 172.21.15.83:0/3658799972 learned_addr learned my addr 172.21.15.83:0/3658799972 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c00b75640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8bfc0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3658799972 >> v2:172.21.15.155:3300/0 conn(0x7f8bfc143f20 0x7f8bfc072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850843217 0 0) 0x7f8bfc0737d0 con 0x7f8bfc0733c0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be4003530 con 0x7f8bfc0733c0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4150813723 0 0) 0x7f8be4003530 con 0x7f8bfc0733c0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 msgr2=0x7f8bfc0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8bfc0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 >> v2:172.21.15.155:3300/0 conn(0x7f8bfc143f20 msgr2=0x7f8bfc072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3658799972 >> v2:172.21.15.155:3300/0 conn(0x7f8bfc143f20 0x7f8bfc072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc0d10d0 con 0x7f8bfc0733c0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f8bf4001a80 con 0x7f8bfc0733c0 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 >> v1:172.21.15.83:6789/0 conn(0x7f8bfc0733c0 legacy=0x7f8bfc06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4003a10 0x7f8be4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8be4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4004d50 0x7f8be4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc0b4110 con 0x7f8be4004d50 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc070770 con 0x7f8be4003a10 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8be400ba80 con 0x7f8bfc0b3d00 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4004d50 0x7f8be4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c00b75640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4003a10 0x7f8be4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8be4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 -- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 msgr2=0x7f8be4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8bfc0b3d00 0x7f8be4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 -- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4003a10 msgr2=0x7f8be4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8be4003a10 0x7f8be4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 -- 172.21.15.83:0/3658799972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4003530 con 0x7f8be4004d50 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.813+0000 7f8c01376640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4004d50 0x7f8be4005160 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400a8f0 tx=0x7f8bf401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7c8ab9aaffc64a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400a4c0 con 0x7f8be4004d50 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf40ad630 con 0x7f8be4004d50 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8bfb7fe640 1 -- 172.21.15.83:0/3658799972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40ada20 con 0x7f8be4004d50 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- 172.21.15.83:0/3658799972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4004d50 msgr2=0x7f8be4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/3658799972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4004d50 0x7f8be4005160 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400a8f0 tx=0x7f8bf401d380 comp rx=0 tx=0).stop 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c01376640 1 -- 172.21.15.83:0/3658799972 reap_dead start 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- 172.21.15.83:0/3658799972 shutdown_connections 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- 172.21.15.83:0/3658799972 >> 172.21.15.83:0/3658799972 conn(0x7f8bfc06cd90 msgr2=0x7f8bfc0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- 172.21.15.83:0/3658799972 shutdown_connections 2024-05-17T22:08:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- 172.21.15.83:0/3658799972 wait complete. 2024-05-17T22:08:21.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 Processor -- start 2024-05-17T22:08:21.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.817+0000 7f8c02dd9640 1 -- start start 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be4004d50 0x7f8bfc140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc0aa310 0x7f8bfc0aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc070400 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc145800 con 0x7f8bfc0aa310 2024-05-17T22:08:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc147600 con 0x7f8be4004d50 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53776/0 (socket says 172.21.15.83:53776) 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 -- 172.21.15.83:0/861143729 learned_addr learned my addr 172.21.15.83:0/861143729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c00b75640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be4004d50 0x7f8bfc140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c01376640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc0aa310 0x7f8bfc0aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 -- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be4004d50 msgr2=0x7f8bfc140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be4004d50 0x7f8bfc140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 -- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc0aa310 msgr2=0x7f8bfc0aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc0aa310 0x7f8bfc0aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 -- 172.21.15.83:0/861143729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc0aac60 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bfbfff640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0011600 tx=0x7f8bf0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0004220 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf00043c0 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc0b0300 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000f960 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1f76f0 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc0b0070 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bf000fb00 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 0x7f8be8071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f8bf008f2b0 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c00b75640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 0x7f8be8071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.821+0000 7f8c00b75640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 0x7f8be8071990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0098f0 tx=0x7f8bec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:21.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.825+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8bfc0aac60 con 0x7f8bfc1f7ef0 2024-05-17T22:08:21.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8bfc069c70 con 0x7f8be806f480 2024-05-17T22:08:21.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8bf97fa640 1 -- 172.21.15.83:0/861143729 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8bfc069c70 con 0x7f8be806f480 2024-05-17T22:08:21.968 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 msgr2=0x7f8be8071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 0x7f8be8071990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0098f0 tx=0x7f8bec008040 comp rx=0 tx=0).stop 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 msgr2=0x7f8bfc160d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0011600 tx=0x7f8bf0011ac0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 shutdown_connections 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f8be806f480 0x7f8be8071990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bfc0aa310 0x7f8bfc0aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc1f7ef0 0x7f8bfc160d20 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 --2- 172.21.15.83:0/861143729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8be4004d50 0x7f8bfc140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:21.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 >> 172.21.15.83:0/861143729 conn(0x7f8bfc06cd90 msgr2=0x7f8bfc1445e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:21.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 shutdown_connections 2024-05-17T22:08:21.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:21.977+0000 7f8c02dd9640 1 -- 172.21.15.83:0/861143729 wait complete. 2024-05-17T22:08:21.989 INFO:tasks.ceph:Scrubbing osd.6 2024-05-17T22:08:21.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.137+0000 7f9f75da1640 1 Processor -- start 2024-05-17T22:08:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.137+0000 7f9f75da1640 1 -- start start 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f75da1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7006cea0 0x7f9f700ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f75da1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7006f7c0 con 0x7f9f700b3520 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f75da1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f700b3c90 con 0x7f9f7015eee0 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f75da1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f700b3f30 con 0x7f9f7006cea0 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6ffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9f700b3520 0x7f9f700ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46242/0 (socket says 172.21.15.83:46242) 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6ffff640 1 -- 172.21.15.83:0/2128931 learned_addr learned my addr 172.21.15.83:0/2128931 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:22.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6effd640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7006cea0 0x7f9f700ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963912354 0 0) 0x7f9f7006f7c0 con 0x7f9f700b3520 2024-05-17T22:08:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f58003530 con 0x7f9f700b3520 2024-05-17T22:08:22.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345610234 0 0) 0x7f9f700b3c90 con 0x7f9f7015eee0 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f7006f7c0 con 0x7f9f7015eee0 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3262659924 0 0) 0x7f9f58003530 con 0x7f9f700b3520 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7006cea0 msgr2=0x7f9f700ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7006cea0 0x7f9f700ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 >> v1:172.21.15.155:6789/0 conn(0x7f9f7015eee0 legacy=0x7f9f7006c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7015e460 con 0x7f9f700b3520 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f9f68001a80 con 0x7f9f700b3520 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 >> v1:172.21.15.83:6789/0 conn(0x7f9f700b3520 legacy=0x7f9f700ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f58004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58004810 0x7f9f58004ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f580053b0 0x7f9f5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f700b3f30 con 0x7f9f58003c20 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5800c140 con 0x7f9f58004810 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5800c340 con 0x7f9f580053b0 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f58004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6effd640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58004810 0x7f9f58004ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6ffff640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f580053b0 0x7f9f5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f580053b0 msgr2=0x7f9f5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f580053b0 0x7f9f5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58004810 msgr2=0x7f9f58004ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f58004810 0x7f9f58004ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.141+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/2128931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f58003530 con 0x7f9f58003c20 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f58004070 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9f60001920 tx=0x7f9f6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13327f74e3d20c2a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f600a8050 con 0x7f9f58003c20 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f600052f0 con 0x7f9f58003c20 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f6e7fc640 1 -- 172.21.15.83:0/2128931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60001570 con 0x7f9f58003c20 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- 172.21.15.83:0/2128931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 msgr2=0x7f9f58004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 --2- 172.21.15.83:0/2128931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f58004070 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9f60001920 tx=0x7f9f6001c520 comp rx=0 tx=0).stop 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f6ffff640 1 -- 172.21.15.83:0/2128931 reap_dead start 2024-05-17T22:08:22.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- 172.21.15.83:0/2128931 shutdown_connections 2024-05-17T22:08:22.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- 172.21.15.83:0/2128931 >> 172.21.15.83:0/2128931 conn(0x7f9f700b4100 msgr2=0x7f9f700b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- 172.21.15.83:0/2128931 shutdown_connections 2024-05-17T22:08:22.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- 172.21.15.83:0/2128931 wait complete. 2024-05-17T22:08:22.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 Processor -- start 2024-05-17T22:08:22.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- start start 2024-05-17T22:08:22.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f700ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f701f6580 0x7f9f70158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70162950 0x7f9f70162d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7006cea0 con 0x7f9f58003c20 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f700b4e40 con 0x7f9f70162950 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.145+0000 7f9f75da1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7006d020 con 0x7f9f701f6580 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f700ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f700ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53792/0 (socket says 172.21.15.83:53792) 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/3948321931 learned_addr learned my addr 172.21.15.83:0/3948321931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6effd640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f701f6580 0x7f9f70158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6ffff640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70162950 0x7f9f70162d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f701f6580 msgr2=0x7f9f70158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f701f6580 0x7f9f70158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70162950 msgr2=0x7f9f70162d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70162950 0x7f9f70162d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f700cc4d0 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6f7fe640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f700ca800 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005040 tx=0x7f9f58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f600a8050 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f600ba070 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f701632a0 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60007a40 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f701f6c90 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f700b59c0 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9f60005270 con 0x7f9f58003c20 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f5c06f400 0x7f9f5c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f9f601299a0 con 0x7f9f58003c20 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6effd640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f5c06f400 0x7f9f5c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7f9f5c075450 0x7f9f5c0778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f5c077f70 con 0x7f9f5c075450 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f9f60129e30 con 0x7f9f58003c20 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6effd640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f5c06f400 0x7f9f5c071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f70068bb0 tx=0x7f9f64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.149+0000 7f9f6ffff640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7f9f5c075450 0x7f9f5c0778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.153+0000 7f9f6ffff640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7f9f5c075450 0x7f9f5c0778a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.153+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== osd.6 v2:172.21.15.155:6800/1025803281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f5c077f70 con 0x7f9f5c075450 2024-05-17T22:08:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.165+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 --> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9f701f71c0 con 0x7f9f5c075450 2024-05-17T22:08:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.165+0000 7f9f57fff640 1 -- 172.21.15.83:0/3948321931 <== osd.6 v2:172.21.15.155:6800/1025803281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9f701f71c0 con 0x7f9f5c075450 2024-05-17T22:08:22.158 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:22.158 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:22.158 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.165+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7f9f5c075450 msgr2=0x7f9f5c0778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.165+0000 7f9f75da1640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.155:6800/1025803281,v1:172.21.15.155:6801/1025803281] conn(0x7f9f5c075450 0x7f9f5c0778a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f5c06f400 msgr2=0x7f9f5c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9f5c06f400 0x7f9f5c071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f70068bb0 tx=0x7f9f64008040 comp rx=0 tx=0).stop 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 msgr2=0x7f9f700ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 --2- 172.21.15.83:0/3948321931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f58003c20 0x7f9f700ca800 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005040 tx=0x7f9f58005840 comp rx=0 tx=0).stop 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f6ffff640 1 -- 172.21.15.83:0/3948321931 reap_dead start 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 shutdown_connections 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 >> 172.21.15.83:0/3948321931 conn(0x7f9f700b4100 msgr2=0x7f9f70163800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 shutdown_connections 2024-05-17T22:08:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.169+0000 7f9f75da1640 1 -- 172.21.15.83:0/3948321931 wait complete. 2024-05-17T22:08:22.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-17T22:08:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.317+0000 7fde3b046640 1 Processor -- start 2024-05-17T22:08:22.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 -- start start 2024-05-17T22:08:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde340b2f70 0x7fde340d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde340ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde3406c930 con 0x7fde340cd4a0 2024-05-17T22:08:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fde3406d540 con 0x7fde340b2f70 2024-05-17T22:08:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde3b046640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde3406cc10 con 0x7fde340d1d80 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde38de2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde340b2f70 0x7fde340d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde395e3640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde340cd4a0 0x7fde340cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46246/0 (socket says 172.21.15.83:46246) 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde395e3640 1 -- 172.21.15.83:0/4124156167 learned_addr learned my addr 172.21.15.83:0/4124156167 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde38de2640 1 -- 172.21.15.83:0/4124156167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 msgr2=0x7fde340ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde38de2640 1 --2- 172.21.15.83:0/4124156167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde340ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde38de2640 1 -- 172.21.15.83:0/4124156167 >> v1:172.21.15.83:6789/0 conn(0x7fde340cd4a0 legacy=0x7fde340cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde337fe640 1 -- 172.21.15.83:0/4124156167 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46163289 0 0) 0x7fde3406c930 con 0x7fde340cd4a0 2024-05-17T22:08:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.321+0000 7fde38de2640 1 -- 172.21.15.83:0/4124156167 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3406cef0 con 0x7fde340b2f70 2024-05-17T22:08:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde38de2640 1 --2- 172.21.15.83:0/4124156167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340b2f70 0x7fde340d1840 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fde280097c0 tx=0x7fde2801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c54629b1edbdb3a server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde337fe640 1 -- 172.21.15.83:0/4124156167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280b0070 con 0x7fde340b2f70 2024-05-17T22:08:22.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde337fe640 1 -- 172.21.15.83:0/4124156167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde280ab600 con 0x7fde340b2f70 2024-05-17T22:08:22.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde337fe640 1 -- 172.21.15.83:0/4124156167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280ab9d0 con 0x7fde340b2f70 2024-05-17T22:08:22.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- 172.21.15.83:0/4124156167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340b2f70 msgr2=0x7fde340d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 --2- 172.21.15.83:0/4124156167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340b2f70 0x7fde340d1840 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fde280097c0 tx=0x7fde2801c410 comp rx=0 tx=0).stop 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- 172.21.15.83:0/4124156167 shutdown_connections 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 --2- 172.21.15.83:0/4124156167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde340ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 --2- 172.21.15.83:0/4124156167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340b2f70 0x7fde340d1840 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- 172.21.15.83:0/4124156167 >> 172.21.15.83:0/4124156167 conn(0x7fde340b4170 msgr2=0x7fde340b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- 172.21.15.83:0/4124156167 shutdown_connections 2024-05-17T22:08:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- 172.21.15.83:0/4124156167 wait complete. 2024-05-17T22:08:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 Processor -- start 2024-05-17T22:08:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.325+0000 7fde3b046640 1 -- start start 2024-05-17T22:08:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde340b2f70 0x7fde340b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde34156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde3416cb00 con 0x7fde340cd4a0 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde3416cc80 con 0x7fde340b2f70 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fde3416ce00 con 0x7fde340d1d80 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53806/0 (socket says 172.21.15.83:53806) 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 -- 172.21.15.83:0/323516483 learned_addr learned my addr 172.21.15.83:0/323516483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 -- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 msgr2=0x7fde34156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde34156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 -- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde340b2f70 msgr2=0x7fde340b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde340b2f70 0x7fde340b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 -- 172.21.15.83:0/323516483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3415bd40 con 0x7fde340cd4a0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde33fff640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fde2000b410 tx=0x7fde2000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20015020 con 0x7fde340cd4a0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20011830 con 0x7fde340cd4a0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3415c020 con 0x7fde340cd4a0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20011c20 con 0x7fde340cd4a0 2024-05-17T22:08:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde340b0420 con 0x7fde340cd4a0 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde340d1840 con 0x7fde340cd4a0 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde20018020 con 0x7fde340cd4a0 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 0x7fde18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fde2008eff0 con 0x7fde340cd4a0 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde38de2640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 0x7fde18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.329+0000 7fde38de2640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 0x7fde18071940 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fde280ad000 tx=0x7fde28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.333+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7fde2001f020 con 0x7fde340cd4a0 2024-05-17T22:08:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fde340cdb70 con 0x7fde1806f430 2024-05-17T22:08:22.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde317fa640 1 -- 172.21.15.83:0/323516483 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fde340cdb70 con 0x7fde1806f430 2024-05-17T22:08:22.476 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 6 to deep-scrub 2024-05-17T22:08:22.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 msgr2=0x7fde18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 0x7fde18071940 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fde280ad000 tx=0x7fde28008040 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 msgr2=0x7fde340b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fde2000b410 tx=0x7fde2000b8d0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 shutdown_connections 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde340d1d80 0x7fde34156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde340cd4a0 0x7fde340b0940 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fde1806f430 0x7fde18071940 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 --2- 172.21.15.83:0/323516483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde340b2f70 0x7fde340b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 >> 172.21.15.83:0/323516483 conn(0x7fde340b4170 msgr2=0x7fde340b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 shutdown_connections 2024-05-17T22:08:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.485+0000 7fde3b046640 1 -- 172.21.15.83:0/323516483 wait complete. 2024-05-17T22:08:22.496 INFO:tasks.ceph:Scrubbing osd.7 2024-05-17T22:08:22.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-17T22:08:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 Processor -- start 2024-05-17T22:08:22.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 -- start start 2024-05-17T22:08:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6480cde20 0x7fe6481603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe648160930 0x7fe64815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe6480d1210 con 0x7fe6480cde20 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7fe648160d40 con 0x7fe648160930 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe65073d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480d0970 con 0x7fe6480d03b0 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6480cde20 0x7fe6481603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe6480cde20 0x7fe6481603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53822/0 (socket says 172.21.15.83:53822) 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 -- 172.21.15.83:0/42830442 learned_addr learned my addr 172.21.15.83:0/42830442 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64e4d9640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64ecda640 1 --2- 172.21.15.83:0/42830442 >> v2:172.21.15.155:3300/0 conn(0x7fe648160930 0x7fe64815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 -- 172.21.15.83:0/42830442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 msgr2=0x7fe648145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 -- 172.21.15.83:0/42830442 >> v2:172.21.15.155:3300/0 conn(0x7fe648160930 msgr2=0x7fe64815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/42830442 >> v2:172.21.15.155:3300/0 conn(0x7fe648160930 0x7fe64815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.661+0000 7fe64dcd8640 1 -- 172.21.15.83:0/42830442 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6480d0c50 con 0x7fe6480cde20 2024-05-17T22:08:22.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481603f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe648072de0 tx=0x7fe63801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c687300cc437 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe64d4d7640 1 -- 172.21.15.83:0/42830442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b0070 con 0x7fe6480cde20 2024-05-17T22:08:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe64d4d7640 1 -- 172.21.15.83:0/42830442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6380ab600 con 0x7fe6480cde20 2024-05-17T22:08:22.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe64d4d7640 1 -- 172.21.15.83:0/42830442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380ab9d0 con 0x7fe6480cde20 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 -- 172.21.15.83:0/42830442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 msgr2=0x7fe6481603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481603f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fe648072de0 tx=0x7fe63801c520 comp rx=0 tx=0).stop 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 -- 172.21.15.83:0/42830442 shutdown_connections 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 --2- 172.21.15.83:0/42830442 >> v2:172.21.15.155:3300/0 conn(0x7fe648160930 0x7fe64815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481603f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 --2- 172.21.15.83:0/42830442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 -- 172.21.15.83:0/42830442 >> 172.21.15.83:0/42830442 conn(0x7fe64806cd90 msgr2=0x7fe6480b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 -- 172.21.15.83:0/42830442 shutdown_connections 2024-05-17T22:08:22.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.665+0000 7fe65073d640 1 -- 172.21.15.83:0/42830442 wait complete. 2024-05-17T22:08:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 Processor -- start 2024-05-17T22:08:22.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- start start 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe648160930 0x7fe648141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe64806f620 con 0x7fe6480cde20 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe64806f4a0 con 0x7fe648160930 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7fe64806f7a0 con 0x7fe6480d03b0 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53824/0 (socket says 172.21.15.83:53824) 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 -- 172.21.15.83:0/2568278069 learned_addr learned my addr 172.21.15.83:0/2568278069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64ecda640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe648160930 0x7fe648141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 -- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 msgr2=0x7fe648140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6480d03b0 0x7fe648140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 -- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe648160930 msgr2=0x7fe648141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe648160930 0x7fe648141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6480b47f0 con 0x7fe6480cde20 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64ecda640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe648160930 0x7fe648141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe64e4d9640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481417b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d5f0 tx=0x7fe64400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440125f0 con 0x7fe6480cde20 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe644019070 con 0x7fe6480cde20 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6480b4ad0 con 0x7fe6480cde20 2024-05-17T22:08:22.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440129b0 con 0x7fe6480cde20 2024-05-17T22:08:22.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.669+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64815f390 con 0x7fe6480cde20 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe648145fd0 con 0x7fe6480cde20 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe644012b50 con 0x7fe6480cde20 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62c06f430 0x7fe62c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62c06f430 0x7fe62c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7fe64408fe70 con 0x7fe6480cde20 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fe62c075480 0x7fe62c0778d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe62c077fa0 con 0x7fe62c075480 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7fe644090300 con 0x7fe6480cde20 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe64dcd8640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62c06f430 0x7fe62c071940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe6380ad000 tx=0x7fe638008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe64ecda640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fe62c075480 0x7fe62c0778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.673+0000 7fe64ecda640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fe62c075480 0x7fe62c0778d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.677+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== osd.7 v2:172.21.15.155:6816/3020759009 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe62c077fa0 con 0x7fe62c075480 2024-05-17T22:08:22.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 --> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe6480b3f40 con 0x7fe62c075480 2024-05-17T22:08:22.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe63effd640 1 -- 172.21.15.83:0/2568278069 <== osd.7 v2:172.21.15.155:6816/3020759009 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe6480b3f40 con 0x7fe62c075480 2024-05-17T22:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-05-17T22:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-17T22:08:22.681 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-05-17T22:08:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fe62c075480 msgr2=0x7fe62c0778d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.155:6816/3020759009,v1:172.21.15.155:6817/3020759009] conn(0x7fe62c075480 0x7fe62c0778d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62c06f430 msgr2=0x7fe62c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7fe62c06f430 0x7fe62c071940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe6380ad000 tx=0x7fe638008040 comp rx=0 tx=0).stop 2024-05-17T22:08:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 msgr2=0x7fe6481417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 --2- 172.21.15.83:0/2568278069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6480cde20 0x7fe6481417b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d5f0 tx=0x7fe64400dab0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe64ecda640 1 -- 172.21.15.83:0/2568278069 reap_dead start 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 shutdown_connections 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 >> 172.21.15.83:0/2568278069 conn(0x7fe64806cd90 msgr2=0x7fe648070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 shutdown_connections 2024-05-17T22:08:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.689+0000 7fe65073d640 1 -- 172.21.15.83:0/2568278069 wait complete. 2024-05-17T22:08:22.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-17T22:08:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 Processor -- start 2024-05-17T22:08:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 -- start start 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09b00b5e10 0x7f09b00b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09b00b3a30 con 0x7f09b006d170 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f09b00b3d10 con 0x7f09b00b5e10 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b67c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09b00b5610 con 0x7f09b00ce920 2024-05-17T22:08:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09b00b5e10 0x7f09b00b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09b00b5e10 0x7f09b00b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53826/0 (socket says 172.21.15.83:53826) 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 -- 172.21.15.83:0/1013789258 learned_addr learned my addr 172.21.15.83:0/1013789258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09af7fe640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 -- 172.21.15.83:0/1013789258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 msgr2=0x7f09b00b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 -- 172.21.15.83:0/1013789258 >> v1:172.21.15.155:6789/0 conn(0x7f09b006d170 legacy=0x7f09b00ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.853+0000 7f09b4d5f640 1 -- 172.21.15.83:0/1013789258 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b00b58f0 con 0x7f09b00b5e10 2024-05-17T22:08:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b4d5f640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b00b6220 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009870 tx=0x7f09a001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c925e06486c9a95e server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09aeffd640 1 -- 172.21.15.83:0/1013789258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00b0070 con 0x7f09b00b5e10 2024-05-17T22:08:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09aeffd640 1 -- 172.21.15.83:0/1013789258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a00ab600 con 0x7f09b00b5e10 2024-05-17T22:08:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09aeffd640 1 -- 172.21.15.83:0/1013789258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00ab9d0 con 0x7f09b00b5e10 2024-05-17T22:08:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 -- 172.21.15.83:0/1013789258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 msgr2=0x7f09b00b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b00b6220 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009870 tx=0x7f09a001c560 comp rx=0 tx=0).stop 2024-05-17T22:08:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 -- 172.21.15.83:0/1013789258 shutdown_connections 2024-05-17T22:08:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b00b6220 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1013789258 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 -- 172.21.15.83:0/1013789258 >> 172.21.15.83:0/1013789258 conn(0x7f09b00b4170 msgr2=0x7f09b00b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 -- 172.21.15.83:0/1013789258 shutdown_connections 2024-05-17T22:08:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.857+0000 7f09b67c2640 1 -- 172.21.15.83:0/1013789258 wait complete. 2024-05-17T22:08:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 Processor -- start 2024-05-17T22:08:22.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- start start 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09b006d170 0x7f09b00c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09b016b0c0 con 0x7f09b00b5e10 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09b015df80 con 0x7f09b006d170 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f09b016b240 con 0x7f09b00ce920 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53828/0 (socket says 172.21.15.83:53828) 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 -- 172.21.15.83:0/1325631183 learned_addr learned my addr 172.21.15.83:0/1325631183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b4d5f640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09affff640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09b006d170 0x7f09b00c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 -- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 msgr2=0x7f09b00c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 -- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09b006d170 msgr2=0x7f09b00c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09b006d170 0x7f09b00c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 -- 172.21.15.83:0/1325631183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b00cc4e0 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b4d5f640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09af7fe640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0998007e30 tx=0x7f099800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998014620 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0998019070 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b00cc7c0 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998014a00 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b00b1d10 con 0x7f09b00b5e10 2024-05-17T22:08:22.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.861+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b00ccce0 con 0x7f09b00b5e10 2024-05-17T22:08:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.865+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0998014ba0 con 0x7f09b00b5e10 2024-05-17T22:08:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.865+0000 7f09acff9640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 0x7f0990071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.865+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f099808f840 con 0x7f09b00b5e10 2024-05-17T22:08:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.865+0000 7f09affff640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 0x7f0990071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:22.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.865+0000 7f09affff640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 0x7f0990071940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f09a800a750 tx=0x7f09a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:22.869+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f09b00ccce0 con 0x7f09b00b5e10 2024-05-17T22:08:23.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.029+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f09b00b6220 con 0x7f099006f430 2024-05-17T22:08:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.029+0000 7f09acff9640 1 -- 172.21.15.83:0/1325631183 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09b00b6220 con 0x7f099006f430 2024-05-17T22:08:23.021 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 7 to deep-scrub 2024-05-17T22:08:23.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 msgr2=0x7f0990071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 0x7f0990071940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f09a800a750 tx=0x7f09a8008040 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 msgr2=0x7f09b0149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0998007e30 tx=0x7f099800e4b0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 shutdown_connections 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09b00ce920 0x7f09b00c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09b00b5e10 0x7f09b0149b30 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f099006f430 0x7f0990071940 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 --2- 172.21.15.83:0/1325631183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09b006d170 0x7f09b00c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 >> 172.21.15.83:0/1325631183 conn(0x7f09b00b4170 msgr2=0x7f09b00d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 shutdown_connections 2024-05-17T22:08:23.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.033+0000 7f09b67c2640 1 -- 172.21.15.83:0/1325631183 wait complete. 2024-05-17T22:08:23.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:08:23.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 Processor -- start 2024-05-17T22:08:23.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 -- start start 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8614073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f861406f040 0x7f861406f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8614073780 con 0x7f8614161d50 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f8614164960 con 0x7f861406f040 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8619b28640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f8614164c40 con 0x7f8614072e30 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86137fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8614161d50 0x7f86140b5960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:46262/0 (socket says 172.21.15.83:46262) 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86137fe640 1 -- 172.21.15.83:0/3697634483 learned_addr learned my addr 172.21.15.83:0/3697634483 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:23.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8613fff640 1 --2- 172.21.15.83:0/3697634483 >> v2:172.21.15.155:3300/0 conn(0x7f861406f040 0x7f861406f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8614073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515237328 0 0) 0x7f86040014a0 con 0x7f8614161d50 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8600003530 con 0x7f8614161d50 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 18432496 0 0) 0x7f8600003530 con 0x7f8614161d50 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 msgr2=0x7f8614073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8614073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 >> v2:172.21.15.155:3300/0 conn(0x7f861406f040 msgr2=0x7f861406f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 --2- 172.21.15.83:0/3697634483 >> v2:172.21.15.155:3300/0 conn(0x7f861406f040 0x7f861406f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8614164f20 con 0x7f8614161d50 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f8604001c50 con 0x7f8614161d50 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 >> v1:172.21.15.83:6789/0 conn(0x7f8614161d50 legacy=0x7f86140b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8600003a10 0x7f8600003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8600004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8600005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8614164960 con 0x7f8600004d50 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8614164c40 con 0x7f8600003a10 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f860000ba80 con 0x7f8614072e30 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f8613fff640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8600005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.201+0000 7f86137fe640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8600003a10 0x7f8600003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8600004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 -- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 msgr2=0x7f8600004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:23.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8614072e30 0x7f8600004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 -- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8600003a10 msgr2=0x7f8600003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:23.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8600003a10 0x7f8600003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 -- 172.21.15.83:0/3697634483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600003530 con 0x7f8600004d50 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8600005160 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f860c009ae0 tx=0x7f860c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cefd53af79fcf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0b2070 con 0x7f8600004d50 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c0a56a0 con 0x7f8600004d50 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f86127fc640 1 -- 172.21.15.83:0/3697634483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c0096b0 con 0x7f8600004d50 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 -- 172.21.15.83:0/3697634483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 msgr2=0x7f8600005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 --2- 172.21.15.83:0/3697634483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8600005160 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f860c009ae0 tx=0x7f860c01cab0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8613fff640 1 -- 172.21.15.83:0/3697634483 reap_dead start 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 -- 172.21.15.83:0/3697634483 shutdown_connections 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 -- 172.21.15.83:0/3697634483 >> 172.21.15.83:0/3697634483 conn(0x7f861406cd90 msgr2=0x7f86140b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 -- 172.21.15.83:0/3697634483 shutdown_connections 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.205+0000 7f8619b28640 1 -- 172.21.15.83:0/3697634483 wait complete. 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 Processor -- start 2024-05-17T22:08:23.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 -- start start 2024-05-17T22:08:23.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 0x7f86140aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 0x7f86140ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86140b45d0 con 0x7f8600004d50 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8614164650 con 0x7f86141fa160 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8619b28640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f86140b4750 con 0x7f86140aa310 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53838/0 (socket says 172.21.15.83:53838) 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 -- 172.21.15.83:0/3907166646 learned_addr learned my addr 172.21.15.83:0/3907166646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 0x7f86140aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8613fff640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 0x7f86140ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 -- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 msgr2=0x7f86140aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 0x7f86140aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 -- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 msgr2=0x7f86140ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 0x7f86140ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 -- 172.21.15.83:0/3907166646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86140b17d0 con 0x7f8600004d50 2024-05-17T22:08:23.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 0x7f86140aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f8613fff640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 0x7f86140ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f86137fe640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8604007a50 tx=0x7f860400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400edf0 con 0x7f8600004d50 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8604018070 con 0x7f8600004d50 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.209+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604013660 con 0x7f8600004d50 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86141fd490 con 0x7f8600004d50 2024-05-17T22:08:23.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86141fb370 con 0x7f8600004d50 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8614073240 con 0x7f8600004d50 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8604013800 con 0x7f8600004d50 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f860bfff640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 0x7f85f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f860408f710 con 0x7f8600004d50 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 0x7f85f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:23.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.213+0000 7f8612ffd640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 0x7f85f4071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f861406fd10 tx=0x7f85fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:23.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.217+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f8614073780 con 0x7f8600004d50 2024-05-17T22:08:23.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.373+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86140aced0 con 0x7f85f406f400 2024-05-17T22:08:23.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f860bfff640 1 -- 172.21.15.83:0/3907166646 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13479 (secure 0 0 0) 0x7f86140aced0 con 0x7f85f406f400 2024-05-17T22:08:23.367 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:23.367 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:08:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 msgr2=0x7f85f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 0x7f85f4071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f861406fd10 tx=0x7f85fc008040 comp rx=0 tx=0).stop 2024-05-17T22:08:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 msgr2=0x7f8614154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8604007a50 tx=0x7f860400e530 comp rx=0 tx=0).stop 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 shutdown_connections 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f85f406f400 0x7f85f4071910 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86141fa160 0x7f86140ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86140aa310 0x7f86140aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 --2- 172.21.15.83:0/3907166646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600004d50 0x7f8614154810 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 >> 172.21.15.83:0/3907166646 conn(0x7f861406cd90 msgr2=0x7f86140b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 shutdown_connections 2024-05-17T22:08:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:23.377+0000 7f8619b28640 1 -- 172.21.15.83:0/3907166646 wait complete. 2024-05-17T22:08:23.388 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-05-17T22:08:23.207367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261828,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712892,"statfs":{"total":773094113280,"available":772826001408,"internally_reserved":0,"allocated":12689408,"data_stored":6171088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.535016"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":285,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-17T22:08:15.857586+0000","last_change":"2024-05-17T21:57:20.216554+0000","last_active":"2024-05-17T22:08:15.857586+0000","last_peered":"2024-05-17T22:08:15.857586+0000","last_clean":"2024-05-17T22:08:15.857586+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:08:15.857586+0000","last_undegraded":"2024-05-17T22:08:15.857586+0000","last_fullsized":"2024-05-17T22:08:15.857586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-17T21:57:19.172065+0000","last_clean_scrub_stamp":"2024-05-17T21:57:19.172065+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-18T22:20:20.466464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2031616,"data_stored":1214162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574984,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32580,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2031616,"data_stored":1214162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:08:23.392 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-17T21:57:19.172065+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=17, tm_hour=21, tm_min=57, tm_sec=19, tm_wday=4, tm_yday=138, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=17, tm_hour=22, tm_min=8, tm_sec=17, tm_wday=4, tm_yday=138, tm_isdst=0) 2024-05-17T22:08:23.392 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-17T22:08:31.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:31.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:43.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-17T22:08:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 Processor -- start 2024-05-17T22:08:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 -- start start 2024-05-17T22:08:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f362806ffd0 0x7f36280703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 0x7f36281609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36281465d0 con 0x7f36281461c0 2024-05-17T22:08:43.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3628160ef0 con 0x7f362806ffd0 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.565+0000 7f362e268640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f362806ef30 con 0x7f36281605a0 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f362806ffd0 0x7f36280703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f362806ffd0 0x7f36280703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40482/0 (socket says 172.21.15.83:40482) 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 -- 172.21.15.83:0/828514809 learned_addr learned my addr 172.21.15.83:0/828514809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f362c805640 1 --2- 172.21.15.83:0/828514809 >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 0x7f36281609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 -- 172.21.15.83:0/828514809 >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 msgr2=0x7f36281609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 --2- 172.21.15.83:0/828514809 >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 0x7f36281609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 -- 172.21.15.83:0/828514809 >> v1:172.21.15.155:6789/0 conn(0x7f36281461c0 legacy=0x7f362815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 -- 172.21.15.83:0/828514809 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3628070920 con 0x7f362806ffd0 2024-05-17T22:08:43.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f362c805640 1 --2- 172.21.15.83:0/828514809 >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 0x7f36281609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f36277fe640 1 --2- 172.21.15.83:0/828514809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f362806ffd0 0x7f36280703e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3628072c00 tx=0x7f361001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15a67ed07541b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f3626ffd640 1 -- 172.21.15.83:0/828514809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100aa610 con 0x7f362806ffd0 2024-05-17T22:08:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f3626ffd640 1 -- 172.21.15.83:0/828514809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36100af070 con 0x7f362806ffd0 2024-05-17T22:08:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f3626ffd640 1 -- 172.21.15.83:0/828514809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100aa9d0 con 0x7f362806ffd0 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f362e268640 1 -- 172.21.15.83:0/828514809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f362806ffd0 msgr2=0x7f36280703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.569+0000 7f362e268640 1 --2- 172.21.15.83:0/828514809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f362806ffd0 0x7f36280703e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3628072c00 tx=0x7f361001c4f0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- 172.21.15.83:0/828514809 shutdown_connections 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 --2- 172.21.15.83:0/828514809 >> v2:172.21.15.155:3300/0 conn(0x7f36281605a0 0x7f36281609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 --2- 172.21.15.83:0/828514809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f362806ffd0 0x7f36280703e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- 172.21.15.83:0/828514809 >> 172.21.15.83:0/828514809 conn(0x7f362806cd90 msgr2=0x7f36280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- 172.21.15.83:0/828514809 shutdown_connections 2024-05-17T22:08:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- 172.21.15.83:0/828514809 wait complete. 2024-05-17T22:08:43.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 Processor -- start 2024-05-17T22:08:43.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- start start 2024-05-17T22:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362806ffd0 0x7f362815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36281461c0 0x7f362813f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f362815e880 con 0x7f36281605a0 2024-05-17T22:08:43.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f362806fab0 con 0x7f362806ffd0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362e268640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f36280d10a0 con 0x7f36281461c0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40484/0 (socket says 172.21.15.83:40484) 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 -- 172.21.15.83:0/3475334349 learned_addr learned my addr 172.21.15.83:0/3475334349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 -- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36281461c0 msgr2=0x7f362813f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36281461c0 0x7f362813f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 -- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362806ffd0 msgr2=0x7f362815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362806ffd0 0x7f362815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.573+0000 7f362c805640 1 -- 172.21.15.83:0/3475334349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36281421c0 con 0x7f36281605a0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f362c805640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f362000cd50 tx=0x7f362000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362000f680 con 0x7f36281605a0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362000f840 con 0x7f36281605a0 2024-05-17T22:08:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36281424a0 con 0x7f36281605a0 2024-05-17T22:08:43.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362000fc30 con 0x7f36281605a0 2024-05-17T22:08:43.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36281f7ce0 con 0x7f36281605a0 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36280703e0 con 0x7f36281605a0 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f362000e3e0 con 0x7f36281605a0 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 0x7f3608071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3627fff640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 0x7f3608071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f362008fe80 con 0x7f36281605a0 2024-05-17T22:08:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.577+0000 7f3627fff640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 0x7f3608071910 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f36280b11f0 tx=0x7f3618006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.581+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f3620014030 con 0x7f36281605a0 2024-05-17T22:08:43.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.741+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 --> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36280b41a0 con 0x7f360806f400 2024-05-17T22:08:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.741+0000 7f3624ff9640 1 -- 172.21.15.83:0/3475334349 <== mgr.4103 v2:172.21.15.83:6832/380985748 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7f36280b41a0 con 0x7f360806f400 2024-05-17T22:08:43.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:43.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-05-17T22:08:43.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 msgr2=0x7f3608071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 0x7f3608071910 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f36280b11f0 tx=0x7f3618006bd0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 msgr2=0x7f362813fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f362000cd50 tx=0x7f362000a650 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 shutdown_connections 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281605a0 0x7f362813fbb0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f36281461c0 0x7f362813f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f360806f400 0x7f3608071910 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 --2- 172.21.15.83:0/3475334349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362806ffd0 0x7f362815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 >> 172.21.15.83:0/3475334349 conn(0x7f362806cd90 msgr2=0x7f362815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 shutdown_connections 2024-05-17T22:08:43.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.745+0000 7f362e268640 1 -- 172.21.15.83:0/3475334349 wait complete. 2024-05-17T22:08:43.757 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-05-17T22:08:43.213124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261820,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712900,"statfs":{"total":773094113280,"available":772826009600,"internally_reserved":0,"allocated":12689408,"data_stored":6171088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003581"},"pg_stats":[{"pgid":"1.0","version":"115'74","reported_seq":296,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-05-17T22:08:23.876219+0000","last_change":"2024-05-17T22:08:23.876385+0000","last_active":"2024-05-17T22:08:23.876219+0000","last_peered":"2024-05-17T22:08:23.876219+0000","last_clean":"2024-05-17T22:08:23.876219+0000","last_became_active":"2024-05-17T21:57:20.211157+0000","last_became_peered":"2024-05-17T21:57:20.211157+0000","last_unstale":"2024-05-17T22:08:23.876219+0000","last_undegraded":"2024-05-17T22:08:23.876219+0000","last_fullsized":"2024-05-17T22:08:23.876219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'74","last_scrub_stamp":"2024-05-17T22:08:23.875939+0000","last_deep_scrub":"115'74","last_deep_scrub_stamp":"2024-05-17T22:08:23.875939+0000","last_clean_scrub_stamp":"2024-05-17T22:08:23.875939+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-19T08:43:00.431287+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2031616,"data_stored":1214162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32580,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":623794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2031616,"data_stored":1214162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-17T22:08:43.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-17T22:08:43.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 Processor -- start 2024-05-17T22:08:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 -- start start 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0ca80d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca809c710 con 0x7f0ca809ba00 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca809c9f0 con 0x7f0ca8071400 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0cac9bb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80d1a40 con 0x7f0ca80b88b0 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca5d9b640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ca809ba00 0x7f0ca809be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:41256/0 (socket says 172.21.15.83:41256) 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca5d9b640 1 -- 172.21.15.83:0/3574776042 learned_addr learned my addr 172.21.15.83:0/3574776042 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca659c640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0ca80d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 417516553 0 0) 0x7f0ca809c710 con 0x7f0ca809ba00 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c90003530 con 0x7f0ca809ba00 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1184177368 0 0) 0x7f0c90003530 con 0x7f0ca809ba00 2024-05-17T22:08:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 msgr2=0x7f0ca80d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0ca80d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 >> v1:172.21.15.155:6789/0 conn(0x7f0ca8071400 legacy=0x7f0ca8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca809ccd0 con 0x7f0ca809ba00 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.921+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3597855946 0 0) 0x7f0c9c001700 con 0x7f0ca809ba00 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 >> v1:172.21.15.83:6789/0 conn(0x7f0ca809ba00 legacy=0x7f0ca809be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c90003a40 0x7f0c90003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0c90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c90004e70 0x7f0c900052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d1a40 con 0x7f0c90003a40 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9000bcd0 con 0x7f0c90004e70 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9000bed0 con 0x7f0ca80b88b0 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c90003a40 0x7f0c90003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca5d9b640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0c90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca6d9d640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c90004e70 0x7f0c900052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 -- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 msgr2=0x7f0c90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ca80b88b0 0x7f0c90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 -- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c90004e70 msgr2=0x7f0c900052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c90004e70 0x7f0c900052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 -- 172.21.15.83:0/3574776042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c90003530 con 0x7f0c90003a40 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca6d9d640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c90004e70 0x7f0c900052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca659c640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c90003a40 0x7f0c90003eb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009fc0 tx=0x7f0ca001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101895963ba75185 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00b2070 con 0x7f0c90003a40 2024-05-17T22:08:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca00a5ba0 con 0x7f0c90003a40 2024-05-17T22:08:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.925+0000 7f0ca559a640 1 -- 172.21.15.83:0/3574776042 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0009c10 con 0x7f0c90003a40 2024-05-17T22:08:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- 172.21.15.83:0/3574776042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c90003a40 msgr2=0x7f0c90003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/3574776042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c90003a40 0x7f0c90003eb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009fc0 tx=0x7f0ca001cca0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0ca6d9d640 1 -- 172.21.15.83:0/3574776042 reap_dead start 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- 172.21.15.83:0/3574776042 shutdown_connections 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- 172.21.15.83:0/3574776042 >> 172.21.15.83:0/3574776042 conn(0x7f0ca80d3620 msgr2=0x7f0ca80d3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- 172.21.15.83:0/3574776042 shutdown_connections 2024-05-17T22:08:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- 172.21.15.83:0/3574776042 wait complete. 2024-05-17T22:08:43.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 Processor -- start 2024-05-17T22:08:43.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- start start 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c90003a40 0x7f0ca8154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca815fe00 0x7f0ca80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca81614b0 con 0x7f0ca815e020 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca809d020 con 0x7f0ca815fe00 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0cac9bb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80cb7a0 con 0x7f0c90003a40 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0ca5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0ca5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40504/0 (socket says 172.21.15.83:40504) 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.929+0000 7f0ca5d9b640 1 -- 172.21.15.83:0/1457791559 learned_addr learned my addr 172.21.15.83:0/1457791559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 -- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c90003a40 msgr2=0x7f0ca8154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c90003a40 0x7f0ca8154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 -- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca815fe00 msgr2=0x7f0ca80ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca815fe00 0x7f0ca80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 -- 172.21.15.83:0/1457791559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca8160210 con 0x7f0ca815e020 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca5d9b640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c001ec0 tx=0x7f0c9c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00c690 con 0x7f0ca815e020 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c018070 con 0x7f0ca815e020 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca815c5b0 con 0x7f0ca815e020 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c00ca70 con 0x7f0ca815e020 2024-05-17T22:08:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca815cb70 con 0x7f0ca815e020 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca8071810 con 0x7f0ca815e020 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c9c00cc10 con 0x7f0ca815e020 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 0x7f0c88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f0c9c08ed20 con 0x7f0ca815e020 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca659c640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 0x7f0c88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.933+0000 7f0ca659c640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 0x7f0c88071910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00af000 tx=0x7f0ca001b4e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:43.937+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f0ca8160210 con 0x7f0ca815e020 2024-05-17T22:08:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.101+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f0ca815c340 con 0x7f0ca815e020 2024-05-17T22:08:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.105+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f0c9c05bf80 con 0x7f0ca815e020 2024-05-17T22:08:44.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0c96ffd640 1 -- 172.21.15.83:0/1457791559 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0c9c092030 con 0x7f0ca815e020 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 msgr2=0x7f0c88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 0x7f0c88071910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00af000 tx=0x7f0ca001b4e0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 msgr2=0x7f0ca815f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c001ec0 tx=0x7f0c9c00ec10 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 shutdown_connections 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f0c8806f400 0x7f0c88071910 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ca815fe00 0x7f0ca80ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca815e020 0x7f0ca815f8c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 --2- 172.21.15.83:0/1457791559 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c90003a40 0x7f0ca8154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 >> 172.21.15.83:0/1457791559 conn(0x7f0ca80d3620 msgr2=0x7f0ca80b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:44.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 shutdown_connections 2024-05-17T22:08:44.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.109+0000 7f0cac9bb640 1 -- 172.21.15.83:0/1457791559 wait complete. 2024-05-17T22:08:44.125 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-17T22:08:44.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-17T22:08:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 Processor -- start 2024-05-17T22:08:44.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 -- start start 2024-05-17T22:08:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9b601461c0 0x7f9b6015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b6015e6b0 0x7f9b6006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b60070510 0x7f9b601605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9b601465d0 con 0x7f9b60070510 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 -- 0x7f9b60070920 con 0x7f9b601461c0 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b67235640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6006ef30 con 0x7f9b6015e6b0 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b60070510 0x7f9b601605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9b60070510 0x7f9b601605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40508/0 (socket says 172.21.15.83:40508) 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 -- 172.21.15.83:0/55787423 learned_addr learned my addr 172.21.15.83:0/55787423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b64fd1640 1 --2- 172.21.15.83:0/55787423 >> v2:172.21.15.155:3300/0 conn(0x7f9b601461c0 0x7f9b6015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b5ffff640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b6015e6b0 0x7f9b6006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 -- 172.21.15.83:0/55787423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b6015e6b0 msgr2=0x7f9b6006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b6015e6b0 0x7f9b6006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 -- 172.21.15.83:0/55787423 >> v2:172.21.15.155:3300/0 conn(0x7f9b601461c0 msgr2=0x7f9b6015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-17T22:08:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 --2- 172.21.15.83:0/55787423 >> v2:172.21.15.155:3300/0 conn(0x7f9b601461c0 0x7f9b6015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.281+0000 7f9b657d2640 1 -- 172.21.15.83:0/55787423 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6006f210 con 0x7f9b60070510 2024-05-17T22:08:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b657d2640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b60070510 0x7f9b601605a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9b6006d5a0 tx=0x7f9b5001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2943dedba8a96b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b5f7fe640 1 -- 172.21.15.83:0/55787423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500b1070 con 0x7f9b60070510 2024-05-17T22:08:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b5f7fe640 1 -- 172.21.15.83:0/55787423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9b500ac620 con 0x7f9b60070510 2024-05-17T22:08:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b5f7fe640 1 -- 172.21.15.83:0/55787423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500aca10 con 0x7f9b60070510 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- 172.21.15.83:0/55787423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b60070510 msgr2=0x7f9b601605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b60070510 0x7f9b601605a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f9b6006d5a0 tx=0x7f9b5001c4f0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- 172.21.15.83:0/55787423 shutdown_connections 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b60070510 0x7f9b601605a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 --2- 172.21.15.83:0/55787423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b6015e6b0 0x7f9b6006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 --2- 172.21.15.83:0/55787423 >> v2:172.21.15.155:3300/0 conn(0x7f9b601461c0 0x7f9b6015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- 172.21.15.83:0/55787423 >> 172.21.15.83:0/55787423 conn(0x7f9b6006cd90 msgr2=0x7f9b600b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- 172.21.15.83:0/55787423 shutdown_connections 2024-05-17T22:08:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- 172.21.15.83:0/55787423 wait complete. 2024-05-17T22:08:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 Processor -- start 2024-05-17T22:08:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.285+0000 7f9b67235640 1 -- start start 2024-05-17T22:08:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 0x7f9b600b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b601461c0 0x7f9b601631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600d1220 con 0x7f9b6015e6b0 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b600d10a0 con 0x7f9b601461c0 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 -- 0x7f9b600d13a0 con 0x7f9b60070510 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40524/0 (socket says 172.21.15.83:40524) 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 -- 172.21.15.83:0/470588763 learned_addr learned my addr 172.21.15.83:0/470588763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b64fd1640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 0x7f9b600b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b5ffff640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b601461c0 0x7f9b601631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 -- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 msgr2=0x7f9b600b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 0x7f9b600b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 -- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b601461c0 msgr2=0x7f9b601631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b601461c0 0x7f9b601631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 -- 172.21.15.83:0/470588763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b600d2380 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b64fd1640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 0x7f9b600b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b657d2640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9b5001aba0 tx=0x7f9b50019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500b1040 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9b500ba070 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b600b51b0 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500b5640 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b600b5f00 con 0x7f9b6015e6b0 2024-05-17T22:08:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b600b4f20 con 0x7f9b6015e6b0 2024-05-17T22:08:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.289+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b500b57e0 con 0x7f9b6015e6b0 2024-05-17T22:08:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.293+0000 7f9b5d7fa640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 0x7f9b4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-17T22:08:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.293+0000 7f9b64fd1640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 0x7f9b4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-17T22:08:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.293+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5115+0+0 (secure 0 0 0) 0x7f9b50129050 con 0x7f9b6015e6b0 2024-05-17T22:08:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.293+0000 7f9b64fd1640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 0x7f9b4c071910 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b11f0 tx=0x7f9b5800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-17T22:08:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.297+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180029 (secure 0 0 0) 0x7f9b50016110 con 0x7f9b6015e6b0 2024-05-17T22:08:44.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.485+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b6015e170 con 0x7f9b6015e6b0 2024-05-17T22:08:44.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.485+0000 7f9b5d7fa640 1 -- 172.21.15.83:0/470588763 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3874 (secure 0 0 0) 0x7f9b500f6190 con 0x7f9b6015e6b0 2024-05-17T22:08:44.477 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-05-17T22:08:44.477 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":111,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5837,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/3075771007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3075771007},{"type":"v1","addr":"172.21.15.155:6833","nonce":3075771007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":6358,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3142408631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3142408631},{"type":"v1","addr":"172.21.15.155:6835","nonce":3142408631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":7977,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/981868671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":981868671},{"type":"v1","addr":"172.21.15.83:6836","nonce":981868671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1485073283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":1485073283},{"type":"v1","addr":"172.21.15.83:6837","nonce":1485073283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[]} 2024-05-17T22:08:44.477 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 111 2024-05-17T22:08:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 msgr2=0x7f9b4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 0x7f9b4c071910 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9b600b11f0 tx=0x7f9b5800b040 comp rx=0 tx=0).stop 2024-05-17T22:08:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 msgr2=0x7f9b600b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-17T22:08:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9b5001aba0 tx=0x7f9b50019290 comp rx=0 tx=0).stop 2024-05-17T22:08:44.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 shutdown_connections 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b6015e6b0 0x7f9b600b45d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.83:6832/380985748,v1:172.21.15.83:6833/380985748] conn(0x7f9b4c06f400 0x7f9b4c071910 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b601461c0 0x7f9b601631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 --2- 172.21.15.83:0/470588763 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9b60070510 0x7f9b600b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 >> 172.21.15.83:0/470588763 conn(0x7f9b6006cd90 msgr2=0x7f9b600d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 shutdown_connections 2024-05-17T22:08:44.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-05-17T22:08:44.489+0000 7f9b67235640 1 -- 172.21.15.83:0/470588763 wait complete. 2024-05-17T22:08:44.498 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-17T22:08:44.498 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-17T22:08:44.498 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-17T22:08:44.499 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.513 INFO:tasks.ceph.mds.a:Stopped 2024-05-17T22:08:44.513 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-17T22:08:44.514 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.574 INFO:tasks.ceph.mds.c:Stopped 2024-05-17T22:08:44.574 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-17T22:08:44.574 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.590 INFO:tasks.ceph.mds.b:Stopped 2024-05-17T22:08:44.590 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-17T22:08:44.590 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.658 INFO:tasks.ceph.mds.d:Stopped 2024-05-17T22:08:44.658 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-17T22:08:44.658 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-17T22:08:44.658 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.731 INFO:tasks.ceph.osd.0:Stopped 2024-05-17T22:08:44.732 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-17T22:08:44.732 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.817 INFO:tasks.ceph.osd.1:Stopped 2024-05-17T22:08:44.817 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-17T22:08:44.817 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.910 INFO:tasks.ceph.osd.2:Stopped 2024-05-17T22:08:44.911 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-17T22:08:44.911 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:44.993 INFO:tasks.ceph.osd.3:Stopped 2024-05-17T22:08:44.994 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-17T22:08:44.994 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:45.080 INFO:tasks.ceph.osd.4:Stopped 2024-05-17T22:08:45.081 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-17T22:08:45.081 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:45.172 INFO:tasks.ceph.osd.5:Stopped 2024-05-17T22:08:45.172 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-17T22:08:45.172 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:50.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-17T22:08:51.274 INFO:tasks.ceph.osd.6:Stopped 2024-05-17T22:08:51.274 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-17T22:08:51.274 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.350 INFO:tasks.ceph.osd.7:Stopped 2024-05-17T22:08:51.351 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-17T22:08:51.351 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-17T22:08:51.351 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.382 INFO:tasks.ceph.mgr.y:Stopped 2024-05-17T22:08:51.383 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-17T22:08:51.383 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.412 INFO:tasks.ceph.mgr.x:Stopped 2024-05-17T22:08:51.412 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-17T22:08:51.412 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-17T22:08:51.413 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.454 INFO:tasks.ceph.mon.a:Stopped 2024-05-17T22:08:51.455 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-17T22:08:51.455 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.479 INFO:tasks.ceph.mon.b:Stopped 2024-05-17T22:08:51.480 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-17T22:08:51.480 INFO:teuthology.orchestra.run:waiting for 300 2024-05-17T22:08:51.550 INFO:tasks.ceph.mon.c:Stopped 2024-05-17T22:08:51.551 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-17T22:08:51.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | egrep -v 'object missing on disk' | egrep -v 'Invalid tag char' | head -n 1 2024-05-17T22:08:51.572 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:51.572 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-17T22:08:51.967 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:51.967 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-17T22:08:52.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:52.188 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-17T22:08:52.379 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:52.379 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-17T22:08:52.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T22:08:52.555 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-17T22:08:52.732 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T22:08:52.732 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-17T22:08:52.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T22:08:52.916 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-17T22:08:53.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T22:08:53.068 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-17T22:08:53.235 INFO:tasks.ceph:Archiving mon data... 2024-05-17T22:08:53.236 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/data/mon.a.tgz 2024-05-17T22:08:53.237 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-05-17T22:08:53.243 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.Xoy7b2yeoe 2024-05-17T22:08:53.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Xoy7b2yeoe 2024-05-17T22:08:53.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.Xoy7b2yeoe 2024-05-17T22:08:53.448 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.Xoy7b2yeoe is 390KB 2024-05-17T22:08:53.509 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.Xoy7b2yeoe 2024-05-17T22:08:53.514 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/data/mon.b.tgz 2024-05-17T22:08:53.515 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-05-17T22:08:53.520 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.TIiVQQx5Eo 2024-05-17T22:08:53.521 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TIiVQQx5Eo 2024-05-17T22:08:53.655 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.TIiVQQx5Eo 2024-05-17T22:08:53.718 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.TIiVQQx5Eo is 395KB 2024-05-17T22:08:53.779 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.TIiVQQx5Eo 2024-05-17T22:08:53.784 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/data/mon.c.tgz 2024-05-17T22:08:53.784 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-05-17T22:08:53.830 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.4wSD1aBOqr 2024-05-17T22:08:53.830 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4wSD1aBOqr 2024-05-17T22:08:53.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.4wSD1aBOqr 2024-05-17T22:08:54.025 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.4wSD1aBOqr is 395KB 2024-05-17T22:08:54.042 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.4wSD1aBOqr 2024-05-17T22:08:54.049 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-17T22:08:54.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-17T22:08:54.052 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-17T22:08:54.111 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-17T22:08:54.154 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-17T22:08:54.154 INFO:tasks.ceph:Archiving crash dumps... 2024-05-17T22:08:54.156 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi083/crash 2024-05-17T22:08:54.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-17T22:08:54.171 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi155/crash 2024-05-17T22:08:54.172 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-17T22:08:54.209 INFO:tasks.ceph:Shutting down logrotate 2024-05-17T22:08:54.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:54.216 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:54.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:54.315 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-17T22:08:54.328 INFO:tasks.ceph:Compressing logs... 2024-05-17T22:08:54.328 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:08:54.361 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-17T22:08:54.379 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37719.log 2024-05-17T22:08:54.382 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35717.log 2024-05-17T22:08:54.382 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49351.log 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37719.log.gz 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42587.log 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35717.log.gz 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36715.log 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39935.log 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49351.log.gz 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6%/var/log/ceph/ceph-client.admin.36715.log: -- replaced with /var/log/ceph/ceph-client.admin.42587.log.gz 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36715.log.gz 2024-05-17T22:08:54.383 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42875.log 2024-05-17T22:08:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42875.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47017.log 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42875.log.gz 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39935.log.gz 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42972.log 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33907.log 2024-05-17T22:08:54.393 INFO:teuthology.orchestra.run.smithi083.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47017.log.gz 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42972.log.gz 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40308.log 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33907.log.gz 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47702.log 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44666.log: /var/log/ceph/ceph-client.admin.40308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36634.log 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40308.log.gz 2024-05-17T22:08:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47702.log.gz 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.0.36634.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46527.log 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37056.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gz 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46527.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46527.log.gz 2024-05-17T22:08:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40126.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49818.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40126.log.gz 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37518.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49818.log.gz 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz /var/log/ceph/ceph-client.admin.49329.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36094.log 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49329.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-05-17T22:08:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43488.log 2024-05-17T22:08:54.397 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49329.log.gz 2024-05-17T22:08:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42710.log 2024-05-17T22:08:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.43488.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36094.log.gz 2024-05-17T22:08:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34974.log 2024-05-17T22:08:54.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41850.log 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27909.log 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27813.log 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27909.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27909.log.gz 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27813.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-17T22:08:54.413 INFO:teuthology.orchestra.run.smithi155.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27813.log.gz 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33464.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33583.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.33464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27741.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.3.33583.log: 88.7% -- replaced with /var/log/ceph/ceph-client.1.33464.log.gz 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27837.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27741.log: 88.5%/var/log/ceph/ceph-osd.5.log: -- replaced with /var/log/ceph/ceph-client.3.33583.log.gz 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27765.log 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27741.log.gz 2024-05-17T22:08:54.414 INFO:teuthology.orchestra.run.smithi155.stderr:gzip/var/log/ceph/ceph-client.admin.27837.log: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27765.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27837.log.gz 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27765.log.gz 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27789.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27789.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.audit.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27789.log.gz 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.d.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-17T22:08:54.415 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.b.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27861.log 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.4.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27885.log 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27861.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27861.log.gz 2024-05-17T22:08:54.416 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.27885.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27885.log.gz 2024-05-17T22:08:54.473 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.34974.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42710.log.gz 2024-05-17T22:08:54.473 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35743.log 2024-05-17T22:08:54.473 INFO:teuthology.orchestra.run.smithi083.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.36634.log.gz 2024-05-17T22:08:54.473 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50521.log 2024-05-17T22:08:54.473 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39603.log 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.34974.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41850.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35743.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45431.log 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39603.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50521.log.gzgzip 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34409.log 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.43488.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39603.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34409.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45431.log.gz 2024-05-17T22:08:54.474 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49272.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34409.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39032.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49598.log: /var/log/ceph/ceph-client.admin.49272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42257.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39032.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49272.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50781.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39032.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46608.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42257.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37771.log 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50781.log.gz 2024-05-17T22:08:54.475 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.46608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40256.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37771.log.gz 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40256.log.gz 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47365.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35795.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46949.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35795.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47365.log.gz 2024-05-17T22:08:54.476 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48283.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35795.log.gz 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34687.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34687.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38095.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34687.log.gz 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45108.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50365.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38095.log: /var/log/ceph/ceph-client.admin.45108.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log 2024-05-17T22:08:54.477 INFO:teuthology.orchestra.run.smithi083.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.38095.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33588.log 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50365.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40483.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45108.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36742.log 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33588.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37875.log 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% 94.4%/var/log/ceph/ceph-client.admin.36742.log: -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz -- replaced with /var/log/ceph/ceph-client.0.46608.log.gz 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-05-17T22:08:54.478 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36742.log.gz 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48413.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50859.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47415.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37875.log.gz 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48984.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48984.log: 91.2% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50859.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48204.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36229.log 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37108.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48984.log.gz 2024-05-17T22:08:54.479 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48204.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42400.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36229.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36147.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42400.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42400.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50241.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36147.log.gz 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41274.log 2024-05-17T22:08:54.480 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41274.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46068.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41274.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48361.log: /var/log/ceph/ceph-client.admin.39283.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46068.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-05-17T22:08:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36042.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5%/var/log/ceph/ceph-client.admin.39283.log: -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34212.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39283.log.gz 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49172.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34212.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36042.log.gz 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41127.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34212.log.gz 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48070.log 2024-05-17T22:08:54.482 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42431.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41127.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50417.log 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33488.log 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48070.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33488.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50417.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37693.log 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33488.log.gz 2024-05-17T22:08:54.483 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35866.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39177.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.49172.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36805.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35866.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37693.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34219.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35866.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36805.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.39177.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40094.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36805.log.gz 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-05-17T22:08:54.484 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46552.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40094.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34219.log.gz 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50729.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40094.log.gz 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41358.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46552.log.gz 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41358.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50729.log.gz /var/log/ceph/ceph-client.admin.47808.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.41358.log.gz 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41900.log 2024-05-17T22:08:54.485 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47808.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42535.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47756.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41900.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38821.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37082.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47756.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42535.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44176.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47756.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37082.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45378.log 2024-05-17T22:08:54.486 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37082.log.gz 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37134.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44176.log.gz 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45378.log: /var/log/ceph/ceph-client.admin.37544.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45378.log.gzgzip 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37134.log.gz 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36398.log 2024-05-17T22:08:54.487 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43198.log: /var/log/ceph/ceph-osd.2.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35840.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35840.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36398.log.gz 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50937.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34215.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46772.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35840.log.gz 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46504.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3%/var/log/ceph/ceph-client.admin.34215.log: -- replaced with /var/log/ceph/ceph-client.admin.50937.log.gz 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42367.log 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-05-17T22:08:54.488 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46504.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34215.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46504.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40691.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42367.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36952.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40776.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40535.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40776.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40509.log 2024-05-17T22:08:54.489 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40776.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40509.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40535.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48018.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40509.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:gzip/var/log/ceph/ceph-client.admin.45003.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40152.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35227.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34429.log 2024-05-17T22:08:54.490 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40152.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40152.log.gz 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47650.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34429.log.gz 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47730.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47650.log.gz 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50911.log 2024-05-17T22:08:54.491 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47730.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45268.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49010.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47241.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50911.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50807.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47241.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45268.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50677.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50807.log.gz 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-05-17T22:08:54.492 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47241.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34204.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37572.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50677.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: /var/log/ceph/ceph-client.admin.38438.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34204.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43611.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38951.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34204.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43611.log: /var/log/ceph/ceph-client.admin.38951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39361.log 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43611.log.gz 2024-05-17T22:08:54.493 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38951.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43276.log: /var/log/ceph/ceph-client.admin.39361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38568.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5%/var/log/ceph/ceph-client.admin.39309.log: -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50833.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39361.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38568.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38568.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39528.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6%/var/log/ceph/ceph-client.admin.34218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37335.log 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50833.log.gz 2024-05-17T22:08:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39528.log.gz 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37335.log.gz 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46038.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip 89.4%/var/log/ceph/ceph-client.admin.41305.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz --verbose 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.38145.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41976.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38145.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46038.log.gz 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-05-17T22:08:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41976.log: /var/log/ceph/ceph-client.admin.47189.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38145.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40813.log 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41976.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46899.log 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48231.log: /var/log/ceph/ceph-client.admin.40813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46899.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40813.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49988.log 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46899.log.gz 2024-05-17T22:08:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz/var/log/ceph/ceph-client.admin.49988.log: 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40568.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49988.log.gz 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41495.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41495.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34115.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41495.log.gz 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37797.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-05-17T22:08:54.497 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34115.log: /var/log/ceph/ceph-client.admin.37797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49544.log 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35195.log 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34115.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37797.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49544.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39882.log 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35195.log: /var/log/ceph/ceph-client.admin.49646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39882.log: 88.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35195.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.49544.log.gz 2024-05-17T22:08:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34609.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.39882.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34609.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34609.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49302.log: /var/log/ceph/ceph-client.admin.40379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45158.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0%/var/log/ceph/ceph-client.admin.46444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34217.log 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45158.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-05-17T22:08:54.499 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45158.log.gz 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33762.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34217.log.gz 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33762.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36173.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42561.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33762.log.gz 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36173.log.gz/var/log/ceph/ceph-client.admin.34713.log: 2024-05-17T22:08:54.500 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38380.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46737.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38380.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33654.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38380.log.gz 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42613.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33654.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42613.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33654.log.gz 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37602.log 2024-05-17T22:08:54.501 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42613.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35450.log: /var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37951.log 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40011.log 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37951.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37951.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34428.log 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40011.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34791.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-05-17T22:08:54.502 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50887.log 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42788.log 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34428.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49470.log 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50887.log.gz 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34428.log.gz 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49470.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.45326.log 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42788.log.gz 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49470.log.gz/var/log/ceph/ceph-client.admin.38795.log: 2024-05-17T22:08:54.503 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38146.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45326.log.gz 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35769.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.38146.log.gz 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35169.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35769.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35169.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35769.log.gz 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46120.log 2024-05-17T22:08:54.504 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35169.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38412.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50066.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47624.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50066.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46120.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41716.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47624.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50066.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48947.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47624.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43142.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34765.log 2024-05-17T22:08:54.505 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6%/var/log/ceph/ceph-client.admin.43142.log: -- replaced with /var/log/ceph/ceph-client.admin.41716.log.gzgzip 2024-05-17T22:08:54.506 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2024-05-17T22:08:54.506 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43142.log.gz 2024-05-17T22:08:54.506 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34765.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34765.log.gz 2024-05-17T22:08:54.506 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-05-17T22:08:54.507 INFO:teuthology.orchestra.run.smithi155.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38270.log 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7%/var/log/ceph/ceph-client.admin.37492.log: -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35613.log 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.38270.log.gz 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-05-17T22:08:54.553 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45710.log: /var/log/ceph/ceph-client.admin.35613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43967.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35613.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40973.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.43967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41335.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36015.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41335.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41335.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.43967.log.gz 2024-05-17T22:08:54.554 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36015.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43581.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36476.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41247.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35665.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35665.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36476.log.gz 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2024-05-17T22:08:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36854.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35665.log.gz 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36201.log: /var/log/ceph/ceph-client.admin.43024.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40353.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47546.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40353.log: gzip 89.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz --verbose 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-client.admin.45632.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40353.log.gz 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47546.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47546.log.gz 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49245.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47782.log 2024-05-17T22:08:54.556 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45632.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38542.log 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49245.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.48821.log 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47782.log: /var/log/ceph/ceph-client.admin.38542.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47782.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38542.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42155.log 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47598.log 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47860.log 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48821.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47598.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42155.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47598.log.gz 2024-05-17T22:08:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46226.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47860.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46226.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47860.log.gz 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39985.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.1.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46226.log.gz 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35691.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39985.log.gz 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40457.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40204.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-05-17T22:08:54.558 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35691.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50144.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40204.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44723.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38068.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44723.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44723.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35359.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38068.log: /var/log/ceph/ceph-client.admin.35359.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38068.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35359.log.gz 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-05-17T22:08:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41051.log: /var/log/ceph/ceph-client.admin.47163.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37901.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37637.log: /var/log/ceph/ceph-client.admin.37901.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45234.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.37901.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37205.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34037.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43081.log 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45234.log: /var/log/ceph/ceph-client.admin.34037.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45234.log.gz 2024-05-17T22:08:54.560 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34037.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39060.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43081.log: /var/log/ceph/ceph-client.admin.44431.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43081.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35639.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39060.log: /var/log/ceph/ceph-client.admin.44202.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39060.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43172.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35639.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35639.log.gz 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38118.log 2024-05-17T22:08:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49684.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43172.log: /var/log/ceph/ceph-client.admin.38118.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43172.log.gz 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.38118.log.gz 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49684.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49684.log.gz 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34220.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48098.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46302.log: /var/log/ceph/ceph-client.admin.34220.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34220.log.gz 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42509.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-05-17T22:08:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48098.log: /var/log/ceph/ceph-client.admin.42509.log: 88.5% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42509.log.gz 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48098.log.gz 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34846.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35989.log: /var/log/ceph/ceph-client.admin.34846.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34846.log.gz 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45777.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37231.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4%/var/log/ceph/ceph-client.admin.45777.log: -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45777.log.gz 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45986.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33555.log 2024-05-17T22:08:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37231.log: /var/log/ceph/ceph-client.admin.45986.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37231.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45986.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42052.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34422.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33555.log: /var/log/ceph/ceph-client.admin.42052.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.42052.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34422.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34422.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42907.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47242.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43354.log: /var/log/ceph/ceph-client.admin.42907.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42907.log.gz 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-05-17T22:08:54.564 INFO:teuthology.orchestra.run.smithi083.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41183.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47242.log: /var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50092.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47242.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.41183.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.41183.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36316.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40665.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36316.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50092.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45580.log 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40665.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36316.log.gz 2024-05-17T22:08:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38639.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40665.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50118.log 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34661.log 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50118.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45580.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35305.log 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50170.log 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34661.log: -- replaced with /var/log/ceph/ceph-client.admin.50118.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35305.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34661.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35119.log 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35305.log.gz 2024-05-17T22:08:54.566 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.2.35119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38886.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50170.log.gz 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr: 90.0%/var/log/ceph/ceph-client.admin.38886.log: -- replaced with /var/log/ceph/ceph-client.2.35119.log.gz 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38886.log.gz 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34419.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34427.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34419.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48309.log 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34419.log.gz/var/log/ceph/ceph-client.admin.34427.log: 2024-05-17T22:08:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42457.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34427.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37823.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33851.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37257.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37823.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37823.log.gz 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45300.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39729.log 2024-05-17T22:08:54.568 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2%/var/log/ceph/ceph-client.admin.45300.log: -- replaced with /var/log/ceph/ceph-client.admin.37257.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45300.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47834.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39729.log: /var/log/ceph/ceph-client.admin.42684.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39729.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44457.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2%/var/log/ceph/ceph-client.admin.42658.log: -- replaced with /var/log/ceph/ceph-client.admin.47834.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38594.log 2024-05-17T22:08:54.569 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44457.log: /var/log/ceph/ceph.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44457.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3%/var/log/ceph/ceph-client.admin.47519.log: -- replaced with /var/log/ceph/ceph-client.admin.38594.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50443.log 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43432.log: /var/log/ceph/ceph-client.admin.50443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44508.log 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44843.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50443.log.gz 2024-05-17T22:08:54.570 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37309.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7%/var/log/ceph/ceph-client.admin.37309.log: -- replaced with /var/log/ceph/ceph-client.admin.44508.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47493.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37309.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40746.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39578.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39578.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48044.log 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47493.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37004.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39578.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-05-17T22:08:54.571 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35419.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40895.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8%/var/log/ceph/ceph-client.admin.35419.log: -- replaced with /var/log/ceph/ceph-client.admin.48044.log.gz 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35504.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35419.log.gz 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40895.log.gz 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47069.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4%/var/log/ceph/ceph-client.admin.41748.log: -- replaced with /var/log/ceph/ceph-client.admin.35504.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.46358.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50040.log 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz/var/log/ceph/ceph-client.0.46358.log: 2024-05-17T22:08:54.572 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47069.log.gz 2024-05-17T22:08:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37179.log 2024-05-17T22:08:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44621.log 2024-05-17T22:08:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50040.log: /var/log/ceph/ceph-client.admin.37179.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50040.log.gz 2024-05-17T22:08:54.595 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37179.log.gzgzip 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48700.log 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44621.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.44621.log.gz 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33706.log 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48700.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.46358.log.gz 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33706.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48700.log.gz 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33706.log.gz 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38217.log 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50014.log 2024-05-17T22:08:54.596 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4%/var/log/ceph/ceph-client.admin.38217.log: -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34063.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50014.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50014.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38217.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34221.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50268.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34063.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46707.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9%/var/log/ceph/ceph-client.admin.34221.log: -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43382.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34221.log.gz 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-05-17T22:08:54.597 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46707.log: /var/log/ceph/ceph-client.admin.33825.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46707.log.gz 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36920.log 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33825.log.gz 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40717.log 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43664.log 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36920.log: /var/log/ceph/ceph-client.admin.40717.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36920.log.gz 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40717.log.gz 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41521.log 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-05-17T22:08:54.598 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43664.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44535.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- 90.7% /var/log/ceph/ceph-client.admin.40405.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41521.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46417.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4%/var/log/ceph/ceph-client.admin.40405.log: -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27750.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46417.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46417.log.gz 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35531.log 2024-05-17T22:08:54.599 INFO:teuthology.orchestra.run.smithi083.stderr: 87.1%/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47952.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27750.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35531.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35531.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40282.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46252.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40282.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50339.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47952.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40282.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46252.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46252.log.gz 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45405.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2024-05-17T22:08:54.600 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36121.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45405.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44817.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50469.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49874.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50469.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41689.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50469.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.49874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44074.log 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41689.log.gz 2024-05-17T22:08:54.601 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46012.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39231.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2%/var/log/ceph/ceph-client.admin.46012.log: -- replaced with /var/log/ceph/ceph-client.0.49874.log.gz 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44074.log.gz -- replaced with /var/log/ceph/ceph-client.admin.46012.log.gz 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49373.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49373.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39231.log.gz 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48650.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-05-17T22:08:54.602 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5%/var/log/ceph/ceph-client.admin.45184.log: -- replaced with /var/log/ceph/ceph-client.admin.49373.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48650.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48650.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37361.log 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6%/var/log/ceph/ceph-client.admin.37361.log: -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37361.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4%/var/log/ceph/ceph-client.admin.41798.log: -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2024-05-17T22:08:54.603 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50755.log 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44328.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36898.log 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48922.log 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36898.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48848.log 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50755.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48922.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36898.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48922.log.gz 2024-05-17T22:08:54.604 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48848.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48848.log.gz 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36346.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50625.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.36346.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36346.log.gz 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50625.log.gz 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38717.log 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-05-17T22:08:54.605 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37030.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7%/var/log/ceph/ceph-client.admin.38717.log: -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38717.log.gz 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45053.log 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44566.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-05-17T22:08:54.606 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43116.log 2024-05-17T22:08:54.612 INFO:teuthology.orchestra.run.smithi155.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45053.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45053.log.gz 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43835.log 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43116.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43116.log.gz 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41401.log 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43835.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43835.log.gz 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41401.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41401.log.gz 2024-05-17T22:08:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36528.log 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42002.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36528.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.36528.log.gz 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39003.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50495.log 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45658.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50573.log 2024-05-17T22:08:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50495.log.gz 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39653.log 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50573.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50573.log.gz 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27853.log 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39653.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.39653.log.gz 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46474.log 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27853.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27853.log.gz 2024-05-17T22:08:54.615 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46094.log 2024-05-17T22:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46474.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46474.log.gz 2024-05-17T22:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33985.log 2024-05-17T22:08:54.617 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46094.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46094.log.gz 2024-05-17T22:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39806.log 2024-05-17T22:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33985.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33985.log.gz 2024-05-17T22:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45684.log 2024-05-17T22:08:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39806.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39806.log.gz 2024-05-17T22:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33799.log 2024-05-17T22:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45684.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45684.log.gz 2024-05-17T22:08:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2024-05-17T22:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33799.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33799.log.gz 2024-05-17T22:08:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-05-17T22:08:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44693.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2024-05-17T22:08:54.623 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49768.log 2024-05-17T22:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46680.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-05-17T22:08:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38320.log 2024-05-17T22:08:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49768.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.49768.log.gz 2024-05-17T22:08:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38769.log 2024-05-17T22:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38320.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38320.log.gz 2024-05-17T22:08:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35476.log 2024-05-17T22:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38769.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38769.log.gz 2024-05-17T22:08:54.628 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-05-17T22:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35476.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35476.log.gz 2024-05-17T22:08:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38977.log 2024-05-17T22:08:54.630 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36068.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-05-17T22:08:54.631 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36578.log 2024-05-17T22:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38977.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38977.log.gz 2024-05-17T22:08:54.632 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33959.log 2024-05-17T22:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36578.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36578.log.gz 2024-05-17T22:08:54.633 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38007.log 2024-05-17T22:08:54.634 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33959.log.gz 2024-05-17T22:08:54.635 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39411.log 2024-05-17T22:08:54.638 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.38007.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.38007.log.gz 2024-05-17T22:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47044.log 2024-05-17T22:08:54.639 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39411.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39411.log.gz 2024-05-17T22:08:54.640 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-05-17T22:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47044.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47044.log.gz 2024-05-17T22:08:54.641 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2024-05-17T22:08:54.642 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.50391.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-05-17T22:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34635.log 2024-05-17T22:08:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38743.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2024-05-17T22:08:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-05-17T22:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34635.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34635.log.gz 2024-05-17T22:08:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35898.log 2024-05-17T22:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44598.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-05-17T22:08:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48152.log 2024-05-17T22:08:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35898.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35898.log.gz 2024-05-17T22:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39703.log 2024-05-17T22:08:54.648 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48152.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48152.log.gz 2024-05-17T22:08:54.649 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39335.log 2024-05-17T22:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39703.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39703.log.gz 2024-05-17T22:08:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47107.log 2024-05-17T22:08:54.651 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39335.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39335.log.gz 2024-05-17T22:08:54.652 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-05-17T22:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47107.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47107.log.gz 2024-05-17T22:08:54.653 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44381.log 2024-05-17T22:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-05-17T22:08:54.654 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35024.log 2024-05-17T22:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44381.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44381.log.gz 2024-05-17T22:08:54.656 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49571.log 2024-05-17T22:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35024.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35024.log.gz 2024-05-17T22:08:54.657 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41077.log 2024-05-17T22:08:54.659 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49571.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.49571.log.gz 2024-05-17T22:08:54.660 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42313.log 2024-05-17T22:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41077.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41077.log.gz 2024-05-17T22:08:54.661 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-05-17T22:08:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.42313.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.42313.log.gz 2024-05-17T22:08:54.663 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-05-17T22:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38854.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-05-17T22:08:54.664 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2024-05-17T22:08:54.665 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40431.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-05-17T22:08:54.666 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49116.log 2024-05-17T22:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44126.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2024-05-17T22:08:54.667 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49395.log 2024-05-17T22:08:54.668 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49116.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49116.log.gz 2024-05-17T22:08:54.669 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41571.log 2024-05-17T22:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49395.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.49395.log.gz 2024-05-17T22:08:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43704.log 2024-05-17T22:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41571.log.gz 2024-05-17T22:08:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2024-05-17T22:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43704.log: 91.4% 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43704.log.gz 2024-05-17T22:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-17T22:08:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48620.log 2024-05-17T22:08:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-05-17T22:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44252.log: /var/log/ceph/ceph-client.admin.48620.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2024-05-17T22:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48620.log.gz 2024-05-17T22:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33680.log 2024-05-17T22:08:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-05-17T22:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.37745.log: /var/log/ceph/ceph-client.admin.33680.log: 89.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33680.log.gz 2024-05-17T22:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-05-17T22:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22825.log 2024-05-17T22:08:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2024-05-17T22:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47920.log: /var/log/ceph/ceph.tmp-client.admin.22825.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22825.log.gz 2024-05-17T22:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-05-17T22:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-05-17T22:08:54.678 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-05-17T22:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43302.log: /var/log/ceph/ceph-client.admin.42762.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2024-05-17T22:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-05-17T22:08:54.679 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45483.log 2024-05-17T22:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49420.log 2024-05-17T22:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45483.log: /var/log/ceph/ceph-client.admin.47886.log: 88.9% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-05-17T22:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45483.log.gz 2024-05-17T22:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-05-17T22:08:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47572.log 2024-05-17T22:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49420.log: /var/log/ceph/ceph-client.admin.48126.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-05-17T22:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49420.log.gz 2024-05-17T22:08:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49742.log 2024-05-17T22:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46844.log 2024-05-17T22:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47572.log: /var/log/ceph/ceph-client.admin.49742.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47572.log.gz 2024-05-17T22:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49742.log.gz 2024-05-17T22:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-05-17T22:08:54.683 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27634.log 2024-05-17T22:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46844.log: /var/log/ceph/ceph-client.admin.47313.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46844.log.gz 2024-05-17T22:08:54.684 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36832.log 2024-05-17T22:08:54.685 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-05-17T22:08:54.685 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.27634.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.27634.log.gz 2024-05-17T22:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2024-05-17T22:08:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-05-17T22:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36832.log: /var/log/ceph/ceph-client.admin.43740.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.36832.log.gz 2024-05-17T22:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gz 2024-05-17T22:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-05-17T22:08:54.687 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36424.log 2024-05-17T22:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42998.log: /var/log/ceph/ceph-client.admin.44893.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-05-17T22:08:54.688 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-05-17T22:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40921.log 2024-05-17T22:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-05-17T22:08:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36424.log: /var/log/ceph/ceph-client.admin.40921.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36424.log.gz 2024-05-17T22:08:54.690 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-05-17T22:08:54.690 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40921.log.gz 2024-05-17T22:08:54.690 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39121.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-05-17T22:08:54.690 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2024-05-17T22:08:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39779.log 2024-05-17T22:08:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36978.log: /var/log/ceph/ceph-client.admin.44278.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-05-17T22:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-05-17T22:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2024-05-17T22:08:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39779.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39779.log.gz 2024-05-17T22:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2024-05-17T22:08:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34011.log 2024-05-17T22:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.41439.log: /var/log/ceph/ceph-client.admin.46773.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-05-17T22:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gzgzip 2024-05-17T22:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46146.log 2024-05-17T22:08:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34011.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34011.log.gz 2024-05-17T22:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-05-17T22:08:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2024-05-17T22:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.46146.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46146.log.gz/var/log/ceph/ceph-client.admin.50651.log: 2024-05-17T22:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-05-17T22:08:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-05-17T22:08:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45554.log 2024-05-17T22:08:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45894.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2024-05-17T22:08:54.697 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-05-17T22:08:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37283.log 2024-05-17T22:08:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48881.log 2024-05-17T22:08:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45554.log: /var/log/ceph/ceph-client.admin.37283.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45554.log.gz 2024-05-17T22:08:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43053.log 2024-05-17T22:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37283.log.gz 2024-05-17T22:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48881.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48881.log.gz 2024-05-17T22:08:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45528.log 2024-05-17T22:08:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45352.log 2024-05-17T22:08:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43053.log: /var/log/ceph/ceph-client.admin.45528.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43053.log.gz 2024-05-17T22:08:54.701 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34426.log 2024-05-17T22:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45528.log.gz 2024-05-17T22:08:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45352.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45352.log.gz 2024-05-17T22:08:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48519.log 2024-05-17T22:08:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41627.log 2024-05-17T22:08:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34426.log: /var/log/ceph/ceph-client.0.48519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34426.log.gz 2024-05-17T22:08:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42181.log 2024-05-17T22:08:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.41627.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.41627.log.gz 2024-05-17T22:08:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35561.log 2024-05-17T22:08:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42181.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42181.log.gz 2024-05-17T22:08:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49066.log 2024-05-17T22:08:54.708 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35561.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35561.log.gz 2024-05-17T22:08:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39832.log 2024-05-17T22:08:54.709 INFO:teuthology.orchestra.run.smithi083.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.48519.log.gz 2024-05-17T22:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49066.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49066.log.gz 2024-05-17T22:08:54.710 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35279.log 2024-05-17T22:08:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48756.log 2024-05-17T22:08:54.712 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39832.log: /var/log/ceph/ceph-client.admin.35279.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35279.log.gz 2024-05-17T22:08:54.712 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39832.log.gz 2024-05-17T22:08:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37849.log 2024-05-17T22:08:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34739.log 2024-05-17T22:08:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.48756.log: /var/log/ceph/ceph-client.admin.37849.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37849.log.gz 2024-05-17T22:08:54.715 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39467.log 2024-05-17T22:08:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34739.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34739.log.gz 2024-05-17T22:08:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36775.log 2024-05-17T22:08:54.718 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.39467.log: 94.3% -- replaced with /var/log/ceph/ceph-client.0.48756.log.gz 2024-05-17T22:08:54.719 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.39467.log.gz 2024-05-17T22:08:54.720 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44949.log 2024-05-17T22:08:54.720 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33881.log 2024-05-17T22:08:54.721 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36775.log: /var/log/ceph/ceph-client.0.44949.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36775.log.gz 2024-05-17T22:08:54.721 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43911.log 2024-05-17T22:08:54.722 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33881.log: 92.1% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33881.log.gz 2024-05-17T22:08:54.722 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.0.44949.log.gz 2024-05-17T22:08:54.722 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44787.log 2024-05-17T22:08:54.723 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48593.log 2024-05-17T22:08:54.724 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43911.log: /var/log/ceph/ceph-client.admin.44787.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43911.log.gz 2024-05-17T22:08:54.724 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44787.log.gz 2024-05-17T22:08:54.724 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38516.log 2024-05-17T22:08:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2024-05-17T22:08:54.725 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.48593.log: /var/log/ceph/ceph-client.admin.38516.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48593.log.gz 2024-05-17T22:08:54.725 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38516.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-05-17T22:08:54.725 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2024-05-17T22:08:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-05-17T22:08:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36876.log 2024-05-17T22:08:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.38691.log: /var/log/ceph/ceph-client.admin.33734.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-05-17T22:08:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49957.log 2024-05-17T22:08:54.728 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-05-17T22:08:54.728 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36876.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36876.log.gz 2024-05-17T22:08:54.729 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40947.log 2024-05-17T22:08:54.729 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42814.log 2024-05-17T22:08:54.729 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.49957.log: /var/log/ceph/ceph-client.admin.40947.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49957.log.gz 2024-05-17T22:08:54.730 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-05-17T22:08:54.730 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40947.log.gz 2024-05-17T22:08:54.730 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.42814.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42814.log.gz 2024-05-17T22:08:54.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39147.log 2024-05-17T22:08:54.731 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44761.log 2024-05-17T22:08:54.731 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.39555.log: /var/log/ceph/ceph-client.admin.39147.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-05-17T22:08:54.732 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35392.log 2024-05-17T22:08:54.732 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39147.log.gz 2024-05-17T22:08:54.732 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.44761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44761.log.gz 2024-05-17T22:08:54.733 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2024-05-17T22:08:54.733 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-05-17T22:08:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.35392.log: /var/log/ceph/ceph-client.admin.38347.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35392.log.gz 2024-05-17T22:08:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34167.log 2024-05-17T22:08:54.734 INFO:teuthology.orchestra.run.smithi083.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2024-05-17T22:08:54.734 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45868.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-05-17T22:08:54.735 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34141.log 2024-05-17T22:08:54.735 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43861.log 2024-05-17T22:08:54.736 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34167.log: /var/log/ceph/ceph-client.admin.34141.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34167.log.gz 2024-05-17T22:08:54.736 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34141.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36502.log 2024-05-17T22:08:54.737 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:54.737 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.43861.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.43861.log.gz 2024-05-17T22:08:54.737 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39095.log 2024-05-17T22:08:54.738 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-05-17T22:08:54.738 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36502.log: /var/log/ceph/ceph-client.admin.39095.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36502.log.gz 2024-05-17T22:08:54.739 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33933.log 2024-05-17T22:08:54.739 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39095.log.gz 2024-05-17T22:08:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.47133.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-05-17T22:08:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-05-17T22:08:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34089.log 2024-05-17T22:08:54.740 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.33933.log: /var/log/ceph/ceph-client.admin.49493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33933.log.gz 2024-05-17T22:08:54.741 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36372.log 2024-05-17T22:08:54.741 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-05-17T22:08:54.741 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34089.log.gz 2024-05-17T22:08:54.742 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-05-17T22:08:54.743 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45606.log 2024-05-17T22:08:54.743 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.36372.log: /var/log/ceph/ceph-client.admin.45457.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36372.log.gz 2024-05-17T22:08:54.746 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40178.log 2024-05-17T22:08:54.746 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-05-17T22:08:54.746 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.45606.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45606.log.gz 2024-05-17T22:08:54.746 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48463.log 2024-05-17T22:08:54.747 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.40178.log: /var/log/ceph/ceph-client.admin.48463.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40178.log.gz 2024-05-17T22:08:54.747 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48463.log.gz 2024-05-17T22:08:54.759 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-17T22:08:54.854 INFO:teuthology.orchestra.run.smithi083.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-17T22:08:54.941 INFO:teuthology.orchestra.run.smithi155.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-17T22:08:54.970 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-17T22:08:55.051 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-17T22:08:55.063 INFO:teuthology.orchestra.run.smithi155.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-17T22:08:55.064 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-05-17T22:08:55.064 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m0.692s 2024-05-17T22:08:55.064 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m3.021s 2024-05-17T22:08:55.065 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.126s 2024-05-17T22:08:55.101 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-17T22:08:55.138 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-17T22:08:55.147 INFO:teuthology.orchestra.run.smithi083.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-17T22:08:55.152 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-17T22:08:55.259 INFO:teuthology.orchestra.run.smithi083.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-17T22:08:55.260 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-05-17T22:08:55.261 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m0.898s 2024-05-17T22:08:55.261 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m4.425s 2024-05-17T22:08:55.261 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.371s 2024-05-17T22:08:55.262 INFO:tasks.ceph:Archiving logs... 2024-05-17T22:08:55.262 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi083/log 2024-05-17T22:08:55.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T22:08:55.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-17T22:08:56.175 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi155/log 2024-05-17T22:08:56.176 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-17T22:08:56.561 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-17T22:08:56.574 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-17T22:08:56.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T22:08:56.577 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-17T22:08:56.645 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-17T22:08:56.654 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-17T22:08:56.655 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:08:56.657 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-17T22:08:56.669 INFO:teuthology.orchestra.run.smithi083.stdout: remote refid st t when poll reach delay offset jitter 2024-05-17T22:08:56.669 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================== 2024-05-17T22:08:56.669 INFO:teuthology.orchestra.run.smithi083.stdout:*hv01.front.sepi 67.205.162.81 3 u 51 64 377 0.097 -8.818 3.201 2024-05-17T22:08:56.669 INFO:teuthology.orchestra.run.smithi083.stdout:+hv02.front.sepi 63.231.80.2 3 u 64 64 377 0.085 -1.530 1.687 2024-05-17T22:08:56.670 INFO:teuthology.orchestra.run.smithi083.stdout:+hv03.front.sepi 74.6.168.72 3 u 57 64 377 0.092 -1.318 1.586 2024-05-17T22:08:56.670 INFO:teuthology.orchestra.run.smithi083.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-17T22:08:56.671 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-05-17T22:08:56.671 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-05-17T22:08:56.671 INFO:teuthology.orchestra.run.smithi155.stdout:+hv01.front.sepi 67.205.162.81 3 u 4 64 377 0.082 -6.293 1.502 2024-05-17T22:08:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:+hv02.front.sepi 63.231.80.2 3 u 64 64 377 0.107 -0.146 2.328 2024-05-17T22:08:56.672 INFO:teuthology.orchestra.run.smithi155.stdout:*hv03.front.sepi 74.6.168.72 3 u 59 64 377 0.087 -1.183 1.292 2024-05-17T22:08:56.672 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-05-17T22:08:56.672 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-17T22:08:56.683 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-17T22:08:56.684 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-17T22:08:56.710 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-17T22:08:56.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-17T22:08:56.775 INFO:teuthology.task.internal:Duration was 1345.500687 seconds 2024-05-17T22:08:56.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-17T22:08:56.811 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-17T22:08:56.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T22:08:56.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-17T22:08:56.853 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-17T22:08:56.853 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-05-17T22:08:56.854 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T22:08:56.901 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-05-17T22:08:56.901 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-17T22:08:56.910 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-17T22:08:56.910 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T22:08:56.944 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-17T22:08:56.982 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-17T22:08:56.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T22:08:57.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-17T22:08:57.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-17T22:08:57.102 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-17T22:08:57.102 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T22:08:57.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-17T22:08:57.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-17T22:08:57.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:08:57.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:08:57.164 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-05-17T22:08:57.179 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-05-17T22:08:57.191 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:08:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:08:57.222 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-17T22:08:57.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-17T22:08:57.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-17T22:08:57.246 INFO:teuthology.task.internal:Transferring archived files... 2024-05-17T22:08:57.247 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi083 2024-05-17T22:08:57.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T22:08:57.283 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-17_18:30:44-fs-wip-pdonnell-testing-20240517.012458-debug-distro-default-smithi/7711748/remote/smithi155 2024-05-17T22:08:57.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-17T22:08:57.301 INFO:teuthology.task.internal:Removing archive directory... 2024-05-17T22:08:57.301 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T22:08:57.320 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-17T22:08:57.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-17T22:08:57.351 INFO:teuthology.task.internal:Not uploading archives. 2024-05-17T22:08:57.351 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-17T22:08:57.378 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-17T22:08:57.378 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T22:08:57.380 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-17T22:08:57.384 INFO:teuthology.orchestra.run.smithi083.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 22:08 /home/ubuntu/cephtest 2024-05-17T22:08:57.390 INFO:teuthology.orchestra.run.smithi155.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 22:08 /home/ubuntu/cephtest 2024-05-17T22:08:57.392 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-17T22:08:57.410 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-17T22:08:57.478 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1345.5006868839264 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-17T22:08:57.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-17T22:08:57.546 INFO:teuthology.run:pass